Cough-counting is harder than it sounds, but AI and smartphones can help

2023-04-20

Everyone was a bit on edge about coughs when Hyfe Chief Product Officer Paul Rieger and his co-founders launched their company in early 2020. Hyfe has developed AI-powered acoustic software models to not only detect and track the number and frequency of coughs, but also to identify likely disease diagnoses and analyze patterns to warn of viral outbreaks. The software-as-a-medical device (SaMD) developer has partnered with pharma giant Merck & Co. Merck is developing a drug for chronic cough patients, some of whom might cough up to 1,000 times in a single day. “Cough is the most common symptom why people seek healthcare, but it is also the one that is not monitored,” Rieger said in an interview with Medical Design & Outsourcing. “You can measure your own blood pressure, your temperature, your oxygen saturation, but how do you measure your cough? It’s not possible. Especially in your sleep, nobody knows — your spouse may know — but it’s not accurate.” What coughs can tell us Hyfe’s algorithms run on smartphones, whose sensors are always listening for coughs. Detecting and time-stamping coughs can find patterns that could indicate changes in an individual patient’s condition. In a larger population, those patterns could sound the alarm of an outbreak of flu or a new virus like SARS-CoV-2, offering extra time for early intervention. “It’s less about a single cough .. and more about the patterns you can derive from continuous monitoring,” Rieger said. “We see ourselves as a company that focuses on the continuous data stream, and then you derive continuous insights from that data stream.” Hyfe is looking into a host of conditions, including chronic cough, refractory chronic cough, gastroesophageal reflux disease (GERD), tuberculosis, chronic obstructive pulmonary disease (COPD) and asthma. “We see the biggest interest especially on chronic cough because there’s no treatment out there right now. That obviously is a big pain point,” Rieger said. “And people have a hard time convincing their healthcare providers — try to go to the doctor and tell them you’re coughing 1,000 times a day.” The sound of a cough isn’t enough to diagnose COVID-19, which doesn’t even present as a cough in all patients. “We are a bit skeptical when it comes to these diagnostic use cases where one cough equals one diagnosis,” he said. “But we have a very positive outlook on the longitudinal one, where you can actually see disease patterns over time if you monitor for long enough and especially if you combine that with other data streams: oxygen saturation, heart rate and so on.” Hyfe is looking at the different properties of a cough such as duration or whether it’s productive, but it all starts with reliably identifying a single cough — and from there, identifying coughing bouts. “The biggest challenge is actually to have clean timestamps initially, and then from those timestamps you can derive things such as intensity of a coughing episode, who is coughing at which times and what things are happening at that time during the day,” Rieger said. GERD patients will cough around the time when they eat, for example, while nervous-tic-based chronic coughs can be identified by nighttime relief. “They will stop coughing the moment they fall into deep sleep, but somebody who has refractory chronic cough will not,” Rieger said. “In just the diurnal pattern of a cougher, you can see really interesting stuff.” Sensors everywhere One of the trends Hyfe is betting on is the commoditization of hardware, something many device developers might overlook. Smartphones are increasingly ubiquitous with a growing array of onboard sensors, powerful processors and smarter software. “It’s just not that easy for people to wrap their heads around that because usually you think a medical device is one sensor, one device, and then you ship it,” Rieger said. “But I think there’s a big opportunity to use those existing sensors and add more capabilities on top of them instead of trying to reinvent that wheel, going bottom up with a new device and so on. That is the approach we’re taking, but I think other companies will look at it similarly just because of the cost-effectiveness in go-to-market.” The ability to collect all that data presents its own challenge. Hyfe has collected more than 600 million data points so far. “But data can also be noise,” Rieger said. “Making sense of the data is one of the primary challenges in any AI approach, especially if you’re focused on a wide spectrum of devices.” There are tens of thousands of different smartphone hardware combinations, with many, many different microphones of various quality collecting data. “You have high-end and low-end phones,” Rieger said. “Working with that data, you need to start normalizing it and you need to have really good labeling pipelines. That has been one of the keys as to why we’ve been able to successfully do this at scale.” Developing algorithms for the real world Rieger distilled the process of building medical AI for sound detection into five high-level steps: Collect diverse sounds from the real world. Label sounds using trained professionals. Train a deep learning algorithm on these labeled sounds. Implement the algorithm on the target platform (smartphone, medical device, etc.). Validate the algorithm in clinical studies. Focusing on data, cleaning it and labeling it is essential for developing algorithms that will work wherever patients and their phones are in the real world. “It’s easy to design a really good algorithm that works in a lab setting — in a hospital, for example, that’s really easy. It’s a very clean environment, you know what the acoustic environment will look like,” Rieger said. “But the real world is noisy. It’s very noisy. People have all kinds of different environments, so you need a lot of samples. You need not just millions, you need at least tens of millions of samples and you need to make sense of these samples. That has definitely been the biggest challenge.” Hyfe’s using a deep learning model. Deep learning models benefit from more data, but if it’s not clean data, performance deteriorates. Deep learning models are also generalizable, making it possible to run Hyfe’s software on all kinds of platforms spanning proprietary hardware to smartphones, smartwatches and computers. “That’s only possible because of that big data set, cleaning it, and then training models on the highly accurate and cleaned subset,” Rieger said. Labeling data The importance of labeling data is one of the biggest lessons Hyfe’s team has learned. The average cough is between 300 and 500 milliseconds, but what happens in the moments before and after the cough is key for determining whether it was a single cough or a coughing episode. While it might be more efficient to record, review and label half-second-long audio snippets, longer chunks offer more context and quality. “For us, the context matters a lot and we’ve realized that makes a lot more sense to focus on continuous labeling and not explosive labeling,” Rieger said. It’s also important to consider how to build pipelines for labeling and how to work with inter-label correlation when labelers disagree, making sure that only data where people agree on labels is included in the training set. “We have had a lot of iterations where we realized something was wrong in the recent batch of labeling,” Rieger said. “It really messes up your performance and it costs a lot of time, it costs a lot of money.” What’s next for Hyfe Hyfe is conducting its human factors study and hoping for FDA authorization at the end of 2023 or the beginning of 2024. Cough counters currently used in clinical studies are sound recorders, and the FDA classifies them as such. But for fully-automated cough detection, Hyfe will need de novo authorization from the FDA. “There are lots of predicates for lung sounds, but the lung sound is very different from cough,” Rieger said. “There are also predicates for wheezing, but wheezing is also different from cough. We have done our rounds with the regulator and there’s no predicate for cough detection.” Asked what kind of products Hyfe might developer next, Rieger wouldn’t say much. “The journey is from detection to insights to treatment,” Rieger said. “On the treatment side, there’s quite a few interesting things that we can do.” Related: SaMD development lessons from Cordio’s voice AI heart failure app