Google is working on AI that can hear signs of sickness

In partnership with

In a groundbreaking move, Google has partnered with an Indian AI startup to unveil a bioacoustics healthcare model that can detect diseases by analyzing human sounds. This innovative technology, which blends biology and acoustics, marks a significant leap in using generative AI to transform health diagnostics.

The Daily Newsletter for Intellectually Curious Readers

• We scour 100+ sources daily

• Read by CEOs, scientists, business owners and more

• 3.5 million subscribers

Sign up today!

Understanding Bioacoustics AI

Bioacoustics AI is a fusion of biology and sound science, allowing us to extract crucial health insights from sounds produced by the human body. Leveraging generative AI, like the technology behind ChatGPT, Google has developed a model that uses sound signals to predict early signs of diseases. This development opens up enormous possibilities, especially in areas with limited access to traditional diagnostic tools.

Google’s AI model, known as HeAR (Health Acoustic Representations), uses a vast dataset of 300 million audio samples, including coughs, sneezes, sniffles, and breathing sounds collected from various sources like YouTube and hospitals in Zambia. These audio clips help the AI learn to identify subtle differences in sounds that may indicate respiratory illnesses such as tuberculosis (TB).

 Tackling Tuberculosis with Sound

Tuberculosis remains the world’s deadliest infectious disease, claiming nearly 4,500 lives daily and infecting 30,000 more, according to the World Health Organization. In India alone, the disease causes approximately 250,000 deaths annually. Early detection is critical to controlling its spread, but many cases go undiagnosed due to a lack of accessible and affordable diagnostic tools.

The HeAR AI model can detect early signs of TB by analyzing cough patterns with a high degree of accuracy. This AI-powered tool, easily integrated into smartphones, can be deployed in remote locations, making disease screening more accessible and cost-effective. Google’s collaboration with Salcit Technologies, a Hyderabad-based AI startup specializing in respiratory health, has further enhanced the accuracy of TB diagnosis and lung health assessments through their AI tool, Swaasa.

Swaasa: Bringing Diagnostics to the Masses

Swaasa, derived from the Sanskrit word for breath, is an AI model developed by Salcit Technologies that works alongside Google’s HeAR model. It allows users to upload a 10-second cough sample via a mobile app to test for respiratory diseases with an impressive 94% accuracy rate. The screening test costs around 200 rupees ($2.40), significantly cheaper than traditional diagnostic tests like spirometry, which can cost up to 3,000 rupees.

Leading healthcare providers in India, such as Apollo Hospitals and the Healing Fields Foundation, are already using Swaasa to screen for TB in remote areas. With approval from India’s medical device regulator, Swaasa represents a milestone as the first software tool deployed as a medical device in India.

Challenges and Future Potential

While this technology is promising, it faces challenges in gaining acceptance among healthcare professionals and users. Changing clinical practices can be difficult, and there are concerns about the quality of audio samples, particularly in noisy environments. Additionally, rural users unfamiliar with technology might struggle to record cough samples accurately. Despite these hurdles, the technology is gaining support from organizations like the StopTB Partnership, which aims to eradicate TB by 2030.

Beyond TB, Google is exploring other bioacoustics applications, such as using AI to detect early breast cancer through ultrasound at the Chang Gung Memorial Hospital in Taiwan. This model aims to provide free breast cancer screenings globally, especially for populations unable to afford expensive mammograms.

The Future of Sound-Based AI in Healthcare

Sound-based generative AI systems hold great potential for democratizing healthcare by making early disease detection more accessible, affordable, and scalable. Companies like Montreal-based Ubenwa are developing AI models to analyze infant cries to understand their needs and health conditions. Others are working on AI tools to detect autism by analyzing vocalizations like oohs, aahs, and gurgles.

As voice and sound emerge as the new frontiers in medicine, the possibilities for AI in healthcare are expanding rapidly. With continuous advancements, AI-powered bioacoustics could revolutionize the way we diagnose and manage health conditions, making high-quality healthcare accessible to everyone, regardless of their location or financial status.