Intuitively we could imagine turning the sound of the cough into spectrograms and use them to feed a network.
What are your thoughts ?
2 Likes
Hi csbrd hope all is well, great post!
I was wondering myself how they may have done it.
Looks like they are using this sort of approach
From this extract from their paper:
Methods: We developed an AI speech processing framework that leverages acoustic biomarker feature extractors to pre-screen for COVID-19 from cough recordings, and provide a personalized patient saliency map to longitudinally monitor patients in real- time, non-invasively, and at essentially zero variable cost. Cough recordings are transformed with Mel Frequency Cepstral Coefficient and inputted into a Convolutional Neural Network (CNN) based architecture made up of one Poisson biomarker layer and 3 pre-trained ResNet50’s in parallel, outputting a binary pre- screening diagnostic. Our CNN-based models have been trained on 4256 subjects and tested on the remaining 1064 subjects of our dataset. Transfer learning was used to learn biomarker features on larger datasets, previously successfully tested in our Lab on Alzheimer’s, which significantly improves the COVID-19 discrimination accuracy of our architecture.
It looks like they are using an approach similar to what you describe to covert the audio recordings into something suitable for a CNN.
Extracting Mel-Frequency Cepstral Coefficients with Python
If I ever get through the back log of projects I have I will give it a try.
Cheers mrfabulous1 
The hard part would be to acquire a sufficient amount of data.
Maybe we could develop a simple web app and ask doctors working in an hospital to record coughing and breathing of trusted positive and negative cases.
It could be a nice community project :
1- Web app to gather data
2- Training on 1000+ cases
3- Distribute self-testing application
4- Everybody can test him/herself early, and isolate to protect others or go to the hospital before it’s too late.
Also a first try could be to implement a simpler project,
For example, determine sex with only breathing and coughing sounds.
I may give it a try soon, I’m currently trying to export Fastai v2 models to ONNX format.
3 Likes