I totally agree with you ! Actually, the article which got me motivated was this one about histopathological data from Nature : https://www.nature.com/articles/s41598-019-50587-1?fbclid=IwAR3-ns4LGBWPOb_AK9sNdPM8X3-7HF1OInobzXPozbjzIHFQYILvF0hE_wM#Abs1
In this article, they do what you said, and study how confidence is an excellent indicator of mislabelled data : misclassified images with low uncertainty were most likely mislabelled.
Yeah about that learn.predict_with_mc_dropout I noticed when working on it that someone had sent a PR, which you can find here PR: Ability to use dropout at prediction time (Monte Carlo Dropout)
I will implement a parallelized version and also post the code, so don’t worry about this little piece of code ^^