@jeremy I know the current deep learning course offering is only available to current students, but I was wondering if you could make an exception?
I am a PhD student at the University of New Mexico (UNM) and have a milestone project coming due at the end of November (and oral presentation the following week). I am using the ULMFit framework to show how transfer learning can be applied to a small (<2000 labeled examples) medical text dataset. I am getting good results, accuracy has gone up from 84% (previous approach) to 92% with ULMFit. It would be really great if you could send me the link to the lecture on IMDb for V3 of the deep learning course. Watching your lecture would help me get the most out of the fast.ai library and greatly help in preparing my oral presentation.
If you can make an exception you can send me the link at:
Thanks for the offer, but it sounds like ULMFit hasn’t been talked about that much in the class so far. I will just look at previously published material.
Yes, Jeremy didn’t talk much about that, I think because it will be covered in much deeper in the advanced part of the course (maybe the next version of part 2).
I am one of the contributor to the Language Model Zoo for Malay language:
You can learn more about ULMFiT in lesson 4 (basic) and lesson 10 (deep dive into AWD-LSTM network, language modelling, fine-tuning, and more).
If you prefer to read, you can check my lesson 10 notes. The exact part where Jeremy started to talked about:
NLP
fastai.text module
IMDB language modelling with fastai.text API
pre-training on Wikipedia dataset
fine-tuning
… and so on and so forth
Although my work is done for ULMFiT using fastai v0.7.0 (the older version), it does not ends there. If you prefer to use fastai v1 for your work, here’s a summary of what’s going on (not trying to be accurate). You can join and follow along.
Many of us are still working to get our ULMFiT language notebooks updated to fastai v1:
@Esteban as @cedric pointed out we are trying to create a multilingual version of ulmfit ported to v1.0, the script are working but the accuracy is inferior to the previous work of Jeremy. I’m looking into that since some time and I think it is due to different tokenization we used (to follow xnli task). So if you are after high accuracy stay with fastai 0.7 for the time being. Once we have the issues fixed in v1.0 I will everyone know in Multilingual ULMFiT so simply start watching this channel.
On the other-hand if you want to help out with the tuning a multilingual version of ulmfit for fastai v1.0 let me know.