Share your work here ✅

Part 1 (2019)
1483

Hello everyone

I recently wrote a medium article on the integration of Fastai with BERT (huggingface’s pretrained pytorch models for NLP) on a multi-label text classification task. After that I compared the performances of BERT and ULMFiT.

Here are few things which I did to integrate Fastai with BERT:

  1. Using BERT’s tokens and vocab
  2. Some modifications in BERT’s tokens for eos and bos parts
  3. Splitting the model for discriminative learning techniques

Here is the link:

I was amazed by the level of accuracy using just 2 epochs:

  1. BERT - 98% accuracy
  2. ULMFiT - 97% accuracy

I would be glad if you have any feedbacks or comments on this.

BR
Abhik

10 Likes