BERT with ULMFiT in FastAi

Thats already a nice finding…

It sounds like you’re keeping focussed on ULMFiT approaches (which is good, I struggle with focus sometimes :stuck_out_tongue:), but if you have time, testing with Ranger could be a super nice addition. Lots of folks here use it as their default for computer vision now, and ULMFiT was inspired by techniques that worked in vision, so…

https://dev.fast.ai/optimizer#ranger

if @LessW2020 (ranger creator) has time he might be able to give you a few pointers…

(ok ok, ranger sales pitch over now I promise :wink: )

1 Like