It might be very helpful if we can bump PyTorch version to 0.4. Its not a bug fix, but will greatly help folks to continue to use the Fast.AI Library along with the latest version of PyTorch until we get to the V3 of the course in the fall.
There have certainly been a lot of additions, some of which you may not have anticipated at the beginning, and it’s true that a big clean-up of the library will be most helpful. Especially if there are new libraries or with the new versions of pytorch.
Let us know if you need help for some parts of it!
1 Like
I’d certainly like to do that - at the moment some of the AWD LSTM stuff isn’t working however (the embedding dropout at the very least - possibly more). If anyone is open to diagnosing and fixing AWD LSTM on 0.4 then I’d really appreciate it! I think that’s all that’s stopping us from doing a version bump.
A new branch sounds like really good plan to iterate faster. But are you sure that you want to start from scratch, instead of doing a “fine-tuning” on the whole library? We can set a larger “learning rate” following the transfer learning analogy and more freely throw-out redundant pieces, like parts of learning rate scheduler, or the way we initialize transforms etc.
Here are advantages of going this way:
- every so often we will have a working library so we are more likely to have something of good quality on the deadline.
- we will have a direction (keep the current API unless you can figure out a simplification),
- we can pragmatically rewrite only parts of the library,
- it will be easier for relative newbies (like me) to contribute.
Anyway I’d like to help, no-matter if you decide to start from scratch or try more iterative process.
How about we get a space on the forum to keep the communication and discussion around the fastai v2 and to grab some contributors to help?
Re.“AWD LSTM” I can have a look as I’m working on the polish language model, and I would like to invest a bit more time in the NLP part, and maybe try to use the CNN with Attention as I’ve read it has better performance than the RNNs.
@jeremy how are the plans for rewriting of fast.ai? I’m asking so I don’t miss the party, not that I don’t have other things to do, I’m still on the Polish language model :).
1 Like
Haven’t even started yet! Honestly been trying to catch up on sleep, email, etc which all got rather behind in the last big push.
1 Like
So getting published is really as hard as they say… I’m off for holidays & weeding in mid June. I’ll drop you a line in July regarding rewrite so that I can get included.
1 Like
What is the current status of Nvidia DALI w.r.t fastai v1.0? Is it being used? Or is its use still pending?
All the transformations are being done using pytorch api so I’m guessing they are run on GPU, See the pipeline described here::
https://docs.fast.ai/vision.image.html#Fastai-internal-pipeline