Wow this is just terrific, once again! You have a real knack for technical writing. I noticed a couple of little issues FYI:
The second way to go about it, and, in fact, the easiest to implement, is to approximate the derivative with the following formula we know from calculus:
I don’t see a formula here.
The most fast method for calculating would be analytically find the derivative for each neural network architecture.
This is what backprop does, right? Otherwise - I’m not sure what distinction you’re making. Backprop simply calculates the derivative using the chain rule. I think the more important thing to mention here is that some libraries like Pytorch can calculate the analytical derivative for arbitrary Python code, so you don’t have to worry about doing it yourself.
Some of them are described in my other post ‘Improving the way we work with learning rate’.
You should link to your post here. (In general, I think your article to use a few more hyperlinks, BTW.)
Finally, I spotted at least one spelling mistake, so perhaps run a spell-checker over it?
Hi All,
Came up with a simple blog post on Embeddings. Attaching the draft here. Please review the same and let me know your thoughts. Would publish it after changes; if any.
Here is a second blog that builds up on ‘fun with small datasets’ : Is the human wearing glasses or not?
I used a script to download 100’s of images from Google Image search. (135 x 2 training, 20 x 2 validation images). I was able to get 100% accuracy using augmentation and differential learning rates. @jeremy I’ve tried to get this done before class, so let me know if there are some gaps to be addressed
In the same spirit, I wrote a blog on Spiderman Vs Deadpool here. Appreciate any feedback on this! @nikhil I found that some images downloaded by the script were corrupt. I added code to remove these before creating csv.
Just put up another easy to read blog on entity embeddings… I actually wrote this up from the earlier version of the course. Feels good to distill the writing and finally put it as a blog…
The draft of my first medium post is here. Would love to hear your thoughts/comments! In either case, I thank each and everyone of you for inspiring me to overcome my fear of technical writing.
Update : the draft is published after having incorporated some comments.
Hey guys! Thank you everybody here! I’ve been getting incredibly encouraging feedback. I wouldn’t even think of doing it without this community.
Also I wrote another post on optimization using SGD with momentum. which is what used by default in fastai library(or at least was used couple days ago). I’d appreciate any feedback.
Great post. I liked visualizations where can be seen the “inertia/lag” of momentum increasing as it gets bigger, also the one of Nesterov’s momentum… I enjoyed all your post but this one my favourite by now!
Great article! It was nice to see intuitive explanations of the code used to achieve learning rate annealing. I guess these explanations will be the most helpful for the general audience.
This article might serve as a Bible for Kaggle competitions related to Classify Images…(atleast I came to know about many new things, hope community finds it useful)
Hey everyone. fastai community has created tons of materials here: from bash, tmux, running aws, kaggle cli to weights decay, adaptive learning rates articles and further how to put this all together and get some good prediction accuracy. I thought it might be useful for further groups, for Jeremy and for us if we join and together extract these materials into some non-forum and rather book-type structure “Depp learning in fastai” .
