Ive written a couple of blog posts on dropout and it’s effect on RNNs.
part 1 is a high level overview of dropout from Hinton et al. 2012’s paper through to last year.
Part 2 shows results of experiments on dropout parameter variation for the fastai language modelling and translation tasks, as well as for Merity et al.'s awd-lstm-lm.
Any feedback is welcome. I note that fixes were made to weight drop by @sgugger since i initially ran these experiments.
Also im not clear on why results for weight drop where wdrop is >0.7 were so different between fastai and awd-lstm-lm. The code for WeightDrop looks almost identical. I will follow this up when i can.
Hi all,
I wrote a small blog post explaining the point of “Gan as a loss function” that Jeremy mentioned in lession 12.
If anyone can give me some feedbacks it would be great.
Hey everyone, I’ve just written my first ever blog posts! Part 1 is an explanation of the adaptive softmax, which is up to 10x faster than the full softmax, and Part 2 is a walk through a Pytorch implementation.
This is the first time I’ve ever shared code publicly, so I’d really appreciate any feedback on the blog posts and the github code:
In Part 2 I give a shout-out to fast.ai and @jeremy 's course, which were instrumental in getting me to where I am now
Thanks much in advance for any helpful feedback or advice!
Just recently completed a draft for a blog post looking at the various sorts of boxes in an SSD. Would really appreciate any comments on it: https://medium.com/@jackchungchiehyu/94d8b0cf5c16 thanks!