Part 2 blogs

adrian · June 24, 2018, 1:33pm

Hi All,

Ive written a couple of blog posts on dropout and it’s effect on RNNs.

part 1 is a high level overview of dropout from Hinton et al. 2012’s paper through to last year.

Part 2 shows results of experiments on dropout parameter variation for the fastai language modelling and translation tasks, as well as for Merity et al.'s awd-lstm-lm.

Any feedback is welcome. I note that fixes were made to weight drop by @sgugger since i initially ran these experiments.

Also im not clear on why results for weight drop where wdrop is >0.7 were so different between fastai and awd-lstm-lm. The code for WeightDrop looks almost identical. I will follow this up when i can.

prajjwal1 · June 24, 2018, 1:59pm

Written a technical article on Wasserstein GANs for Intel Dev Zone
https://software.intel.com/en-us/articles/better-generative-modelling-through-wasserstein-gans

divyansh · July 2, 2018, 2:55pm

Have a read at my recent blog on self-attention GANs.

miguel_perez · September 19, 2018, 12:46pm

It’s been a while since I don’t post but had this one in my mind since part2 lessons, glad that summer brougth me the time,

Even · September 20, 2018, 4:34am

These articles, the first one in particular are masterful. Thank you for sharing.

lkhphuc · October 30, 2018, 6:23am

Hi all,
I wrote a small blog post explaining the point of “Gan as a loss function” that Jeremy mentioned in lession 12.
If anyone can give me some feedbacks it would be great.

DavidBressler · November 14, 2018, 11:16pm

Hey everyone, I’ve just written my first ever blog posts! Part 1 is an explanation of the adaptive softmax, which is up to 10x faster than the full softmax, and Part 2 is a walk through a Pytorch implementation.

This is the first time I’ve ever shared code publicly, so I’d really appreciate any feedback on the blog posts and the github code:

In Part 2 I give a shout-out to fast.ai and @jeremy 's course, which were instrumental in getting me to where I am now

github.com

DavidWBressler/adaptivesoftmax/blob/master/Adaptive_softmax_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### This is a step by step walk through a Pytorch implementation of the adaptive softmax from Grave et al.'s 2017 paper, using Pytorch’s built-in AdaptiveLogSoftmaxWithLoss function. I use the wikitext-2 dataset. Check out my blog posts for further explanation:\n",
    "https://towardsdatascience.com/speed-up-your-deep-learning-language-model-up-to-1000-with-the-adaptive-softmax-part-1-e7cc1f89fcc9\n",
    "\n",
    "https://towardsdatascience.com/speed-up-your-deep-learning-language-model-up-to-1000-with-the-adaptive-softmax-part-2-pytorch-d47fe9a56152"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download and preprocess data"
   ]
  },
  {

This file has been truncated. show original

Thanks much in advance for any helpful feedback or advice!

DavidBressler · December 11, 2018, 11:57pm

Check out my latest blog post! I use fast.ai tools to build a convolutional neural network (CNN) for natural language processing (NLP): https://towardsdatascience.com/how-to-build-a-gated-convolutional-neural-network-gcnn-for-natural-language-processing-nlp-5ba3ee730bfb . I appreciate any feedback… Thanks!

melll · July 3, 2019, 12:35pm

Hey guys, I just wrote a blog post explaining SSD (single shot object detector) that Jeremy addresses in lesson 9 of 2018 course.

I would appreciate your feedback. Thanks in advance.

tapashettisr · July 8, 2019, 5:41pm

Is the notebook for SSD still accessible??

immarried · July 17, 2019, 9:48am

Just recently completed a draft for a blog post looking at the various sorts of boxes in an SSD. Would really appreciate any comments on it: https://medium.com/@jackchungchiehyu/94d8b0cf5c16 thanks!