@davecg@kelvin Did either of you successfully implement the DEVISE paper? If so, can I look over your shoulder one day at your code to understand how you did it?
Sadly don’t have the hardware setup to run that notebook correctly. Amazon P2 cracks out on even the smaller LSUN dataset for me, much less ImageNet. I meant - have you been able to run this on all of ImageNet? Know that you two have downloaded it to your boxes.
You don’t need the full dataset. You can use the subset of Imagenet that Jeremy posted. It won’t be as accurate but you’ll see how the model works and the results are good enough to illustrate the effect. That dataset should work on a P2.
Per Jeremy’s post today on accuracy - Imagenet accuracy is fine down to about 50% of the dataset, then drops off rapidly - plus seemed like the nature of the classifications changed with more epochs (classified fish in nets, then after more epochs classified the specific species of fish). Wanted to poke at a model trained on full ImageNet data to see other quirks & capabilities.
I was in the middle of training it on all of Imagenet. However I looked at how much time tqdm projected, then looked at all the other things I wanted to get done, and I promptly killed it and trained the model on a sample.
Couldn’t train mine enough to the point where it generated reasonable looking digits. Kept oscillating between very bad and sorta better, not really converging towards better results.