Question 1: In lesson 1, I checked the images with the top losses and they seem to be augmented ones than original ones. I have uploaded a few of the top losses here https://ibb.co/0CXhWD6. Is this expected? The course video and colab notebook original have proper images but when I run the notebook I see these images with artifacts as the top losses.
Question 2: In Colab, one epoch training takes 1:52 mins while the videos show 0:28 or so. Is this expected. I have switched to GPU as instructed.
Suggestion for people using n1-highmem-8 on GCP for lesson 1. You can use bs=192 and get the pets training done faster. Otherwise it appears that the GRAM is lying unused (when checked with nvidia-smi)
For lesson 1 homework, besides running a copy of the lesson notebook myself, I tried on my own dataset, largely mimicking the process in lesson2-download.ipynb, which was a lifesaver.
First, I downloaded three sets of images from google images for dog-like things: huskies, malamutes, and wolves.
On initially creating a cnn_learner with a resnet34 based model, and using fit_one_cycle with 4 epochs, I got to a validation error rate of 8.9% (8 out of 90 in a validation set), with the below confusion matrix, which made sense to me since to my eyes wolves look pretty distinct but malamutes seem like slightly poofier versions of huskies (and i wouldn’t be surprised if there was miscategorization in my google image search).
When i went to unfreeze and re-applied fit_one_cycle over 4 epochs with a learning rate from the lr plot before the loss started to skyrocket, the error rate got a bit worse (12%), but the confusion matrix was qualitatively similar.
This seemed pretty good, so I tried another hack using images from google images of paper currency from: (1) Euro, (2) USD, (3) Japanese Yen and (4) UK pound. Google image search produced a few dozen “good” images of the paper currency and then below the fold started showing less ideal results with things like coins a few clip art, etc. Applying the same general process, learning 4 epochs, initially produced a 14% error rate. Then unfreezing, and doing 3 more epochs got me to this confusion matrix (10.5% error rate).
Looking at top losses, a few of them were perhaps explainable as being the wrong type of pictures (only coins for instance), although a few of these misclassifications are a bit disappointing. Its hard to tell visually what features the model is focusing on but is tripping over…perhaps there were patterns in the way the bills were arrange in the pictures, which overshadowed the features of the notes themselves in a dataset of 600 images.
Anyway, this seems to be pretty good for a first hack at things. Do others think it makes sense to move on to Lesson 2 at this point?
Thanks
You’re way beyond where I was able to get following lesson one, so, yes, you are more than prepared for lesson 2!
Hi everyone,
As part of the lesson 1 suggested homework, I was trying use the fastai “Planet_Sample” Dataset. The path includes two sub directories tiled “train” and “labels.csv”.
When trying to use “ImageDataBunch.from_csv” on the path in order to get the data from the csv file, I got the following error - “UserWarning: You are labelling your items with CategoryList.”.
Additionally, when trying to use “ImageDataBunch.from_folder” on the path from the “train” directory, I was successful in displaying a handful of the images using “data.show_batch”, however all of them had the word “train” printed above them rather than a label. So when I inputed “data.classes”, it just returned “[‘train’]”.
Neither option is allowing me to access the image and it’s corresponding label.
I am not exactly sure why I am encountering this issue or how to resolve it, so any help would be appreciated. Thanks!
It turns out that the code is showing error when i am trying to run it through GPU, when i switch the run time type to none it worked out fine.
Can someone tell me where the problem is?
Lesson#1 key takeaways :
Cats and Dogs (http://www.robots.ox.ac.uk/~vgg/publications/2012/parkhi12a/parkhi12a.pdf) has 12 cats + 25 dogs breeds = 37 categories
Fine-grained classification is for categories which are very similar.
A standard image size is 224 pixels.
Centre cropping is really useful to resize images to desired dimensions. It does a combination of cropping, resizing, padding etc.
Images have pixel values in the range [0, 255]
Normalizing images is important (Note : Here its referring to standardization https://stats.stackexchange.com/questions/10289/whats-the-difference-between-normalization-and-standardization) where then each of the 3 RGB channels will have a mean of 0 and std dev of 1 using \frac{x-\mu}{\sigma}
Pre-trained ResNet models on ImageNet with two architectural variants are used : 34 & 50 layers
Transfer Learning is the core behind getting ridiculously high accuracy with a relatively few training examples.
Can use very few examples of images (from new classes) to train a new classifier by using transfer learning
One Cycle Learning (https://sgugger.github.io/the-1cycle-policy.html)
Error rate = 100 - classification accuracy
Neural Style Transfer : For transferring the style of paintings/art to a new image.
Inception model is memory intensive; ResNet works really well for most cases.
Fine tuning -> makes a classifier better incase of fine grained categories.
Why we need to train last layers --more abstract, high level combination of features are captured by the later layers and they might need some additional parameter tuning to improve classification of fine grained classes.
To fine tune things we should not use a large learning rate (alpha) and instead use the LR finder to search for optimal range of values. In general, larger learning rates might not lead to convergence to the local/global optima.
Rule of thumb : add slice(x, y) in max_lr where,
x -> value from LR finder plot before things started getting worse (ideally 10x before such point)
y -> value 10 times smaller than learning rate used in the 1st stage
Hope it helps someone! 
I am trying image classification using resnet34 architecture. I trained the model for 5 initial cycles and then did unfreeze and trained again. My loss spiked magnanimously after unfreezing. I know this behaviour has been discussed before on the forum but my question is more granular. When we unfreeze the network and train, how could weights get worse than before unfreezing?
My understanding is, weights should only improve. What exactly happens to network weights after we unfreeze?
So after this lesson, I worked really hard on gathering a data set of images of Goku and Vegeta from the show Dragon Ball Z. I quickly learned how difficult it is to get a working, good data set of images. My data set now has over images 300 images (on private) on my kaggle account. I did all of the instructions and it felt really good to go from start to finish (gathering to training). There were a lot of errors to overcome. My model is around 90% accurate at telling if the picture is of Goku or Vegeta.
Even though I did these things, there’s still so many things I don’t understand. I don’t really get anything besides how to feed things to these functions. But even then, I don’t understand how many cycles I should be doing, I don’t understand what exactly the learner is and how it’s connected to the model, I don’t understand what to be looking for when fitting or tweaking a model, I don’t really get anything. I know these things will be cleared up in the future, and I’m excited to move onto the next lesson.
Continuing the discussion from 
Deep Learning Lesson 1 Notes:
Hi there- I just finished the first lesson and it was awesome. However, using a p2.xlarge on an ec2 instance following the instructions in the course pages, fitting the model each time was very slow. For example, training the CNN first time for the pets for me took ~5 minutes vs the 2 for Jeremy. The MNIST example took 10 minutes. I wonder if there’s something i’m missing about configuring to process using the GPU?
Thanks in advance
Hi,
Under the “Other data formats” heading in Lesson 1:
When running the following 2 lines:
tfms = get_transforms(do_flip=False)
data = ImageDataBunch.from_folder(path, ds_tfms=tfms, size=26)
I get the following error:
“/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py:451: UserWarning: Your training set is empty.”
(followed by an index error).
I haven’t modified anything, just run the whole notebook top to bottom. Everything above this works fine.
Any help on this would be much appreciated 
I’m running on Paperspace.
Thanks.
Ian
Hi Everyone,
I was trying to build an Image Classifier to classify rugby vs american football images after watching Lesson 1 2019 video.I am getting an error rate of 0.02 but I have the following doubts:
1.I have stuffed around 100 images each of American Football and rugby into a google drive folder and used the Path() function to specify the path of that folder as the path of my images.Is that the right way to specify a dataset?
2.The most confused images(as attached) show that the model predicts rugby less confidently but actually it should have predicted rugby with a higher probability.Can someone please explain exactly what does that mean?
3.interp.most_confused(min_val=2) outputs [ ] .Is that the case for all datasets which have only 2 classes?
4.In the cricket vs baseball example(from the video),plot_with_title() was used but I am currently unable to use that function.Is the function still available?
If anyone could clear these doubts,it would be really helpful.
Thank You!
Sharing my test application for Lesson 1.
I took a database of face expressions http://www.kasrl.org/jaffe.html (213 images of 7 facial expressions (6 basic facial expressions + 1 neutral) posed by 10 Japanese female models)
DI - Disgust
SA - Sadness
AN - Anger
HA - Happy
FE - Fear
SU - Surprise
NE - Neutral
I ran the resnet50, over 50 cycles (I trust something on the fastai library won’t let me overfit. The error rate went from 80% to 30% on 10 cycles, and finally 12% on all 50 cycles. Wow.
then I tried to unfreeze. For the learning rate plot, I chose 1e-5,1e-2
Unfreezing the learning, and doing 20 cycles. It went up from 11% error to 50% error in the 4th cycles, and then slowly down to 4% in the last 3 cycles. I have the strong suspicion this is is still overfitting. It is astonishingly good.
Hi,can you tell me what is meant when interp.plot_top_losses gives results like AN/AN/0.45/0.64?
From what I understood after watching Lesson 1,in this case,The model predicted the expression of the face as Anger with a probability of 45% while it should have predicted the expression of the face as Anger with a probability of 64%.What is the significance of this slight difference and why does such an error occur?
Overfitting would mean that you validation loss gets worse as you continue training. It’s in the nature of fut_one_cycle that the loss gets worse at first, but as long as it’s going down in the end you’re fine.
One thing I noticed: when you unfreeze your model, you have to call lr_find after you call unfreeze(). In the notebook you call it before, which really doesn’t give you the information you need for choosing the learning rate.
ImportError Traceback (most recent call last)
in
----> 1 from fastai.vision import *
2 from fastai.metrics import error_rate
ImportError: No module named ‘fastai’
i am running GCP … i did everything as mentioned in the tutorial post about using GCP. this is such a shame that i am stuck before i can even start.
i feel so shitty…i already feel like giving up…
“conda list” command into my remote computer from GCP shows that fastai is installed
basically everything is installed and the notebook kernel (python3) can not even find fast ai module. what do i do?
@gluttony47 I have also just stared trying to go through the tutorials and I am trying to use gcp as well. I have hit the same issue as you. I also have fastai installed in my conda list. I was wondering if you had heard anything and I wanted the issue to be seen again as well.
