That’s where the kaggle competition data for submission is.
Hello Everybody,
First of all, I would like to thank Rachel and Jeremy for creating this fantastic course and this knowledge hub.
I am trying to improve my performance in the Dogs vs. Cats problem. For that, I thought It would be a good idea to, not only training the last layer, but training a bigger part of the network.
I have tried several approaches:
- Train the whole network (CONV + FC)
- Train only the fully connected block (FC)
- Train the last 4096 FC layer
and the more layers I make trainable, the worse is the error (both, train and validation) the net achieves. With the approaches above, I get a validation accuracy of 0.5, 0.5 and 0.96 respectively. If I only train the last layer (as Jeremy does in the course) I get 0.986. My idea behind it is that the model would fine-tune the weights of the original VGG yielding a better solution, but this logic seems to be wrong and I would like to know the reason behind it…
Thank you in advance,
Iván
Hello Mr. Howard, Ms. Thomas, and fast.ai members!
Thank you for setting up this wonderful community to help us learn about Deep Learning!
I noticed that http://www.platform.ai/files/ is no longer available and I cannot find [dogscats.zip] on https://github.com/fastai/courses.
Where I can I download the [dogscats.zip] data set now that the page is no longer available?
Thanks!
you can find it on the lesson 1 page! at the bottom of ‘necessary files’ http://wiki.fast.ai/index.php/Lesson_1_Notes
Wow, didn’t expect such a quick reply!
ecase (Elizabeth), you’re a life saver! Thank you very much, 
@james_goldfarb I have the same problem as @martin, I would like to not have to recompute the model for the cats and dogs redux data, but instead use the model already computed in the lesson1 notebook.
I tried
vgg = Vgg16()
vgg.load_model(path+‘Data/dogscats/models/dogscats.h5’)
it says “Vgg16 instance has no attribute ‘load_model’”
I also tried
vgg=model.load_model(path+‘Data/dogscats/models/dogscats.h5’)
it says “NameError: name ‘model’ is not defined”
I also tried
vgg=load_model(path+‘Data/dogscats/models/dogscats.h5’)
it says “NameError: name ‘load_model’ is not defined”
@martin Did you solve the problem since?
@Ptilulu You need to understand a few concepts to do this.
If you look in vgg16.py you can search for model and weights.
“model” is your convolutional neural network.
The Model is defined:
model = self.model = Sequential()
model.add(Lambda(vgg_preprocess, input_shape=(3,224,224)))
self.ConvBlock(2, 64)
self.ConvBlock(2, 128)
self.ConvBlock(3, 256)
self.ConvBlock(3, 512)
self.ConvBlock(3, 512).........................
Next step is to train the model. This yields the model “weights”. Jeremy is using a pretrained model. The weights are saved in fname = ‘vgg16.h5’. We do not see this, but at some point in the past someone saved these weights using model.save_weights(‘vgg16.h5’).
He then loads the weights with
model.load_weights(get_file(fname, self.FILE_PATH+fname, cache_subdir=‘models’))
You can do the same in lesson1.py.
You can train your model with
vgg.fit(batches, val_batches, nb_epoch=1)
and then save the weight with
vgg.save_weights('myfile.h5')
If you want to train further at another time, use
vgg = Vgg16()
vgg.load_weights(‘myfile.h5’)
vgg.fit(batches, val_batches, nb_epoch=1)
In practice, this is used all the time.
We train the model, save the weights and then to use the model (which is the purpose of all this work).
We load the weights and then model.predict() to actually use the model and predict whether an image is a cat or dog.
During model training it is also very useful to save the weights (and the loss as well as other metrics ) for each epoch, since overfitting (lower accuracy) can occur with too much training.
model.save(fname) saves more than the CNN. See the keras faq.
https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model
start with model.save_weights and model.load_weights.
Use can be seen in vgg16.py.
For datasets that are less similar to imagenet (i.e. the dataset used for the pretrained weights) you’ll need to retrain more layers. Explaining and understanding this is the basis of much of this course. Hopefully by the time you’re done with lesson 7 you’ll have a good understanding of this issue - but trying to explain it now in a brief forum reply may be getting ahead of ourselves! 
3 Likes
@ivallesp Imagenet has a much bigger dataset from which convolution layers have extracted enough structure and spatial information. So you can leave that part, @jeremy’s lectures do cover training the fully connected block and was able to achieve higher accuracies. This is useful as it helps you to try different dropout values in the fully connected block, learning rate and optimizers, etc.
Here is a comment from Andrej Karpathy on similar topic of using convnets in practice.
In practice: use whatever works best on ImageNet. If you’re feeling a bit of a fatigue in thinking about the architectural decisions, you’ll be pleased to know that in 90% or more of applications you should not have to worry about these. I like to summarize this point as “don’t be a hero”: Instead of rolling your own architecture for a problem, you should look at whatever architecture currently works best on ImageNet, download a pretrained model and finetune it on your data. You should rarely ever have to train a ConvNet from scratch or design one from scratch. I also made this point at the Deep Learning school.
Here is the link to the above comment. http://cs231n.github.io/convolutional-networks/
Hope this helps.
Best Regards,
Guru Medasani
3 Likes
Why is the 3rd element in the tuple produced by vgg.predict(imgs, True) indexed by the FULL set of categories instead of the constrained set of simply dogs and cats?
Is there a way to change this?
@wgpubs You can execute vgg.classes = ["cat", "dog"] to make the class names correspond to this dataset.
3 Likes
Very helpful! I figured there was some way … the lecture made it seem like it was inferred somehow.
Here’s how I found out how to change the classes:
- I looked inside
vgg.predictbecause that was the method of interest. Its definition is found invgg16.py. - I saw that the
classesvariable was based onself.classes. - I noticed that
__init__calls theget_classesmethod. - The
get_classesmethod downloads a json with the ImageNet class names and uses it to setself.classes.
I had to look at an example from vgg.predict to know whether “cat” or “dog” should be first in vgg.classes.
4 Likes
Hi all,
Does anyone know where I can find the homework assignments for each lesson?
I looked at the end of the Lesson 1 video, but I didn’t hear mention of the homework in the video itself (Maybe I wasn’t paying close enough attention?)
Thank you!
You can find them on the lesson wiki pages.
For example, http://wiki.fast.ai/index.php/Lesson_1#Overview_of_homework_assignment
2 Likes
Thank you, that wiki seems like a great resource! I’ll take a look at the overview.
Much appreciated.
Is there a video and/or discussion about the last part of the Lesson 1 notebook?
There is a section on building your own model but nothing in the Lesson 1 video, notes, or wiki. I’m digging through this part trying to get a real handle on how to code a model myself.
Thanks
One more question. Working on the Dogs and Cats redux. When I run the notebook on all the data i get a large error message as seen below. However if i just run it on the sample data it works fine. So either i have the directory structure incorrect or something else I dont understand is going on. I believe I setup the directory correct. I followed the layout as in the dogs and cats redux notebook, and ran the scripts in that note book to move and create the data and associated folders. Thanks ahead of time.
Found 23000 images belonging to 6 classes.
Found 2000 images belonging to 2 classes.
Epoch 1/1
22976/23000 [============================>.] - ETA: 0s - loss: 0.1479 - acc: 0.9626
Exception Traceback (most recent call last)
in ()
5 val_batches = vgg.get_batches(path+‘valid’, batch_size=batch_size*2)
6 vgg.finetune(batches)
----> 7 vgg.fit(batches, val_batches, nb_epoch=1)
/home/ubuntu/nbs/lesson1/vgg16.pyc in fit(self, batches, val_batches, nb_epoch)
115 def fit(self, batches, val_batches, nb_epoch=1):
116 self.model.fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=nb_epoch,
–> 117 validation_data=val_batches, nb_val_samples=val_batches.nb_sample)
118
119
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/models.pyc in fit_generator(self, generator, samples_per_epoch, nb_epoch, verbose, callbacks, validation_data, nb_val_samples, class_weight, max_q_size, nb_worker, pickle_safe, **kwargs)
872 max_q_size=max_q_size,
873 nb_worker=nb_worker,
–> 874 pickle_safe=pickle_safe)
875
876 def evaluate_generator(self, generator, val_samples, max_q_size=10, nb_worker=1, pickle_safe=False, **kwargs):
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in fit_generator(self, generator, samples_per_epoch, nb_epoch, verbose, callbacks, validation_data, nb_val_samples, class_weight, max_q_size, nb_worker, pickle_safe)
1469 val_outs = self.evaluate_generator(validation_data,
1470 nb_val_samples,
-> 1471 max_q_size=max_q_size)
1472 else:
1473 # no need for try/except because
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in evaluate_generator(self, generator, val_samples, max_q_size, nb_worker, pickle_safe)
1552 'or (x, y). Found: ’ + str(generator_output))
1553 try:
-> 1554 outs = self.test_on_batch(x, y, sample_weight=sample_weight)
1555 except:
1556 _stop.set()
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in test_on_batch(self, x, y, sample_weight)
1251 x, y, sample_weights = self._standardize_user_data(x, y,
1252 sample_weight=sample_weight,
-> 1253 check_batch_dim=True)
1254 if self.uses_learning_phase and type(K.learning_phase()) is not int:
1255 ins = x + y + sample_weights + [0.]
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_dim, batch_size)
963 output_shapes,
964 check_batch_dim=False,
–> 965 exception_prefix=‘model target’)
966 sample_weights = standardize_sample_weights(sample_weight,
967 self.output_names)
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in standardize_input_data(data, names, shapes, check_batch_dim, exception_prefix)
106 ’ to have shape ’ + str(shapes[i]) +
107 ’ but got array with shape ’ +
–> 108 str(array.shape))
109 return arrays
110
Exception: Error when checking model target: expected dense_13 to have shape (None, 6) but got array with shape (128, 2)
@cmeff1Cats and Dogs competition should have 2 classes - cats and dogs.
Found 23000 images belonging to 6 classes.
This looks incorrect. I believe you’ll need to check your training directory structure. It should be more on the lines of your validation set - Found 2000 images belonging to 2 classes.
The number of classes is the number of folders in your train or valid directory.
2 Likes
Gotcha I’ll check that out. Thanks for the feedback.