Hi guys, there’s a typo in this wiki: http://wiki.fast.ai/index.php/Lesson_1
make a copy of the lesson 1 notebook and use the new copy to draw in the new Dogs Vs. Cats data (if you copy the notebook outside of the course folder, don’t forget the vgg26.py files etc)
I believe here, vgg26.py should really be vgg16.py. I don’t have the privilege to modify it, so I have to post it here.
Spend many hours over the past day or two struggling to get a submission in shape for Kaggle. Starting to feel like my python skills aren’t sufficient for this course. The people on this forum seem to breeze through things that take me hours and hours to figure out. For now I will continue, but was tempted many times to give up.
My final submission was generated thanks to a lot of helpful code from @Matthew. My final score on kaggle was 1.08402, which seems to be pretty bad in the rankings.
I’m going to go back through the code I wrote up now, using this as a model for how to try to improve. Feeling very drained / dispirited. Maybe if I take a bit of a break from things, then I’ll return and try the same workflow / codeflow on a different data set.
Still have very little idea of what’s going on in terms of how this all works / how finetuning reshapes the model etc. Anyway, just posting here for posterity’s sake. Will keep chugging along.
I wanted to post some resources I found useful while struggling through putting together the dogs vs cats .csv file. Perhaps they’ll be useful for someone else.
So I have my own DL server with 8GB GeForce GTX1080 installed. While running Lesson 1 training set (the whole thing), I got an outofmemory exception. Is it common to run out of memory @ 8gb with that size of a training set? Should I be worried about using a bigger dataset in the future?
I had figured that initially I would be okay with just 1 GPU
I think you can work with bigger dataset but you will need to use small batch size to avoid memory issues. I also have the same Nvidia video board and even using not big dataset but batch size of 128,256 or bigger, I get the out of memory issue. So, I need stay around 32, 64 batch size.
smaller batch will make it longer to train, I assume?
So to be clear, finetuning pretty much takes the result from batches = vgg.get_batches(path+‘train’, batch_size=batch_size) where it says (Found 22500 images belonging to 2 classes.) and then it filters my dataset into those 2 classes?
Also, what exactly does vgg.fit do? Why do we need the val_batches? Does that help make the accuracy percentage?
Hello,
Could someone help me? I tried to reproduce the results on the lesson 1. The only changes is I use tensorflow with batchsize = 8. The accuaracy I can get only is around 0.91. Is that normal?
Thanks.
Hi there,
I’m trying to do the assignment by creating and training the network myself, just to check understanding. However my network does not converge (stuck on 0.5 accuracy forever). Would anyone be able to give any hints on why?
I’ve tested it on mnist and aside from being overkill, it works fine. I’ve tried with putting the pixel data in [0, 1] and [-1,1], doesn’t make any difference.
Any help would be greatly appreciated!
Model is below. I’m running it on my own machine, with tensorflow-gpu backend and tf dimension ordering.
BATCH_SIZE = 32 # 8
IMG_SIZE = 56 # 224
N_CHANNELS = 3
N_OUTPUTS = 2
IMG_SHAPE = (IMG_SIZE, IMG_SIZE, N_CHANNELS)
train_datagen = image.ImageDataGenerator()
train_generator = train_datagen.flow_from_directory(
'data\\train',
target_size=(IMG_SIZE,IMG_SIZE), class_mode='categorical',
batch_size=BATCH_SIZE, shuffle=True)
valid_datagen = image.ImageDataGenerator()
valid_generator = valid_datagen.flow_from_directory(
data\\valid',
target_size=(IMG_SIZE,IMG_SIZE), class_mode='categorical',
batch_size=BATCH_SIZE, shuffle=True)
model = Sequential()
model.add(ZeroPadding2D(1, input_shape=IMG_SHAPE))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(ZeroPadding2D(1))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(2, 2))
model.add(ZeroPadding2D(1))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(ZeroPadding2D(1))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(MaxPooling2D(2, 2))
model.add(ZeroPadding2D(1))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(ZeroPadding2D(1))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(ZeroPadding2D(1))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(MaxPooling2D(2, 2))
model.add(Flatten())
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(N_OUTPUTS, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.fit_generator(
train_generator,
steps_per_epoch=int(np.ceil(train_generator.samples/BATCH_SIZE)),
epochs=40,
validation_data=valid_generator,
validation_steps=int(np.ceil(valid_generator.samples/BATCH_SIZE)) )Hi @why_no_https,
The model should work, but there might be a problem with Learning Rate or setting the shuffle=True on the validation set.
Try using a smaller rate (0.001 or smaller), and setting Shuffle=False in the validation set.
If it does not work, upload the copy of your notebook on gist-Github, I will try to figure out the problem
I’m in the lesson1.ipynb.
I’m at the cell that is 'Downloading data from http://files.fast.ai/models/vgg16.h5’
and was wondering how to modify vgg16py so that it grabs vgg16.h5 and future files like it from my local disk. Because I already have them on my local disk.
I figured this was a useful problem to know how to solve, eliminating redundancy and all, but after many hours of changing code, searching the web, and trying to put the file where I think the downloaded file would be (like in the root folder env of the notebook), no luck.
It seems to be grabbing through a script in Keras but I can’t figure out what exactly is calling it to do this.
In vgg16py: I commented out line 46 and merged it with line 57,
then I tried to get line 140 to look at the local dir alone within model.load_weights()
Hi @irshaduetian thanks so much for responding!
I tried a smaller and a larger learning rate, as well as turning off shuffle, augmenting the data, removing dropout (to deliberately try and overfit to at least get some convergence) and a few other things.
My hypothesis is that either there’s just way too little data, or my weights are initialized in a really bad way. But I really don’t know!
Here is my Jupyter notebook: https://www.dropbox.com/sh/u3p21a8hhi7kwr8/AADXLRkjedxe-bsowbXZvvjXa?dl=0
Note that the data folders don’t contain any of the images, because that’s too much to download. Just unzip the cats and dogs archive and things should work. I use tensorflow backend and dimension ordering, but I can always modify it to use theano.
I really appreciate your help, thank you again!
Your assumption is right, VGG model is too big to learn good features from the small amount of data (cats and dogs) in a few epochs. Unfortunately, I didn’t have an environment to run your notebook otherwise I would have tested it myself.
My recommendation would be to run this model on the Mnist dataset and see if you can get a good accuracy out of that in around 30 epochs.
You can consult the mnist.ipynb in fast.ai course repo.
Let me know if you need further help
@irshaduetian I tried it on MNIST (I had to remove some conv2d layers since the inputs are smaller) and was able to get high accuracy after less than 1 epoch! However I’m worried there’s some problem with my network design or how I’m loading data, since I get the exact same results for every epoch. So each epoch ends with identical error and accuracy metrics - I would have thought it at least changes randomly a bit!
I’m currently downloading all of image-net to try training myself - although it seems like it might take a few months to download at the moment 
Hello Haroun,
I’m looking to go through the lessons without the p2 instance because I have a machine that matches your specs.
Would you be willing to help me setup the configuration the lesson 1 Jupyter notebook.
I’m getting really confused, but I really wanna learn machine learning!!
Anything would help, thank you and good luck in future endeavors!
If you really wanna Machine Learning then you are in the wrong thread : stuck_out_tongue:
Here is the new course about Machine Learning Another treat! Early access to Intro To Machine Learning videos
Thank you Irshad 
super excited to start this over the next few days!!
How can i copy the files on my mac into crestle?
