redux I believe: https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition. Data is on the left hand side. You must click on at least one file from the web browser before the kaggle CLI will work.
Thanks. I was able to get that working.
Any idea how the data was rearranged or modified? I notice in the lesson1 notebook that there are paths of data/dogscats and data/dogscats/sample, but I’m not seeing that in the files downloaded from Kaggle, nor do I see instructions on what files to move around or what not for that.
Yes, you need to put data into 5 subdirectories like this:
data/train/dogs <— put dogs here
data/train/cats <— put cats here
data/valid/dogs <— move (not copy) some percentage of data/train/dogs here
data/train/cats <— move (not copy) some percentage of data/train/cats here
data/test/unknown <— put test data here
It’s up to you what percentage of your training data into validation. Maybe 20%?.
Oh yes, and the ‘sample’ directory is just a copy of same 5 directories, but with many fewer files so it will run quickly when you’re experimenting.
I had the same doubt. I ended up downloading the files from Kaggle and then creating the folder structure. But then you can avoid that; just download the file from here.(http://www.platform.ai/data/dogscats.zip)
Thanks. That is awesome…
Thanks for the link to the Dogs and Cats Redux notebook, Matthew. Very helpful.
I am not sure if this question was already answered but…
How do I know how many layers I need to add to my model? And which type of layers and in which order?
No sure if that sounds crazy
You will get a better idea when you go to the next lessons. I will tell few tricks that I learnt from the class.
For images that look similar to Imagenet you need not train the Convolutional layers. Since most of them can be reused.
If you want to build a model from scratch , then start with a single dense layer and try experimenting by adding more nodes or layers.
Hope it helps.
Exactly the issue I was looking to fix–thank you for the solution!
As Gelu74 said, this is because the exercise uses Theano, and you’re using Tensorflow. I had the same error because I gave up on trying to install Theano on my Windows 7 configuration with a GTX960 GPU, and decided to use Anaconda Python 3.5 and Tensorflow instead.
The easy way to overcome this problem is to edit the keras.json file as such:
You can find this file in your .keras directory.
I hope this helps!
Thanks, I had this same issue. This solution worked for me.
Hi @jeremy Is there any way to change code in lesson one( stead of .keras json) so that it works in tensorflow backend? I am using keras in container i cloned on cloud 9 and have no access to .keras folder
Thanks for the response. I’m pretty new to these concepts. Can you explain further what you mean by ‘pop the last dense layer’? Any help would be appreciated.
Thank you for the response. I was able to get the first few sections of the notebook working for my purposes. However, the first line in ‘Generate Predictions’ is giving me trouble. I posted the error below. Any suggestions? I have image files contained in the test_path location, but am returning 0 images from 0 classes. Thanks.
Let’s use our new model to make predictions on the test dataset
batches, preds = vgg.test(test_path, batch_size = batch_size*2)
Found 0 images belonging to 0 classes.
Exception in thread Thread-24:
Traceback (most recent call last):
File “/home/ubuntu/anaconda2/lib/python2.7/threading.py”, line 801, in __bootstrap_inner
File “/home/ubuntu/anaconda2/lib/python2.7/threading.py”, line 754, in run
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.py”, line 433, in data_generator_task
generator_output = next(generator)
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/preprocessing/image.py”, line 596, in next
index_array, current_index, current_batch_size = next(self.index_generator)
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/preprocessing/image.py”, line 444, in _flow_index
current_index = (self.batch_index * batch_size) % N
ZeroDivisionError: integer division or modulo by zero
what is the directory structure of the test folder as pointed to by test_path
when you do vgg.finetune(batches) you are actually calling this function in the vgg16.py file:
def finetune(self, batches):
model = self.model
for layer in model.layers: layer.trainable=False
basically it removes the last layer of the Imagenet model that is intended to classify 1000 categories and adds a new one to classify your two categories. it also sets all the layers (except the last new one) to not trainable
Directory structure: ~/nbs/courses/deeplearning1/nbs/data/redux/test/unknown/
are there any files inside that folder?