Lesson 1 discussion

Part 1 (2017)
347

redux I believe: https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition. Data is on the left hand side. You must click on at least one file from the web browser before the kaggle CLI will work.

348

Thanks. I was able to get that working.

Any idea how the data was rearranged or modified? I notice in the lesson1 notebook that there are paths of data/dogscats and data/dogscats/sample, but I’m not seeing that in the files downloaded from Kaggle, nor do I see instructions on what files to move around or what not for that.

349

Yes, you need to put data into 5 subdirectories like this:

data/train/dogs <— put dogs here
data/train/cats <— put cats here
data/valid/dogs <— move (not copy) some percentage of data/train/dogs here
data/train/cats <— move (not copy) some percentage of data/train/cats here
data/test/unknown <— put test data here

It’s up to you what percentage of your training data into validation. Maybe 20%?.

-Caleb

1 Like
350

Oh yes, and the ‘sample’ directory is just a copy of same 5 directories, but with many fewer files so it will run quickly when you’re experimenting.

351

I had the same doubt. I ended up downloading the files from Kaggle and then creating the folder structure. But then you can avoid that; just download the file from here.(http://www.platform.ai/data/dogscats.zip)

2 Likes
352

Thanks. That is awesome…

353

Thanks for the link to the Dogs and Cats Redux notebook, Matthew. Very helpful.

354

I am not sure if this question was already answered but…
How do I know how many layers I need to add to my model? And which type of layers and in which order?
No sure if that sounds crazy :slight_smile:

355

You will get a better idea when you go to the next lessons. I will tell few tricks that I learnt from the class.

  1. For images that look similar to Imagenet you need not train the Convolutional layers. Since most of them can be reused.

  2. If you want to build a model from scratch , then start with a single dense layer and try experimenting by adding more nodes or layers.

Hope it helps.

356

Exactly the issue I was looking to fix–thank you for the solution!

357

As Gelu74 said, this is because the exercise uses Theano, and you’re using Tensorflow. I had the same error because I gave up on trying to install Theano on my Windows 7 configuration with a GTX960 GPU, and decided to use Anaconda Python 3.5 and Tensorflow instead.

The easy way to overcome this problem is to edit the keras.json file as such:

{
“image_dim_ordering”: “th”,
“epsilon”: 1e-07,
“floatx”: “float32”,
“backend”: “tensorflow”
}

You can find this file in your .keras directory.

I hope this helps!
Christina

358

Thanks, I had this same issue. This solution worked for me. :slight_smile:

359

Hi @jeremy Is there any way to change code in lesson one( stead of .keras json) so that it works in tensorflow backend? I am using keras in container i cloned on cloud 9 and have no access to .keras folder

360

Thanks for the response. I’m pretty new to these concepts. Can you explain further what you mean by ‘pop the last dense layer’? Any help would be appreciated.

361

Thank you for the response. I was able to get the first few sections of the notebook working for my purposes. However, the first line in ‘Generate Predictions’ is giving me trouble. I posted the error below. Any suggestions? I have image files contained in the test_path location, but am returning 0 images from 0 classes. Thanks.

Generate Predictions
Let’s use our new model to make predictions on the test dataset
In [53]:

batches, preds = vgg.test(test_path, batch_size = batch_size*2)
Found 0 images belonging to 0 classes.

Exception in thread Thread-24:
Traceback (most recent call last):
File “/home/ubuntu/anaconda2/lib/python2.7/threading.py”, line 801, in __bootstrap_inner
self.run()
File “/home/ubuntu/anaconda2/lib/python2.7/threading.py”, line 754, in run
self.__target(*self.__args, **self.__kwargs)
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.py”, line 433, in data_generator_task
generator_output = next(generator)
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/preprocessing/image.py”, line 596, in next
index_array, current_index, current_batch_size = next(self.index_generator)
File “/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/preprocessing/image.py”, line 444, in _flow_index
current_index = (self.batch_index * batch_size) % N
ZeroDivisionError: integer division or modulo by zero

1 Like
362

what is the directory structure of the test folder as pointed to by test_path

363

when you do vgg.finetune(batches) you are actually calling this function in the vgg16.py file:
def finetune(self, batches):
model = self.model
model.pop()
for layer in model.layers: layer.trainable=False
model.add(Dense(batches.nb_class, activation=‘softmax’))
self.compile()

basically it removes the last layer of the Imagenet model that is intended to classify 1000 categories and adds a new one to classify your two categories. it also sets all the layers (except the last new one) to not trainable

1 Like
364

Directory structure: ~/nbs/courses/deeplearning1/nbs/data/redux/test/unknown/

365

are there any files inside that folder?

366

Hi @maciej. I believe Tensorflow installation and configuration will covered in part 2 of the class that starts on Feb 27th and then that material will be released as a MOOC afterwards.

1 Like