Fixed : Keras 2.x seems to have some breaking changes
Solution is to install Keras 1.2.2 which is last version before 2.x as below
pip install keras==1.2.2
2 Likes
So, despite changing the backend in the keras.json file to point to theano, I get the following error
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_12/MaxPool' (op: 'MaxPool') with input shapes: [?,1,112,128].
while running the code in cell 36
I installed keras 1.2.2, and I am using Python3.5 on Windows 10
1 Like
Hi Peter,
i get the same error, did you manage to resolve it?
Thanks,
Tom
hi! I have troubles running lesson1 notebook …
When it tries to initialize Vgg16 it came back with an error, it says requires h5py (when it tries to load the weights) I downloaded the model properly and install h5py also.
Do you have any suggestions?
Thanks in advance,
Balint
@bejczib, See this comment in another thread
Use the new vgg16.py script from github and also remove the cache for any previously saved models.
Thank you very much for that, Jose! I want to add some color here so anyone else seeing similar scores can learn from my mistakes. While it’s true that bounding probabilities does improve performance, my problem was a typo. Notice in my screenshot above how none of the probabilities were close to 0? Well that should have been my first clue. I had simply typo’d my python code that took the output from vgg.predict.
With the typo, my score was 15.41851. With the typo fixed, my score dropped to 0.20384. Using Jose’s clipping technique, I got all the way down to 0.10064!
Thanks again!
While training in the jupyter notebook, getting invalid unicode characters rendering as boxes in the end, after the accuracy values etc Should I set some encoding in environment variables?
I am running this course in local machine GTX 970 , 8GB RAM, Manjaro Linux
Would setting this up in docker be any better? Anyone using Docker?
I took a little pain setting it up correctly, would love to make it replicable.
In Arch + docker, combine with steps provided in install_gpu.sh (in git repo of the course) :
$pacaur -S nvidia-docker
If you are using bumblebee, load nvidia module (without primusrun or optirun)
$modprobe nvidia
$git clone https://github.com/fastai/courses/tree/master/deeplearning1/nbs
$nvidia-docker run -ti -v /path/to/gitclone/nbs:/nbs nvidia/cuda /bin/bash
In container, (following install_gpu.sh) apt-get update, install anaconda, set PATH, install cunn (CUDA comes with the container image), install/configure keras/theano, configure jupyter without password.
1 Like
Here is something annoying I was struggling with in the last hour and now I solved, I’m posting it here in case someone has the same issue:
In notebook1 we are doing this transformation in the first layer:
vgg_mean = np.array([123.68, 116.779, 103.939]).reshape((3,1,1))
x = (x - vgg_mean)
return x[:, ::-1]
notice that vgg_mean is created with default dtype (float64). In my .theanorc the default dtype is float32:
[global]
device=gpu0
floatX=float32
So unless there’s a casting, you will ran into this error when you add the Dense layer (for some reason the convolutional layers don’t ignore the dtype):
ValueError: Input 0 is incompatible with layer dense_8: expected dtype=float32, found dtype=float64
In our case the casting is done by the ZeroPadding layer. When I tried to code it without it (using border_mode=‘same’ instead) it didn’t work).
Solution:
vgg_mean needs to be created with an explicit dtype:
vgg_mean = np.array([123.68, 116.779, 103.939], dtype=np.float32).reshape((3,1,1))
I’ve finally set the whole thing up to run into my local machine, it seems to be working ok.
The question I have might be silly, and I believe it’s not important at all, but I can’t figure it out. In the following lines of code:
batches = vgg.get_batches(path+‘train’, batch_size=batch_size)
val_batches = vgg.get_batches(path+‘valid’, batch_size=batch_size*2)
Why is the batch_size multiplied by 2 for the validation samples? I tried replicating the code and not multiplying by 2 and it works fine, but I can’t think of a reason to do that right now.
Also, about the second part of the lesson on building the model from scratch with Keras, are we supposed to understand it all right now? I understood quite well the first part using the vgg wrapper, but totally lost the point on the second part with Keras.
Anyway, the course looks great, let’s do it!
I am getting the following error while trying to run the 7 line custom model.
I have installed all the required libraries and configured Theano and Keras properly as well based on the given instruction. I am running this on a local Ubunut 14.04, NVIDIA 759Ti, 16GB, i7 machine.
Please help me with this, as I unable to proceed ahead because of this roadblock.
NotImplementedError Traceback (most recent call last)
in ()
----> 1 vgg = Vgg16()
2 # Grab a few images at a time for training and validation.
3 # NB: They must be in subdirectories named based on their category
4 print(path+‘train’)
5 batches = vgg.get_batches(path+‘train’, batch_size=batch_size)
/home/prsahu/Downloads/courses-master/deeplearning1/nbs/vgg16.pyc in init(self)
30 def init(self):
31 self.FILE_PATH = ‘http://www.platform.ai/models/’
—> 32 self.create()
33 self.get_classes()
34
/home/prsahu/Downloads/courses-master/deeplearning1/nbs/vgg16.pyc in create(self)
65 def create(self):
66 model = self.model = Sequential()
—> 67 model.add(Lambda(vgg_preprocess, input_shape=(3,224,224), output_shape=(3,224,224)))
68
69 self.ConvBlock(2, 64)
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/models.pyc in add(self, layer)
278 else:
279 input_dtype = None
–> 280 layer.create_input_layer(batch_input_shape, input_dtype)
281
282 if len(layer.inbound_nodes) != 1:
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/engine/topology.pyc in create_input_layer(self, batch_input_shape, input_dtype, name)
368 # and create the node connecting the current layer
369 # to the input layer we just created.
–> 370 self(x)
371
372 def assert_input_compatibility(self, input):
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/engine/topology.pyc in call(self, x, mask)
512 if inbound_layers:
513 # this will call layer.build() if necessary
–> 514 self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
515 input_added = True
516
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/engine/topology.pyc in add_inbound_node(self, inbound_layers, node_indices, tensor_indices)
570 # creating the node automatically updates self.inbound_nodes
571 # as well as outbound_nodes on inbound layers.
–> 572 Node.create_node(self, inbound_layers, node_indices, tensor_indices)
573
574 def get_output_shape_for(self, input_shape):
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/engine/topology.pyc in create_node(cls, outbound_layer, inbound_layers, node_indices, tensor_indices)
147
148 if len(input_tensors) == 1:
–> 149 output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
150 output_masks = to_list(outbound_layer.compute_mask(input_tensors[0], input_masks[0]))
151 # TODO: try to auto-infer shape if exception is raised by get_output_shape_for
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/keras/layers/core.pyc in call(self, x, mask)
554 if ‘mask’ in arg_spec.args:
555 arguments[‘mask’] = mask
–> 556 return self.function(x, **arguments)
557
558 def get_config(self):
/home/prsahu/Downloads/courses-master/deeplearning1/nbs/vgg16.pyc in vgg_preprocess(x)
21 def vgg_preprocess(x):
22 x = x - vgg_mean
—> 23 return x[:, ::-1] # reverse axis rgb->bgr
24
25
/home/prsahu/anaconda2/envs/fastAI/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.pyc in _SliceHelper(tensor, slice_spec)
305 if s.step not in (None, 1):
306 raise NotImplementedError(
–> 307 “Steps other than 1 are not currently supported”)
308 start = s.start if s.start is not None else 0
309 if start < 0:
NotImplementedError: Steps other than 1 are not currently supported
Why is the batch_size multiplied by 2 for the validation samples? I tried replicating the code and not multiplying by 2 and it works fine, but I can’t think of a reason to do that right now.
“Because it doesn’t need backprop, so needs less memory.” - Jermey
Increasing batch size while training will require lots of memory, but during validation we can validate more images with same memory.
No, how to use Keras directly is explained in the second week video.
This is a bug in the keras printing invalid characters for status bar. @jeremy
https://github.com/fchollet/keras/issues/5906
To fix this temporarily, in vgg16.py, we have to change verbosity level for fit_generator function to just log the accuracy data once per epoch. verbose=2
Change
def fit(self, batches, val_batches, nb_epoch=1):
self.model.fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=nb_epoch,
validation_data=val_batches, nb_val_samples=val_batches.nb_sample)
To
def fit(self, batches, val_batches, nb_epoch=1):
self.model.fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=nb_epoch,
validation_data=val_batches, nb_val_samples=val_batches.nb_sample,verbose=2)
1 Like
Thank you very much for your answer!!
Nice thing about this vgg model for classification is that it can be applied for any kind of classification problem as far as you can change the data into image. I applied it on some business data, and converted table aggregated data (each row) to an image. It worked perfectly for classification and outperformed standard ML models.
2 Likes
Whoa!!!, but how?
Isn’t the vgg model trained to recognize shapes in a image?
@jeremy can VGG model be used for transfer learning non image data?
You should be able to find some local relationship about your data. These models are sensitive to local positions/relationship. simple way, put columns of table with close relationships (meaning, date) close to each other … I went into some pre-processing to change table aggregate data into image which should be done wisely …
1 Like
So I have to engineer the feature columns to have spatial relation matching their actual relationships.Sounds intriguing.
Is it really better than other approaches though? And is there any paper relevant to this type of transfer learning?
And finally, how did you encode your data to an image?