Keras uses tensorflow by default, and you need to configure to use theano instead. If you’re using the official scripts for this course (setup_p2.sh) this is done for you by default. If not, here are the official docs for how to change it: https://keras.io/backend/
Just as a note, here are the changes I had to do on utils.py to get the lesson 1 working Keras 2 on Windows (likely same changes required on Linux):
utils.py
#import cPickle as pickle
import six.moves.cPickle as pickle
#from keras.regularizers import l2, activity_l2, l1, activity_l1
from keras.regularizers import l2, l1
#from keras.utils.layer_utils import layer_from_config
from keras.layers import deserialize as layer_from_config
Additionally I had to change set the image dim ordering via code, changing image_dim_ordering in keras.json didn’t seem to work (changing backend worked though). Make sure to do it as early as possible, personally I’m doing right after the %matplotlib line
from keras import backend backend.set_image_dim_ordering('th')
Also, nb_class seems to have been renamed to num_class and nb_sample to samples, so when running Vgg16.fit, they need to aliased (or probably better to just modify vg16.py directly):
batches = vgg.get_batches(path+'train', batch_size=batch_size) val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2) batches.nb_class = batches.num_class batches.nb_sample = batches.samples val_batches.nb_class = batches.num_class val_batches.nb_sample = val_batches.samples vgg.finetune(batches) vgg.fit(batches, val_batches, nb_epoch=1)
35 Likes
Another pitfall with Keras 2:
Looks like the steps_per_epoch parameter in fit_generator is NOT the same as samples_per_epoch.
It counts each batch as 1 instead of incrementing by your batch size. I just accidentally ran my models x32 times.
5 Likes
Ouch! Good find.
The image_dim_ordering setting has been replaced with image_data_format. By default this is set to “channels_last”, see https://keras.io/backend/#kerasjson-details. This means that in keras.json you need to replace "image_dim_ordering": "th" with "image_data_format": "channels_first".
The documentation isn’t very clear on this, but keras/backend/common.py confirms it:
def image_dim_ordering():
"""Legacy getter for `image_data_format`.
"""
if _IMAGE_DATA_FORMAT == 'channels_first':
return 'th'
else:
return 'tf'
Is this what you meant?
#was:
#self.model.fit_generator(batches, samples_per_epoch=batches.samples, nb_epoch=nb_epoch,
# validation_data=val_batches, nb_val_samples=val_batches.samples)
# see https://github.com/fchollet/keras/wiki/Keras-2.0-release-notes:
# and: https://keras.io/models/sequential/#sequential-model-methods
# steps_per_epoch:
# Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting
# the next epoch. It should typically be equal to the number of unique samples of your dataset divided by the batch
# size.
self.model.fit_generator(batches,
steps_per_epoch=int(batches.samples/batches.batch_size),
epochs=nb_epoch,
validation_data=val_batches,
validation_steps=int(val_batches.samples/val_batches.batch_size))
2 Likes
Yes.
Previous behavior in Keras 1 with samples_per_epoch was automatically taking batch size into account.
I don’t understand why Keras 2 doesn’t implement backward compatibility if the user would still use samples_per_epoch, but the conversion is easy once you know about it.
BTW, I’m using ceil(samples/batch_size) rather than int(samples/batch_size) in my Keras 2 code.
I tried the lessons’ notebooks with Keras 2 (and Python 3.5/3.6) and I noticed that when (samples/batch_size) is not an integer Keras in some cases does not use the “remainder” samples in the last batch (probably due to rounding); using ceil(samples/batch_size) seems to solve this issue.
1 Like
This is due to Python 3.5 switching to floating point division when you divide two integers, e.g.
Python2 >>> 2/3
0
Python3 >>> 2/3
0.666666
There is a new // operator in Python 3 that does the same thing as Python 2, so 2 // 3 == 0 in Python 3. Also note that // is available in Python 2 as well, so your code should still be backwards compatible if you use it, and overall it imho better expresses the intent (it’s immediately obvious you meant to do integer division).
Good to know, thanks (I am a Python basic user indeed). I have found that // is a floor integer division, so perhaps there might be a decision to be made whether to use // or ceil in certain situations.
As an example, with a train dataset of 947 samples and a batch size of 100, using // would give 9 steps per epoch while using ceil would give 10 steps. In the former case I believe Keras will not use the last 47 samples, in the latter it will use all the samples and - but I am not sure even after a quick look at the source code - it will add 53 samples made of 0’s to the last batch to make it a complete 100-sample batch (which sometimes might not be desirable). One solution could be to pick up and duplicate actual samples in order to have a total number of real samples that is a multiple of the batch size.
However, training with shuffling probably makes // the best-simplest solution, since when running several epochs all the samples should eventually be used.
I created a Github repo with my Keras 2/Python 3 notebooks:
It is almost complete, the only missing pieces being statefarm-sample.ipynb and state_farm.ipynb, but I hope to work on those notebooks soon (by the way, my code in the repo is still using “ceil”).
Any corrections/suggestions/comments will be greatly appreciated, as I am definitely in the learning phase.
16 Likes
If you’re using Python 2, you really should import print_function and division from __future__ at the beginning of every script and notebook. This will give you Python 3 like behavior for both.
Using the right print syntax and true division will make your eventual transition easier. Not to mention that you might sometimes forget that 2 / 3 == 1 (or assume it will when your variable is actually a float) and create annoying bugs.
You can always use floor division in either version of Python with “//”.
Thank you for you input. I have just updated a number of notebooks where the “from future” statement was not yet positioned at the beginning. Print syntax and true division should already be fine (I hope!). Just to avoid misunderstanding, my goal is not trying to keep compatibility between Python 2 and Python 3, but only to create a working Python 3 version of the code.
Any thoughts regarding “best practice” approaches when the number of samples is not a multiple of the batch size?
With Keras 2, I think that it keeps track of the number of samples that still need to be processed. If the number of samples is not a multiple of the batch size, just do one more batch to use the remaining samples (hence the use of the ceil() function).
The / in Python 2 is also floor integer division. Sure there are examples when you might want to round up instead, but that is not covered by Python 2 either.
Thank you all for your comments. I have just added to my Github repo the missing statefarm-sample.ipynb and state_farm.ipynb notebooks, so now it should be complete.
2 Likes
Thanks so much!
You are welcome!
Here is a good explanation on how you should change it in vgg16.py: https://stackoverflow.com/questions/43457862/whats-the-difference-between-samples-per-epoch-and-steps-per-epoch-in-fit-g