Keras 2 Released

Yes.

Previous behavior in Keras 1 with samples_per_epoch was automatically taking batch size into account.

I don’t understand why Keras 2 doesn’t implement backward compatibility if the user would still use samples_per_epoch, but the conversion is easy once you know about it.

BTW, I’m using ceil(samples/batch_size) rather than int(samples/batch_size) in my Keras 2 code.

I tried the lessons’ notebooks with Keras 2 (and Python 3.5/3.6) and I noticed that when (samples/batch_size) is not an integer Keras in some cases does not use the “remainder” samples in the last batch (probably due to rounding); using ceil(samples/batch_size) seems to solve this issue.

1 Like

This is due to Python 3.5 switching to floating point division when you divide two integers, e.g.

Python2 >>> 2/3
0
Python3 >>> 2/3
0.666666

There is a new // operator in Python 3 that does the same thing as Python 2, so 2 // 3 == 0 in Python 3. Also note that // is available in Python 2 as well, so your code should still be backwards compatible if you use it, and overall it imho better expresses the intent (it’s immediately obvious you meant to do integer division).

Good to know, thanks (I am a Python basic user indeed). I have found that // is a floor integer division, so perhaps there might be a decision to be made whether to use // or ceil in certain situations.

As an example, with a train dataset of 947 samples and a batch size of 100, using // would give 9 steps per epoch while using ceil would give 10 steps. In the former case I believe Keras will not use the last 47 samples, in the latter it will use all the samples and - but I am not sure even after a quick look at the source code - it will add 53 samples made of 0’s to the last batch to make it a complete 100-sample batch (which sometimes might not be desirable). One solution could be to pick up and duplicate actual samples in order to have a total number of real samples that is a multiple of the batch size.

However, training with shuffling probably makes // the best-simplest solution, since when running several epochs all the samples should eventually be used.

I created a Github repo with my Keras 2/Python 3 notebooks:


It is almost complete, the only missing pieces being statefarm-sample.ipynb and state_farm.ipynb, but I hope to work on those notebooks soon (by the way, my code in the repo is still using “ceil”).
Any corrections/suggestions/comments will be greatly appreciated, as I am definitely in the learning phase.
16 Likes

If you’re using Python 2, you really should import print_function and division from __future__ at the beginning of every script and notebook. This will give you Python 3 like behavior for both.

Using the right print syntax and true division will make your eventual transition easier. Not to mention that you might sometimes forget that 2 / 3 == 1 (or assume it will when your variable is actually a float) and create annoying bugs.

You can always use floor division in either version of Python with “//”.

Thank you for you input. I have just updated a number of notebooks where the “from future” statement was not yet positioned at the beginning. Print syntax and true division should already be fine (I hope!). Just to avoid misunderstanding, my goal is not trying to keep compatibility between Python 2 and Python 3, but only to create a working Python 3 version of the code.
Any thoughts regarding “best practice” approaches when the number of samples is not a multiple of the batch size?

With Keras 2, I think that it keeps track of the number of samples that still need to be processed. If the number of samples is not a multiple of the batch size, just do one more batch to use the remaining samples (hence the use of the ceil() function).

The / in Python 2 is also floor integer division. Sure there are examples when you might want to round up instead, but that is not covered by Python 2 either.

Thank you all for your comments. I have just added to my Github repo the missing statefarm-sample.ipynb and state_farm.ipynb notebooks, so now it should be complete.

2 Likes

Thanks so much!

You are welcome!

Here is a good explanation on how you should change it in vgg16.py: https://stackoverflow.com/questions/43457862/whats-the-difference-between-samples-per-epoch-and-steps-per-epoch-in-fit-g

Great thanks to @eljas and others in the focum, I’ve put together this note, mainly for my own benefit but hopefully will be handy for others here:

Notes on Python 2.x / Keras 1.x to Python 3.x / Keras 2.x transition

Some notes on moving from Python 2.x & Keras 1.x -> to Python 3.x & Keras 2.x. (Note Keras currently supports Python 2.7 to 3.5 only. i.e. Python 3.6 will not work on Keras - yet).

Change accordingly in vgg16.py and utils.py.

Keras 1.x -> Keras 2.x

Keras 1.x:

#1 from keras.layers.convolutional import Convolution2D
#2 from keras.regularizers import l2, activity_l2, l1, activity_l1
#3 from keras.utils.layer_utils import layer_from_config
#4 from keras import backend
#5 Convolution2D
#6 batches.nb_sample
#7 batches.nb_class
#8 model.add(Convolution2D(filters, 3, 3, activation="relu"))
#9 fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=nb_epoch, validation_data=val_batches, nb_val_samples=val_batches.nb_sample)
#10 nb_epoch
#11 self.model.predict_generator(test_batches, test_batches.nb_sample)

Keras 2.x:

#1 from keras.layers.convolutional import Conv2D
#2 from keras.regularizers import l2, l1
#3 from keras.layers import deserialize as layer_from_config
#4 from keras import backend; backend.set_image_dim_ordering('th')
#5 Conv2D
#6 batches.samples
#7 batches.num_class
#8 model.add(Conv2D(filters, (3, 3), activation="relu"))
#9 fit_generator(batches, steps_per_epoch=batches.samples//batches.batch_size, epochs=nb_epoch, validation_steps=val_batches.samples//val_batches.batch_size)
#10 epochs
#11 self.model.predict_generator(test_batches, test_batches.samples//test_batches.batch_size)

Note:

  • #8 goes from: “3, 3” … to “(3, 3)”… i.e. with the brackets.
  • #9 in Keras 1, the progress bar shows total number of training samples processed. In Keras 2, the progress bar shows total number of batches processed (steps_per_epoch). Recall that total batches = total samples // batch size. (I floor it to ensure integer value).
  • #11 in Keras 1, regarding model.predict_generator(), 2nd argument corresponds to number of total samples. In Keras 2, that 2nd argument becomes total number of batches. Recall that total batches = total samples // batch size. (I floor it to ensure integer value).

Note: it looks like Keras 1 talks in “number of samples”. Keras 2 prefers “number of batches”. Beware.

Python 2.x -> Python 3.x

Python 2.x:

#1 import cPickle as pickle
#2 reload()

Python 3.x:

#1 import _pickle as pickle
#2 from importlib import reload; reload()

References:

8 Likes

In case any one wants keras 1.1 docs they can be found here https://faroit.github.io/keras-docs/1.1.1/

So I am trying to create a file for submission for Cats & Dogs redux. My test directory has 12499 images. I modified the test function slightly to comply with keras 2.0 as follows

def test(self, path, batch_size=8):

        test_batches = self.get_batches(path, shuffle=False, batch_size=batch_size, class_mode=None)
        test_batches.nb_sample = test_batches.samples             #RC added to comply with Keras 2.x API
        return test_batches, self.model.predict_generator(test_batches, test_batches.samples//test_batches.batch_size)

As shown in the screenshot below, the length of the np array is 12480 (multiple of the number of batches and batch size), however I need 12499 predictions. Wondering if I make the batch size = 1 and what would be the performance hit in that case.

Friends,

I am facing the following error while running the code mentioned above. Could you please assist me on this?

ValueError: When using a generator for validation data, you must specify a value for validation_steps.

1 Like

Hi eljas,you mean change vgg16.py, where can I insert this code lines to it?
Thank you!

Hi guys , I am trying to do the similar exercise of converting the program into Keras 2 . Thanks @Robi for the great repo .

I thought i can start with Keras 2 application model of Vgg16 and I compared it with the implementation of Vgg16 in the repo .

I found the later has additionally Zero padding to every convolutional layers and Drop out layers to the Linear layers . Any particular reason the implementation has a change .

Also I am still in Lesson 1 so If it will explained later then I will wait for the same. Thanks