Keras 2 Released


(Johnny Chan) #42

Great thanks to @eljas and others in the focum, I’ve put together this note, mainly for my own benefit but hopefully will be handy for others here:

Notes on Python 2.x / Keras 1.x to Python 3.x / Keras 2.x transition

Some notes on moving from Python 2.x & Keras 1.x -> to Python 3.x & Keras 2.x. (Note Keras currently supports Python 2.7 to 3.5 only. i.e. Python 3.6 will not work on Keras - yet).

Change accordingly in vgg16.py and utils.py.

Keras 1.x -> Keras 2.x

Keras 1.x:

#1 from keras.layers.convolutional import Convolution2D
#2 from keras.regularizers import l2, activity_l2, l1, activity_l1
#3 from keras.utils.layer_utils import layer_from_config
#4 from keras import backend
#5 Convolution2D
#6 batches.nb_sample
#7 batches.nb_class
#8 model.add(Convolution2D(filters, 3, 3, activation="relu"))
#9 fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=nb_epoch, validation_data=val_batches, nb_val_samples=val_batches.nb_sample)
#10 nb_epoch
#11 self.model.predict_generator(test_batches, test_batches.nb_sample)

Keras 2.x:

#1 from keras.layers.convolutional import Conv2D
#2 from keras.regularizers import l2, l1
#3 from keras.layers import deserialize as layer_from_config
#4 from keras import backend; backend.set_image_dim_ordering('th')
#5 Conv2D
#6 batches.samples
#7 batches.num_class
#8 model.add(Conv2D(filters, (3, 3), activation="relu"))
#9 fit_generator(batches, steps_per_epoch=batches.samples//batches.batch_size, epochs=nb_epoch, validation_steps=val_batches.samples//val_batches.batch_size)
#10 epochs
#11 self.model.predict_generator(test_batches, test_batches.samples//test_batches.batch_size)

Note:

  • #8 goes from: “3, 3” … to “(3, 3)”… i.e. with the brackets.
  • #9 in Keras 1, the progress bar shows total number of training samples processed. In Keras 2, the progress bar shows total number of batches processed (steps_per_epoch). Recall that total batches = total samples // batch size. (I floor it to ensure integer value).
  • #11 in Keras 1, regarding model.predict_generator(), 2nd argument corresponds to number of total samples. In Keras 2, that 2nd argument becomes total number of batches. Recall that total batches = total samples // batch size. (I floor it to ensure integer value).

Note: it looks like Keras 1 talks in “number of samples”. Keras 2 prefers “number of batches”. Beware.

Python 2.x -> Python 3.x

Python 2.x:

#1 import cPickle as pickle
#2 reload()

Python 3.x:

#1 import _pickle as pickle
#2 from importlib import reload; reload()

References:


(Aseem Bansal) #43

In case any one wants keras 1.1 docs they can be found here https://faroit.github.io/keras-docs/1.1.1/


(Deep Learner) #44

So I am trying to create a file for submission for Cats & Dogs redux. My test directory has 12499 images. I modified the test function slightly to comply with keras 2.0 as follows

def test(self, path, batch_size=8):

        test_batches = self.get_batches(path, shuffle=False, batch_size=batch_size, class_mode=None)
        test_batches.nb_sample = test_batches.samples             #RC added to comply with Keras 2.x API
        return test_batches, self.model.predict_generator(test_batches, test_batches.samples//test_batches.batch_size)

As shown in the screenshot below, the length of the np array is 12480 (multiple of the number of batches and batch size), however I need 12499 predictions. Wondering if I make the batch size = 1 and what would be the performance hit in that case.


(Jilan) #45

Friends,

I am facing the following error while running the code mentioned above. Could you please assist me on this?

ValueError: When using a generator for validation data, you must specify a value for validation_steps.


Any plan to upgrade notebooks to work with newer APIs?
#46

Hi eljas,you mean change vgg16.py, where can I insert this code lines to it?
Thank you!


(satish) #47

Hi guys , I am trying to do the similar exercise of converting the program into Keras 2 . Thanks @Robi for the great repo .

I thought i can start with Keras 2 application model of Vgg16 and I compared it with the implementation of Vgg16 in the repo .

I found the later has additionally Zero padding to every convolutional layers and Drop out layers to the Linear layers . Any particular reason the implementation has a change .

Also I am still in Lesson 1 so If it will explained later then I will wait for the same. Thanks


(Roberto Castrioto) #48

I just updated my repo which now includes also the “Python 3.5 - Keras 2” adaptation for Part 2 of the course. I hope someone will find it useful!
Thanks in advance to everybody who will have a chance to provide any comments, suggestions or corrections.
For any questions or issues related to the repo I suggest to directly visit its issues section.
https://github.com/roebius/deeplearning_keras2


Examples in lessons 1-7(python 2) in python 3 code somewhere?
(Yuzhou Liu) #49

Hi Robi,

Thanks for sharing the python3/keras2 version of the notebooks. I tried executing lesson1 and am getting the following error when Vgg class is instantiated:

~\AppData\Local\Continuum\Anaconda3\envs\dlwin36\lib\site-packages\tensorflow\python\framework\common_shapes.py in _call_cpp_shape_fn_impl(op, input_tensors_needed, input_tensors_as_shapes_needed, debug_python_shape_fn, require_shape_fn)
674 missing_shape_fn = True
675 else:
–> 676 raise ValueError(err.message)
677
678 if missing_shape_fn:

ValueError: Negative dimension size caused by subtracting 2 from 1 for ‘max_pooling2d_2/MaxPool’ (op: ‘MaxPool’) with input shapes: [?,1,112,128].

Any idea why?


(Roberto Castrioto) #50

It looks like your Keras is set to use TensorFlow instead of Theano (part 1 notebooks use Theano).
One thing to check is the keras.json file in your .keras directory. There is a template of this file for Theano in my repo.
Let me know if this helps!

P.S. I updated the template yesterday


(Yuzhou Liu) #51

oh never mind. I was using tensorflow as backend. After fixing the keras config file this worked fine.


(Yuzhou Liu) #52

Yup that’s what I did . Thansk a lot!


(Soen Surya Soenaryo) #53

Thanks, Elijas!


(Arman Sharafshahi) #54

Hi Robi,
I tried using your updated course file for part 1 lesson 1 and can never get the vgg = Vgg16() to succeed. It keeps giving me this error:


ValueError Traceback (most recent call last)
in ()
----> 1 vgg = Vgg16()
2 # Grab a few images at a time for training and validation.
3 # NB: They must be in subdirectories named based on their category
4
5 #batches = vgg.get_batches(path+‘train’, batch_size=batch_size)

~/deep-learning/fastai/deeplearning_keras2/nbs/vgg16.py in init(self)
30 def init(self):
31 self.FILE_PATH = ‘http://files.fast.ai/models/
—> 32 self.create()
33 self.get_classes()
34

~/deep-learning/fastai/deeplearning_keras2/nbs/vgg16.py in create(self)
73 self.ConvBlock(3, 512)
74
—> 75 model.add(Flatten())
76 self.FCBlock()
77 self.FCBlock()

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/models.py in add(self, layer)
474 output_shapes=[self.outputs[0]._keras_shape])
475 else:
–> 476 output_tensor = layer(self.outputs[0])
477 if isinstance(output_tensor, list):
478 raise TypeError('All layers in a Sequential model ’

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/engine/topology.py in call(self, inputs, **kwargs)
613 # Infering the output shape is only relevant for Theano.
614 if all([s is not None for s in _to_list(input_shape)]):
–> 615 output_shape = self.compute_output_shape(input_shape)
616 else:
617 if isinstance(input_shape, list):

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/layers/core.py in compute_output_shape(self, input_shape)
479 raise ValueError('The shape of the input to “Flatten” '
480 'is not fully defined '
–> 481 '(got ’ + str(input_shape[1:]) + '. '
482 'Make sure to pass a complete “input_shape” '
483 'or “batch_input_shape” argument to the first ’

ValueError: The shape of the input to “Flatten” is not fully defined (got (0, 7, 512). Make sure to pass a complete “input_shape” or “batch_input_shape” argument to the first layer in your model.


(Roberto Castrioto) #55

@asharafshahi
It might be that you are using the TensorFlow backend instead of the Theano backend. Check the content of your keras.json file like I suggested above at Keras 2 Released

Hope this will help!


(Arman Sharafshahi) #57

Thanks for the help, it did turn out to be an issue in the ~/.keras/keras.json file. Here is the correct format that made it work:
{
“epsilon”: 1e-07,
“floatx”: “float32”,
“image_data_format”: “channels_first”,
“backend”: “theano”,
“image_dim_ordering”: “th”
}


(Roberto Castrioto) #58

Happy that the issue was solved. As a side note: with Keras 2 I believe that in the keras.json file only “image_data_format” is required, while “image_dim_ordering” is obsolete.


(Arman Sharafshahi) #59

Well the next issue I seem to have run into is that when I hit the vgg.fit(…) line in lesson 1, it get stuck and after about a minute the kernel dies and restarts. It never shows any output from that line. I’m using an Azure NC6 instance with one K80 GPU and it does seem to be correctly setup to utilize it. When I switch to CPU it does seem to work but it’s so slow it would take hours to complete. :frowning:


(Roberto Castrioto) #60

It could be an insufficient GPU memory issue. You might try a small batch size (like 4) and see what happens when you use the GPU. Finding the maximum batch size that the GPU memory can accomodate often requires some trial-and-error.


(Arman Sharafshahi) #61

I had gone all the way down to a batch size of 1, and I check the GPU and it doesn’t seem to be utilized hardly:

±----------------------------------------------------------------------------+
| NVIDIA-SMI 384.81 Driver Version: 384.81 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00009EE9:00:00.0 Off | 0 |
| N/A 38C P0 70W / 149W | 680MiB / 11439MiB | 0% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 26569 C …n/anaconda3/envs/tensorflow/bin/python3 669MiB |
±----------------------------------------------------------------------------+

There must be something going on in the fit function before it gets to the GPU that’s causing my kernel to die…


(Pierre Guillou) #62

Hi @nolanchan,

I’m trying to setup my Jupyter notebook for lesson 1. I’m using Python 2.7, Anaconda2 and Cygwin (under Windows 10). My pip version is pip 9.0.1

I tried to install manually Keras 1.1.1 by downloading the file and then I typed “pip install Keras-1.1.1.tar.gz” in my terminal. The process started but failed at the end (see below, the displayed text in my terminal). I tried to solve the issue through https://stackoverflow.com/posts/36025294/revisions but does not work.
Do you have any idea about that ? Thanks.

$ pip install keras-1.1.1.tar.gz
Processing ./keras-1.1.1.tar.gz
Collecting theano (from Keras==1.1.1)
  Using cached Theano-0.9.0.tar.gz
Requirement already satisfied: pyyaml in /usr/lib/python2.7/site-packages (from Keras==1.1.1)
Requirement already satisfied: six in /usr/lib/python2.7/site-packages (from Keras==1.1.1)
Collecting numpy>=1.9.1 (from theano->Keras==1.1.1)
  Using cached numpy-1.13.3.zip
Collecting scipy>=0.14 (from theano->Keras==1.1.1)
  Using cached scipy-0.19.1.tar.gz
    Complete output from command python setup.py egg_info:
    Running from numpy source directory.
    /tmp/easy_install-ayEJIv/numpy-1.13.3/setup.py:367: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      run_build = parse_setuppy_commands()
    /tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/system_info.py:572: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()
    /tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      self.calc_info()
    /tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      self.calc_info()
    /bin/sh: svnversion : commande introuvable
    /bin/sh: svnversion : commande introuvable
    /tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()
    /tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
      warnings.warn(msg)
    non-existing path in 'numpy/distutils': 'site.cfg'
    Could not locate executable g77
    Could not locate executable f77
    Could not locate executable ifort
    Could not locate executable ifl
    Could not locate executable f90
    Could not locate executable efl
    Could not locate executable gfortran
    Could not locate executable f95
    Could not locate executable g95
    don't know how to compile Fortran code on platform 'posix'
    _configtest.c:1:5: attention : conflicting types for built-in function « exp »
     int exp (void);
         ^~~
    _configtest.c:1:20: erreur fatale : Python.h : No such file or directory
     #include <Python.h>
                        ^
    compilation terminée.
    _configtest.c:1:20: erreur fatale : Python.h : No such file or directory
     #include <Python.h>
                        ^
    compilation terminée.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-2zSIir/scipy/setup.py", line 416, in <module>
        setup_package()
      File "/tmp/pip-build-2zSIir/scipy/setup.py", line 412, in setup_package
        setup(**metadata)
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 335, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 456, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 863, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1141, in best_match
        return self.obtain(req, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1153, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 522, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 672, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 698, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 879, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1104, in run_setup
        run_setup(setup_script, args)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 257, in run_setup
        raise
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 199, in setup_context
        yield
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 170, in save_modules
        saved_exc.resume()
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 145, in resume
        six.reraise(type, exc, self._tb)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 158, in save_modules
        yield saved
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 199, in setup_context
        yield
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 254, in run_setup
        _execfile(setup_script, ns)
      File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 49, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/setup.py", line 392, in <module>
        # higher up in this file.
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/setup.py", line 384, in setup_package
        if "--force" in sys.argv:
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/core.py", line 169, in setup
      File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 160, in run
        self.run_command("egg_info")
      File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/command/egg_info.py", line 26, in run
      File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/command/build_src.py", line 148, in run
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/command/build_src.py", line 165, in build_sources
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/command/build_src.py", line 324, in build_extension_sources
      File "/tmp/easy_install-ayEJIv/numpy-1.13.3/numpy/distutils/command/build_src.py", line 377, in generate_sources
      File "numpy/core/setup.py", line 421, in generate_config_h

      File "numpy/core/setup.py", line 47, in check_types
        Programming Language :: Python :: 3.2
      File "numpy/core/setup.py", line 280, in check_types
        test="""
    SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-2zSIir/scipy/

Making your own server