Setup problems: Running the Lesson 1 Notebook


(Rachel Thomas) #1

This is a place to ask (and answer) questions about running the Lesson 1 Notebook. You should also check out: the lesson 1 wiki for fixes to a few problems.


Lesson 1 discussion
(vedshetty) #2

I installed latest versions of Theanos, Keras and Anaconda locally on my Windows machine - so I can play with sample data sets without hoping to use the t2.large instance.

At this line:
vgg = Vgg16()
I ran into this error:

(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-7-2b6861506a11> in <module>()
----> 1 vgg = Vgg16()
      2 # Grab a few images at a time for training and validation.
      3 # NB: They must be in subdirectories named based on their category
      4 batches = vgg.get_batches(path+'train', batch_size=batch_size)
      5 val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2)

C:\Users\Shetty\DeepLearning_USFCA\vgg16.pyc in __init__(self)
     24         self.FILE_PATH = 'http://www.platform.ai/models/'
     25         self.vgg_mean = np.array([123.68, 116.779, 103.939]).reshape((3,1,1))
---> 26         self.create()
     27         self.get_classes()
     28 

C:\Users\Shetty\DeepLearning_USFCA\vgg16.pyc in create(self)
     72         self.ConvBlock(3, 512)
     73 
---> 74         model.add(Flatten())
     75         self.FCBlock()
     76         self.FCBlock()

C:\Users\Shetty\Anaconda\lib\site-packages\keras\keras\models.pyc in add(self, layer)
    306                  output_shapes=[self.outputs[0]._keras_shape])
    307         else:
--> 308             output_tensor = layer(self.outputs[0])
    309             if type(output_tensor) is list:
    310                 raise Exception('All layers in a Sequential model '

C:\Users\Shetty\Anaconda\lib\site-packages\keras\keras\engine\topology.pyc in __call__(self, x, mask)
    512         if inbound_layers:
    513             # this will call layer.build() if necessary
--> 514             self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
    515             input_added = True
    516 

C:\Users\Shetty\Anaconda\lib\site-packages\keras\keras\engine\topology.pyc in add_inbound_node(self, inbound_layers, node_indices, tensor_indices)
    570         # creating the node automatically updates self.inbound_nodes
    571         # as well as outbound_nodes on inbound layers.
--> 572         Node.create_node(self, inbound_layers, node_indices, tensor_indices)
    573 
    574     def get_output_shape_for(self, input_shape):

C:\Users\Shetty\Anaconda\lib\site-packages\keras\keras\engine\topology.pyc in create_node(cls, outbound_layer, inbound_layers, node_indices, tensor_indices)
    150             output_masks = to_list(outbound_layer.compute_mask(input_tensors[0], input_masks[0]))
    151             # TODO: try to auto-infer shape if exception is raised by get_output_shape_for
--> 152             output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
    153         else:
    154             output_tensors = to_list(outbound_layer.call(input_tensors, mask=input_masks))

C:\Users\Shetty\Anaconda\lib\site-packages\keras\keras\layers\core.pyc in get_output_shape_for(self, input_shape)
    400             raise Exception('The shape of the input to "Flatten" '
    401                             'is not fully defined '
--> 402                             '(got ' + str(input_shape[1:]) + '. '
    403                             'Make sure to pass a complete "input_shape" '
    404                             'or "batch_input_shape" argument to the first '

Exception: The shape of the input to "Flatten" is not fully defined (got (0, 7, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.

When I ran the identical notebook on the t2.large instance, it worked just fine. Since t2.large also has CPUs, any pointers why I am running into this error?

Thanks.


(Jeremy Howard) #3

Try running the steps in part 3 - ‘Create a VGG model from scratch in Keras’. Do you get the same error if you do that?


(vedshetty) #4

Yep … unfortunately exactly the same error.


(Taro) #5

vshets, i can help you with this using Zoom if you’d like. It allows you to share your screen with me. Message me on slack @taro


(Taro) #6

solved the problem with vshets.

turns out the image_dim_ordering in his ~/.keras/keras.json file was set to “tf” instead of “th”

not sure how that happened but we changed it to “th” and is working now


(yad.faeq) #7

new version of Keras defaults to tf if installed standalone. Ref


(sethiavivek2006) #8

I tried the installing locally all the required packages on Ubuntu 16.04 locally. But when I tried to import Keras I got an error stating that module tensorflow is missing, although I did install that too.


(Jeremy Howard) #9

@taro.s.chiba congrats on some great debugging!


(Jeremy Howard) #10

@sethiavivek2006 you need to follow the instructions on the keras site showing how to change your backend to theano


(sethiavivek2006) #11

@jeremy Thanks for the reply. I have followed that and changed my backend to tensorflow as mentioned in this link.


(Jeremy Howard) #12

@sethiavivek2006 hopefully you meant that you’ve changed your backend to theano ?


(lin.crampton) #13

Seth – Are you successfully running Anaconda3 w Python3?

lin


(sethiavivek2006) #14

@jeremy @lin.crampton Thanks. I was able to import keras after changing the backend to theano. Previously there was a typo and hence it didn’t work.


(carlos roberto) #15

Hi @vshets,
Are you using Windows 10 Bash environment?
Tks


Getting error when try to instanciate vgg16()
(Le Dac Liem) #16

It works. Thanks!!!


(Yohan ) #18

I’m having an issue I’m hoping someone can help me with :

ImportError Traceback (most recent call last)
in ()
----> 1 import utils; reload(utils)
2 from utils import plots

/home/ubuntu/nbs/utils.py in ()
50
51 from vgg16 import *
—> 52 from vgg16bn import *
53 np.set_printoptions(precision=4, linewidth=100)
54

ImportError: No module named vgg16bn

even though I have vgg16bn.py from the github downloaded into ~/nbs/. I thought it might be a permissions issue, but the acl on vgg16, utils, and vgg16 are the same. I have also tried restarting the python notebook kernel. I appreciate the help!


(Jeremy Howard) #19

Be sure that you are in ~/nbs when you run ‘jupyter notebook’. You can run ‘%pwd’ in a cell to check you’re in the right directory.


(Yohan ) #20

I fixed the issue, though I am not clear on what caused it. I’ve been connecting to aws via the aws-ssh alias where is when i applied wget to retrieve vgg16bn.py. However, this time I connected with the connect command I saved from the setup_p2.sh , and there I saw the python script was missing. I think the ip address was not stable when i do aws-get-p2 and then aws-ssh. I just remembered you mentioning this in the beginning of lesson 1, so I’ll go back and try to see what I need to do to avoid this issue in the future. Thank you for the quick response!


(dt) #21

Hi guys, I tried running the Lesson 1 Notebook on my 2015 MBP (macOS Sierra 10.12.2)

I installed Anaconda/Theano/Keras/Python libraries this way:
Anaconda: Using the installer from https://repo.continuum.io/archive/Anaconda2-4.2.0-MacOSX-x86_64.pkg
Theano: pip install theano
Keras: pip install keras
cv2: conda install -c https://conda.binstar.org/jjhelmus opencv
bcolz: conda install bcolz

When running import utiles, I get the following error:

Googling the error doesn’t give much relevant results. Can anyone help please? Thanks

Edit: Deleted Anaconda and reinstalled using the command line installer and commands from install-gpu.sh and it works now :slight_smile: