Lesson 1 discussion


(Elle Idan) #266

installed bcolz with conda… trying again…


#267

You can fix that by running “pip3 install bcolz”, you will also probably find a few other missing modules that you can fix the same way. If you’re missing a module named PIL you have to run “pip3 install Image”.

I see you’re using conda, that should work the same way as pip.


(Elle Idan) #268

Yep, that resolved the issue. Obviously, Anaconda doesn’t come with bcolz pre-installed.

One step closer to running all that code…


(Elle Idan) #269

One more problem:

So close, yet so far …

When running this code:
import utils; reload(utils)
from utils import plots

I get this error


NameError Traceback (most recent call last)
in ()
----> 1 import utils; reload(utils)
2 from utils import plots

NameError: name ‘reload’ is not defined


I understand that the ‘reload’ is another difference between python 2 and 3. Apparently removed from 3, and the suggested work around is this

from imp import reload

That loads perfectly from within my Spyder IDE, but it doesn’t work at all, as you can see in the error message, within Jupyter Notebook.

Yes, I did add that code into the correct utils. py :slight_smile:

Any suggestions?


(Elle Idan) #270

I fixed that issue as well now.

It’s odd though that i needed to add the from imp import reload ( or the the python 3 version of from importlib import reload) into the Juypyter Notebook code section for it to work.

So the code is now:
import utils;
from importlib import reload
reload(utils)
from utils import plots

I would have thought that adding the from importlib import reload into utils.py would have loaded that code with the import utils; statement. … :confounded:


(gaurav gupta) #271

I am getting error while importing utils on windows.

ImportError                               Traceback (most recent call last)
<ipython-input-4-834d59d32016> in <module>()
----> 1 import utils; reload(utils)
      2 from utils import plots

F:\Technology\Learning\DeepLearning\kerasTuts\fastai\courses\deeplearning1\nbs\utils.py in <module>()
     20 from scipy.ndimage import imread
     21 from sklearn.metrics import confusion_matrix
---> 22 import bcolz
     23 from sklearn.preprocessing import OneHotEncoder
     24 from sklearn.manifold import TSNE

c:\anaconda2\lib\site-packages\bcolz\__init__.py in <module>()
     53     array2string, set_printoptions, get_printoptions )
     54 
---> 55 from bcolz.carray_ext import (
     56     carray, blosc_version, blosc_compressor_list,
     57     _blosc_set_nthreads as blosc_set_nthreads,

ImportError: DLL load failed: The specified procedure could not be found.

(Tim Bernal) #272

On my instance when I tried git clone https://github.com/fastai/courses.git

I got: The program 'git' is currently not installed. You can install it by typing: sudo apt-get install git

However, if I try sudo apt-get install git I get the following message:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 git : Depends: liberror-perl but it is not installable
E: Unable to correct problems, you have held broken packages.

I setup this t2 (I’m still waiting on amazon to give me my p2) using setup_t2.sh

Any ideas what’s wrong here?


(Niyas Mohammed) #273

Did you do an update before trying to install git?

sudo apt-get update

and then

sudo apt-get install git

Your problem is well documented over the web (see this answer on askubuntu).

Next time, you can search for a significant line in the returned error message. Here, you can google for

git : Depends: liberror-perl but it is not installable

and you’ll probably find some possible solutions to the problem online.

Cheers!


(Niyas Mohammed) #274

@jeremy Rachel’s article talks about some unintentional biases in AI, such as tagging people as animals and Nikon’s camera misinterpreting people as blinking. George Hotz, in a video related to Comma.ai states how difficult it is to debug deep learning models.

Is there a way to handle these types of errors which are spectacularly off the charts without retraining the entire model?

It seems that these problems are significant enough to be handled by deep learning models, but retraining the entire model seems wasteful due to the huge amount of data to be crunched.


(Jeremy Howard) #275

reload is in the global namespace by default in py2.


(Jeremy Howard) #276

Be sure to install bcolz with conda, not pip.


(Jeremy Howard) #277

Yes, you can use fine-tuning with a dataset of the kinds of images you’re getting wrong.


#278

I’ve been trying to debug the ZeroDivisionError for the last few hours and now I’m confused… Why does there have to be an additional ‘unknown’ directory?


#279

Just to add, here’s a screenshot of what I’m seeing. Everything seems to work well up to the end of the training stage. get_batches doesn’t seem to pick up any of the images in the test directory, but I’ve made absolutely sure that this directory is not empty. I would really appreciate a nudge in the right direction!


(gaurav gupta) #280

Has anyone uploaded the saved model for lesson1?


(Teemu Kurppa) #281

munyari, Have you put images inside test directory to another subfolder (e.g. called unknown). Keras ImageDataGenerator expects that kind of structure.

E.g.
test/unknown/112.jpg

You can check your structure by:
os.listdir(path+'test')


#282

No, I hadn’t done that. Let me try it.

EDIT: That worked, thank you :slight_smile:


(James Puderer) #283

Thanks. However, the Wiki points to the old files in http://www.platform.ai/files

So… you may get this question again.

I can’t edit the Wiki, or I would have changed it for you. :slight_smile:

Actually, I would be quite happy to fix minor issues with the Wiki as I go through the lessons. I just need a Wiki account to do so, or a place to report all the minor issues.


(Anshaj Goel) #284

Hello,

I have the following error an my mac:

In [16]:

from utils import plots
import utils; reload(utils)
from utils import plots

ImportError Traceback (most recent call last)
in ()
----> 1 import utils; reload(utils)
2 from utils import plots

/Users/anshaj/utils.py in ()
20 from scipy.ndimage import imread
21 from sklearn.metrics import confusion_matrix
—> 22 import bcolz
23 from sklearn.preprocessing import OneHotEncoder
24 from sklearn.manifold import TSNE

ImportError: No module named bcolz

How can i solve this ?


(sai kiran) #285

Running,
vgg = Vgg16() is giving me the following error. Kindly help me out…

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\layers\core.py:577: UserWarning: output_shape argument not specified for layer lambda_2 and cannot be automatically inferred with the Theano backend. Defaulting to output shape (None, 3, 224, 224) (same as input shape). If the expected output shape is different, specify it via the output_shape argument.
.format(self.name, input_shape))
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)
(Subtensor{int64}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Subtensor{int64}.0)

ValueError Traceback (most recent call last)
in ()
----> 1 vgg = Vgg16()

C:\Users\Sai Kiran\courses\deeplearning1\nbs\vgg16.pyc in init(self)
31 def init(self):
32 self.FILE_PATH = ‘http://www.platform.ai/models/
—> 33 self.create()
34 self.get_classes()
35

C:\Users\Sai Kiran\courses\deeplearning1\nbs\vgg16.pyc in create(self)
75 self.ConvBlock(3, 512)
76
—> 77 model.add(Flatten())
78 self.FCBlock()
79 self.FCBlock()

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\models.pyc in add(self, layer)
325 output_shapes=[self.outputs[0]._keras_shape])
326 else:
–> 327 output_tensor = layer(self.outputs[0])
328 if isinstance(output_tensor, list):
329 raise TypeError('All layers in a Sequential model ’

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\engine\topology.pyc in call(self, x, mask)
567 if inbound_layers:
568 # This will call layer.build() if necessary.
–> 569 self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
570 # Outputs were already computed when calling self.add_inbound_node.
571 outputs = self.inbound_nodes[-1].output_tensors

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\engine\topology.pyc in add_inbound_node(self, inbound_layers, node_indices, tensor_indices)
630 # creating the node automatically updates self.inbound_nodes
631 # as well as outbound_nodes on inbound layers.
–> 632 Node.create_node(self, inbound_layers, node_indices, tensor_indices)
633
634 def get_output_shape_for(self, input_shape):

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\engine\topology.pyc in create_node(cls, outbound_layer, inbound_layers, node_indices, tensor_indices)
166 # TODO: try to auto-infer shape
167 # if exception is raised by get_output_shape_for.
–> 168 output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
169 else:
170 output_tensors = to_list(outbound_layer.call(input_tensors, mask=input_masks))

C:\Users\Sai Kiran\Anaconda2\lib\site-packages\keras\layers\core.pyc in get_output_shape_for(self, input_shape)
434 raise ValueError('The shape of the input to “Flatten” '
435 'is not fully defined '
–> 436 '(got ’ + str(input_shape[1:]) + '. '
437 'Make sure to pass a complete “input_shape” '
438 'or “batch_input_shape” argument to the first ’

ValueError: The shape of the input to “Flatten” is not fully defined (got (0, 7, 512). Make sure to pass a complete “input_shape” or “batch_input_shape” argument to the first layer in your model.

I remember reading in keras documentation that keras doesn’t require to explicitely ask for the output_shape in the convolutions. Correct me if I’m wrong. Thank you!