Part 2 Lesson 8 wiki

(Minh Nguyen) #275

What you can do to come up with the number 25088 is removing the nn.Linear() part and simply checking the size of the flattened final layer.

head_reg4 = nn.Sequential(Flatten())
learn = ConvLearner.pretrained(f_model, md, custom_head=head_reg4)
learn.opt_fn = optim.Adam
learn.crit = nn.L1Loss()

It will show you

OrderedDict([('input_shape', [-1, 512, 7, 7]),
             ('output_shape', [-1, 25088]),
             ('nb_params', 0)]))])

at the end of the output.

(Minh Nguyen) #276

Hi @yggg, ‘quit’ pdb properly prevents your Jupyter from hang

(Wayne Nixalo) #277

I found a few things.

Moving in Atom – handy navigation manual.

atom-ui-ide. Among other things, it lets you find all references of a function. (either this or python-ide gives you hover-documentation like VSC). It’s sort-of a base package for IDE functionality.

python-ide builds atop that and allows you to search symbols / function declarations in the current project - not just file. However, it requires python language server to work, which is maintained by Palantir – so I don’t know how shady/safe that is. It also let’s you hover over functions/classes for documentation, even for out-of-project imports. It also let’s you CMD-click on a function and go straight to its declaration, even out-of-project. I haven’t seen this work in all cases (worked for sklearn.ndimage imports but not sklearn.metrics), but I’ve been able to CMD-click directly into the NumPy source code w/ this.

atom-ctags enables the built-in Atom search features. It builds a ctags file of recognized symbols, per project. I think VSC does this automatically behind the scenes. It also let’s you use CMD-Shift-R (Mac) to search symbols in a project (the “opim” search Jeremy did).

The functionality does come with a price. On my MacBook, enabling auto-ide-ui adds a solid half-second to Atom’s start time. Enabling it with python-ide makes that almost a full second but feels longer.

Having played with it a bit, I think if I want to keep Atom’s speed & minimalism, I’d stick with using the atom-ctags to let me search symbols or go to definitions (CMD-Shift-Down or CMD-Shift-Up to come back) – although it doesn’t always work: I’m not sure when/not symbols/ctags are generated.

I may check out VSC for Mac (or just Visual Studio?) if I find I need the functionality, but that’s what I’ve found so far.

(Evgeny Sushko) #278

I was wondering why we use “predict_batch” function for making predictions in case of largest item classifier but don’t use it in other cases. As far as I can see this code:

x,y = next(iter(md.val_dl))
predict_batch(learn.model, x)

and this code:

x,y = next(iter(md.val_dl))

give the same output. Why do we use “predict_batch” then?

(Jeremy Howard) #279

Just because it ensures that eval and reset are called first. I don’t use it in some of the lessons since I want to teach how to do it manually.

(Cedric Chee) #280

All the videos are “Unlisted” in Youtube. Now that part 2 is officially launched, is this intended?

(Mariam) #281

The links in the time line are broken.

(Jeremy Howard) #282

Many thanks - fixed now.

(Jon Gold) #283

re: learning greek letters - my comprehension skyrocketed when I picked up Mathematical Notation: A Guide for Engineers and Scientists and could finally search for symbols online / vocalize them etc etc. highly recommend it.

(Mariam) #284

I got it dinning table too. the highest two probabilities were dining table ( 5.9479e-01) and chair (3.5335e-01).

(Jeremy Howard) #285

Seconded! Really great book.

(Gene Sobolev) #286

I tried multiple times, but couldn’t make out the name of the debugger one of the students (Elsa I think) mentioned in the video Did anyone manage to capture the name of the debugger he uses?

(Junxian) #287

I think he said something like:
ipython.core.debugger import tracer

Is that what you are looking for?

(Gene Sobolev) #288

Indeed, Tracer takes color as a parameter so it makes sense in the context of what’s mentioned in the video. Thank you!

(Naresh) #289

I guess accessing images from internet to train a model would be very slow as we add up network latency. I would prefer to download it. Further, u may have to train test for multiple times. Accessing the data from internet in that case can be would not be recommendable. Furthermore, if there is a network issue during the training, the process will fail abruptly I believe. We may have to handle those cases.

(Naresh) #290

I am having issue while running ImageClassifier.from_csv.
This is the line of codes I have

tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, crop_type=CropType.NO)
md = ImageClassifierData.from_csv(PATH, JPEGS, CSV, tfms=tfms, bs=bs)

With error stack trace:

IndexError Traceback (most recent call last)
in ()
3 CSV=’…/data/pascal-localisation/tmp/lrg.csv’
4 tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, crop_type=CropType.NO)
----> 5 md = ImageClassifierData.from_csv(PATH, JPEGS, CSV, tfms=tfms, bs=bs)

~/anaconda3/lib/python3.6/site-packages/fastai/ in from_csv(cls, path, folder, csv_fname, bs, tfms, val_idxs, suffix, test_name, continuous, skip_header, num_workers)
351 “”"
352 fnames,y,classes = csv_source(folder, csv_fname, skip_header, suffix, continuous=continuous)
–> 353 ((val_fnames,trn_fnames),(val_y,trn_y)) = split_by_idx(val_idxs, np.array(fnames), y)
355 test_fnames = read_dir(path, test_name) if test_name else None

~/anaconda3/lib/python3.6/site-packages/fastai/ in split_by_idx(idxs, *a)
364 def split_by_idx(idxs, *a):
365 mask = np.zeros(len(a[0]),dtype=bool)
–> 366 mask[np.array(idxs)] = True
367 return [(o[mask],o[~mask]) for o in a]

IndexError: arrays used as indices must be of integer (or boolean) type

I felt that its due to the wrong folder location. I ran the code from github pointing to the correct folder, it worked. Later, I moved my ipynb to the same folder and ran one more time. It ran. But When I run from the actual location I am intending to run, it fails with the above error. I am truck with this error for a couple of days. Any help would help.

(Peter Correia) #291

Looking into the VOC Documentation I received the impression the bbox was represented as such:
[x_min, y_min, x_max, y_max]
[155, 96, 196, 174] <- car bbox

How did you know that the last two items in the bounding box list represented width and height?


Hi Vijay,

Followed these exact steps and I’m still not able to use symbols. I extracted to C:\Program Files\Microsoft VS Code\bin which is in my PATH when I run set command at the terminal and I still can’t search for something like open_img. Any thoughts?

(Vijay Kumar) #293

Hope you have selected the interpreter, with fastai environment (environment.yml) available with downloaded code.
Once interpreter set you are able to navigate.

(Pierre Guillou) #294

[ EDIT ] : it works :slight_smile: The problem came from the selection of the python interpreter (ctrl+shift-p : Python: Select Interpreter).
The default path to my fastai environement is well setup in my user parameters in Visual Studio Code ("python.pythonPath": "C:\\Users\\username\\Anaconda3\\envs\\fastai\\python.exe",) but I have to select it (ctrl+shift-p) after each restart of Visual Studio Code. Any advice to avoid that ?

Hi @Vijay and @Patrick, I can not get “Go to symbol (ctr+t)” works in Visual Studio Code.

  • I’m using Windows 10 and Visual Studio Code is working.
  • I did open my fastai folder and select the python interpreter of my fastai virtual environment (I’m using an NVIDIA GPU on Windows).
  • I downloaded ctags (universal ctags and I tried as well exuberant ctags) and unzip it in my fastai folder in a folder called ctags : C:\Users\username\fastai\ctags\ctags.exe
  • I updated my Windows PATH with the path to ctags.exe
  • I updated my user parameters in Visual Studio Code with : "python.workspaceSymbols.ctagsPath": "C:\\Users\\username\\fastai\\ctags\\ctags.exe",

What else can I do ? Thanks.