Lesson 3 In-Class Discussion

[datafool (pawan)]

Done, thanks!

[afuentes5 (Adriana Fuentes)]

All links are at the beginning of this wiki thread: Lesson 3 In-Class Discussion

[vikbehal (Vikrant Behal)]

In Otavio’s video we see that a letter is matched against template to find the probability. I wonder if we’ve standard template for:

1. Species e.g. dogs … there are so many breeds so what does a standard template have?
2. Same question for multi-class classification problems.

[jeremy (Jeremy Howard)]

@vikbehal

I wonder if we’ve standard template for:

Species e.g. dogs … there are so many breeds so what does a standard template have?
Same question for multi-class classification problems.

Yes - it’s the last layer of a pre-trained network. That’s what we’ve been using for all our models so far!

[yinterian]

Later today I will post code doing model customization that may help you understand what is going on.

[vikbehal (Vikrant Behal)]

Since we are adding our layers at the end and training model, wouldn’t model be given by the last layer or so?

[vikbehal (Vikrant Behal)]

Question - Size vs resize:

Resize will modify size of every image and store them for future usage?
Size is a parameter. Everytime code request images, say for printing or processing, it’ll modify images are run time?
Edit: I think yes? - Lesson 2 In-Class Discussion

Question - One hot encoding

We saw that labels are one hot encoded and for saving memory instead of one hot encoded tensor/array, an index is stored. Eventually single output which is index of predicted class!

Is this true for multi-class classification as well? Since we can have multiple classes, there is no single ‘1’ that’ll be returned. Rather multiples 1’s. Does fastai, PyTorch returns indices for them as well? or the output is one hot encoded form than indices?

[jeremy (Jeremy Howard)]

@vikbehal

Resize will modify size of every image and store them for future usage?

That’s right. As opposed to in transforms, where it’s done dynamically.

@vikbehal

Is this true for multi-class classification as well? Since we can have multiple classes, there is no single ‘1’ that’ll be returned. Rather multiples 1’s. Does fastai, PyTorch returns indices for them as well? or the output is one hot encoded form than indices?

Not “one hot” encoded, since that specifically means there’s just a single ‘1’, but yes it uses ones and zeros, not indexes. See the Excel sheet I showed in the last lesson to see how it looks.

[beecoder (Nikhil)]

I’m curious what range of speedup people are seeing on the planet dataset in lesson2-image_models.ipynb.
I have been running on EC2. Wondering if it is time to fully make a switch to Paperspace going ahead for bigger problems?

sz=128 , with learn.freeze() -> Epoch 100% 7/7 [12:54<00:00, 110.57s/it]
sz=128 , with learn.unfreeze() -> Epoch 100% 7/7 [35:15<00:00, 302.28s/it]
sz=256, with learn.freeze() -> 100% 7/7 [43:01<00:00, 368.84s/it]
sz=256, with learn.unfreeze() -> 29% 2/7 [33:49<1:24:32, 1014.60s/it] ( took about 3 hours to get here, with some time spent browsing forums etc. )

[ramesh (Ramesh Sampath)]

Are you using EC2 P2 machines or G2?

If you search in the forums, there was some problem with paperspace. I would stick with AWS (if you also hv the credits) over the next 6 weeks particularly if you have a setup that’s working well with the AMI.

[beecoder (Nikhil)]

Thanks, I’ve been using P2 with the Fastai AMI. This has been running for several hours now to get to 86% done on the last learn model. Which makes knowing how to load/save models quite important I guess. I was looking for the saved model (f’{sz}’) but couldn’t find it yet…do you know where that is stored?

[ramesh (Ramesh Sampath)]

That’s strange it’s taking several hours. Which notebook is this? Do you have your code in Github? Can you give some more specifics and I can try to replicate. I use AWS P2 as well and mostly happy with it.

[memetzgz (Maureen Metzger)]

@beecoder @ramesh I am running the same planets notebook on Crestle right now and can tell you that my times are in line with Nikhil’s. I’m getting about 986s/it on sz=256 (after unfreezing).

[beecoder (Nikhil)]

Thanks @memetzgz. Time to get familiar with loading/saving models. @ramesh this is the usual lesson file in the path “fastai/courses/dl1/lesson2-image_models.ipynb” I’m not doing anything fancy, I did a git pull ~28 hours ago. The AMI is fastai-part1v2-p2 (ami-8c4288f4).

[ramesh (Ramesh Sampath)]

Yeah…Interesting. I am running it now. The sz=64 parameter is the dimensions of the Image runs OK. So larger image size does cause slow runs in general (since it’s a bigger input data dimensions).

Unfreeze all of the layers with larger sz image definitely make it much slower. So may be we unfreeze with smaller dimensions but keep it frozen for larger sz arrays. It might be useful to have a parameter in unfreeze so that we can choose to unfreeze only limited number of top layers and not all the way. I will add this to the other thread on feature requests - Wiki: Fastai Library Feature Requests

[jeremy (Jeremy Howard)]

@ramesh you can use freeze_to() for that

[ramesh (Ramesh Sampath)]

I looked into it…but it has only two layers above for Resnet34. They were both huge. Is it possible to freeze_to a sub layer or breakdown to more layers in the pretrained network?

The caveat is we have to give more learning Rates. Might be better to give an option to specify a dictionary of layer names we want to unfreeze and learning rates for them? Thoughts / suggestions - @jeremy

[Sree (K Sreelakshmi)]

What is “num_workers” used for in the ImageClassifierData function?

[radek]

Number of CPU cores you want to use

[Sree (K Sreelakshmi)]

Thank you. If we don’t specify anything, what is it by default?