All links are at the beginning of this wiki thread: Lesson 3 In-Class Discussion
In Otavio’s video we see that a letter is matched against template to find the probability. I wonder if we’ve standard template for:
- Species e.g. dogs … there are so many breeds so what does a standard template have?
- Same question for multi-class classification problems.
Yes - it’s the last layer of a pre-trained network. That’s what we’ve been using for all our models so far! 
1 Like
Later today I will post code doing model customization that may help you understand what is going on.
8 Likes
Since we are adding our layers at the end and training model, wouldn’t model be given by the last layer or so?
Question - Size vs resize:
Resize will modify size of every image and store them for future usage?
Size is a parameter. Everytime code request images, say for printing or processing, it’ll modify images are run time?
Edit: I think yes? - Lesson 2 In-Class Discussion
Question - One hot encoding
We saw that labels are one hot encoded and for saving memory instead of one hot encoded tensor/array, an index is stored. Eventually single output which is index of predicted class!
Is this true for multi-class classification as well? Since we can have multiple classes, there is no single ‘1’ that’ll be returned. Rather multiples 1’s. Does fastai, PyTorch returns indices for them as well? or the output is one hot encoded form than indices?
That’s right. As opposed to in transforms, where it’s done dynamically.
Not “one hot” encoded, since that specifically means there’s just a single ‘1’, but yes it uses ones and zeros, not indexes. See the Excel sheet I showed in the last lesson to see how it looks.
1 Like
I’m curious what range of speedup people are seeing on the planet dataset in lesson2-image_models.ipynb.
I have been running on EC2. Wondering if it is time to fully make a switch to Paperspace going ahead for bigger problems?
sz=128 , with learn.freeze() -> Epoch 100% 7/7 [12:54<00:00, 110.57s/it]
sz=128 , with learn.unfreeze() -> Epoch 100% 7/7 [35:15<00:00, 302.28s/it]
sz=256, with learn.freeze() -> 100% 7/7 [43:01<00:00, 368.84s/it]
sz=256, with learn.unfreeze() -> 29% 2/7 [33:49<1:24:32, 1014.60s/it] ( took about 3 hours to get here, with some time spent browsing forums etc. )
Are you using EC2 P2 machines or G2?
If you search in the forums, there was some problem with paperspace. I would stick with AWS (if you also hv the credits) over the next 6 weeks particularly if you have a setup that’s working well with the AMI.
Thanks, I’ve been using P2 with the Fastai AMI. This has been running for several hours now to get to 86% done on the last learn model. Which makes knowing how to load/save models quite important I guess. I was looking for the saved model (f’{sz}’) but couldn’t find it yet…do you know where that is stored?
That’s strange it’s taking several hours. Which notebook is this? Do you have your code in Github? Can you give some more specifics and I can try to replicate. I use AWS P2 as well and mostly happy with it.
@beecoder @ramesh I am running the same planets notebook on Crestle right now and can tell you that my times are in line with Nikhil’s. I’m getting about 986s/it on sz=256 (after unfreezing).
Thanks @memetzgz. Time to get familiar with loading/saving models. @ramesh this is the usual lesson file in the path “fastai/courses/dl1/lesson2-image_models.ipynb” I’m not doing anything fancy, I did a git pull ~28 hours ago. The AMI is fastai-part1v2-p2 (ami-8c4288f4).
Yeah…Interesting. I am running it now. The sz=64 parameter is the dimensions of the Image runs OK. So larger image size does cause slow runs in general (since it’s a bigger input data dimensions).
Unfreeze all of the layers with larger sz image definitely make it much slower. So may be we unfreeze with smaller dimensions but keep it frozen for larger sz arrays. It might be useful to have a parameter in unfreeze so that we can choose to unfreeze only limited number of top layers and not all the way. I will add this to the other thread on feature requests - Wiki: Fastai Library Feature Requests
1 Like
I looked into it…but it has only two layers above for Resnet34. They were both huge. Is it possible to freeze_to a sub layer or breakdown to more layers in the pretrained network?
The caveat is we have to give more learning Rates. Might be better to give an option to specify a dictionary of layer names we want to unfreeze and learning rates for them? Thoughts / suggestions - @jeremy
What is “num_workers” used for in the ImageClassifierData function?
Number of CPU cores you want to use
Thank you. If we don’t specify anything, what is it by default?
Not sure away from computer - maybe 4 or -1. -1 means use max.
BTW to see this play out I would recommend installing htop and running it from terminal, really nice way to visualize what your CPU cores are doing.