When you say start with smaller images, you mean cropped versions of the original ones?
then switch to normal ones.?
These models donât have an âinput sizeâ, any size works.
2 Likes
what is cycle_mult forgot what it does exactly.
When you are using the whole dataset for training at the end, how do you know you are not overfitting?
The multiplication of the number of epochs per cycle.
Does making copies of the rare cases give the neural network a false sense of the chances of that event happening or overfit that specific result?
e.g. you have 1 pink unicorn and say you have 100 pink unicorns completely dissing the purple unicorn community.
1 Like
I donât know specifics of the architecture we use here, but, generally, you can deal with different dimensions of CNN inputs by adjusting size of a pooling layer.
1 Like
You usually donât use your validation for training.
Is the dog breed notebook available anywhere ?
3 Likes
What strategy do you recommend if the input image size is very small (i.e. 10x10 or similar) or it has more channels (RGB + additional)? To my understanding for these two cases the pre-trained models (i.e. resnet34) canât be used, right? Is building from scratch the only option?
1 Like
Oh, okk, thanks. I thought Jeremy mentioned about training with the whole data at the end for the dog breeding competition.
IMO: What matters with setting dimensions is the final FC layers - CNN layers can adapt to to different sizes. However when you connect to FC layer - it will cause error. However, if you use Fully convolutional network (no Dense / fully connected layers) then it should work.
2 Likes
Are you going to be sharing the Dogs notebook from the lecture as well?
Yeah but doesnât the pooling layer basically downsample to match the pretrained modelâs input layer?
If Iâm understanding what he did, it was to crop a larger image to 299x299 instead of 224x224. The pooling layer then effectively downsamples it back down to 224x224.
Whats the difference between precompute=True and freezing layers?
@nafizh He did. He recommends to use the whole training set at the end. But this is when youâve already confirmed that your training strategy (lr rate, schedule, âŚ) works and youâre not severely overfitting with validation set.
2 Likes
@yinterian @jeremy Could you please cover how to save interim weights, lets say if I wanted to average the params at specific points based on some conditions?
learn.save
@yinterian Can that be done while the learning is running? Does the optimization not need to be paused before the params are saved?