Why set the order in the class or function rather than constructing the transform list in the order you want? I can see hard-coding the order makes it more foolproof but less flexible. I’m curious about the design decision.
The images are usually bigger than what you can fit in RAM. Imagenet is classically trained with 224 by 224 images, but has images with a higher resolution usually.
1 Like
You don’t want your end user to have to worry about order.
1 Like
saw in segmentation notebooks that the sizes were not the same ?. may be I am wrong
So it’s a question of choosing w x h of the images and batch size to fit on the GPU?
Why do we set an _order? If we are passing an array to tfms isn’t that the order of the transforms ?
Has Jeremey/the fastai Dev community considered making databunches lazy? If not, why? It seems like this could be useful for larger than memory datasets.
1 Like
Yes, unless you are dealing with tiny images like MNIST or CIFAR10.
Since your validation data ends up being of a different scale than the training data, the model tends to not learn how to interpret the validation data. Meaning that it pretty much random guesses on the validation set. (training loss 0.5 validation loss 6.0 on first epoch)
This is not exactly a question on what Jeremy is discussing but some of the recommended papers are on regularization. I have seen gradient clipping as another method of regularization. Is that something that would be discussed at any point?
Like I said earlier. You don’t want your end user (yourself tired two months after) to have to worry about the order 
Could you advise any good references or tips about writing efficient code in Python? Or maybe some generic advice? Like, how do you usually come up with fast and efficient solutions? I believe it is mostly about profiling and maybe reading the source code but probably there are some other tricks?
5 Likes
There’s a tradeoff then…
Is the current research clear on whether it’s better to have larger images or larger batch sizes?
3 Likes
Aren’t pixels mostly in fixed range? Why do we normalize images.
1 Like
It’s unclear. For classification for instance, it’s not obvious that going higher in resolution always helps. As for the batch size, it’ll make training faster, but you may have to use additional tricks to regularize your training. There a lot of papers on that, Jeremy will mention LARS a little bit later for instance.
3 Likes
Because we want the inputs to have a mean of 0. and a standard deviation of 1.
2 Likes
last time we saw average pooling (part1), it was mentioned that we will look at concat pooling in part 2
Thx I haven’t noticed the sort. Although that works as long as you have all the labels in the valid dataset which might not be the case if you split by % but that an edge case.
Jeremy covered that part. In any case, you always have to be very careful of how you create your validation set.
Why average pooling? Is max pooling no longer used?
(This is probably answered in one of the readings…)