What are your favorite ways to reduce overfitting?

Part 1 (2018)
1

Dropout.
Higher resolution images.
…

2 Likes
2
  • data augmentation
  • pretrained network
  • gradually increasing image size
  • differential learning rates
  • SGDR
18 Likes
3

Normalization
Cross-Validation

1 Like
4

Less nodes in fc layers

1 Like
5

Let’s say we use gradually increasing image sizes (say 200x200, 300x300, 400x400) to get a final best model.

When we decide to productionize the final trained model then do we need to send images of at the largest size (in this case 400x400) when doing inference? is my understanding correct?

6

Just wanted to ask whether it’s doable to have a FC layer as such,

4096->512
(shouldn't we have another layer joined in 
between as we have such a high drop
 in the number of nodes)
512->No of Classes

Thanks in advance…

7

Yes that’s right. Or at least, ensure your transforms are set to resize to this.

8
  • Regularization (Weight decay) ?
9

Nothing stops you if you would like to make this configuration of FC. You can use xtra_fc to do this.

(shouldn’t we have another layer joined in between as we have such a high drop in the number of nodes)

Thats the quest you need to solve for every particular task. Just do it and compare to different other options you think might be reasonable.

1 Like