Lesson 1 Experiment - Aeroplanes vs Helicopters Classification Doubts

Part 1 (2018) Beginner (2018)
1

I have trained a ‘Aeroplanes vs Helicopters’ dataset using ResNet34 pre-trained model and here is the following information -

  1. Images - Aeroplanes - 216, helicopters - 350

  2. 25% of this data is kept in validation set

  3. Initial Validation accuracy with bs=8 and LR=0.01 is 91%

  4. Learning rate finder plot is slightly weird
    image

  5. I have chosen 0.01 as learning rate though.

  6. Augmenting the data and training the model again didn’t improve the accuracy (It actually came down to 89.5%)

  7. Later used 3 cycles of 1 epoch each and reached 92.3% accuracy (Train and Validation losses at 0.24 and 0.23 respectively)

  8. Unfroze the initial layers and learn.fit(lr, 5, cycle_len=1, cycle_mult=2)
    This seems to overfit my data as I can see that my training loss is decreasing but my validation loss is more or less fluctuating around same range. Final accuracy is 92.3%
    image

My 2 questions -

  1. Why did Data Augmentation didn’t exactly improve the model in this case?
  2. Although TTA gave final accuracy of 94.4% accuracy, I would like to know if this is the maximum I can get with this data. (6 incorrect aeroplanes, 2 incorrect helicopters)
    Incorrect Aeroplanes -
    image
    image.jpg943×300 96 KB

    Incorrect Helicopters -
    image
    image.jpg938×446 243 KB

Thanks!

2

Can you point to where can I obtain the dataset?(No results googling ;))

3

I have used Firefox’s Google Image downloader extension to download images.

4

Hey Murali,
Your size of your dataset is small, so 5 epochs would definitely overfit.
Could you provide some more details about the model

  1. The accuracy before you unfroze the layers.
  2. Differential learning rate that you are using.
  3. Data Augmentation used.
5

Hi Kiran,

All the details are posted in my blog here -