Kaggle Comp: Plant Seedlings Classification

Part 1 (2018)
49

So I have been trying climb up the leaderboard and seem to have stagnated a bit. Currently @ 12th on LB with 95.34 score.

Tried the following:

  • couple of resnet architectures, tried inception but the accuracies on validation was not in the same league
  • using data-aug
  • played around with dropout (trained separate models with and without dropout)
  • used two image sizes, 300 and 400. Maybe I should try 200? most of the images seem to be sub 300x300
  • did not unfreeze early layers ( the model was giving an OOM if I try to unfreeze and train at bs=64 on aws-p2)
  • used all the training data (somehow if I hold back some of the training data, model predictions seem to be doing better, smells like overfitting)
  • ensembled models

What else can I try?

  • Wonder what it takes to get to 98% accuracy some of you have been seeing?
  • Meaning to try: pseudo-labeling, k-fold cv
  • or need to up my ensembling game?
50

I got 96% accuracy, using resnet50 (based on comments here) and using all of the techniques i know (including resizing images and a final training step of removing the validation set). I’ll go through tomorrow and see if I missed anything from the lectures. :slight_smile:

I have a ton of questions (apologies for so many at once):

I had an issue with my model where it appeared it was “overfit” pretty early on in my training process. Does it make sense in this case to discard your model and restart your training? I wound up with what looked like overfitting in step 3 of 7 that I did and model showed “overfit” numbers the rest of the way through my training steps. I figured the model would eventually “correct” itself, I’m not confident if this was the right approach.

I noticed that the model seemed to make it’s biggest improvements when unfreezing and training earlier layers. Intuitively, this makes sense since the image set is quite different from image net. I thought perhaps I should put some more training epochs into the earlier layers than what I wound up doing. How often do you adapt your training procedure based on this sort of in-training observation?

Finally, with discarding the validation set, is there any heuristic as to how to decide how much training to do? Since I can’t compare training with validation, I can’t tell if the model is overfitting or not. I was pretty conservative and only did 1 training step
learn.fit(.1, 3, cycle_len=1, cycle_mult=2)

I thought to possibly unfreeze and train earlier layers as well discarding the validation set because that seemed to work better for the model but I didn’t wind up doing it.

Finally, since the model seemed to overfit pretty quickly, is this at all because the training set is pretty small (iirc, 4500 images over 12 categories) and should we tweak number of epochs based on training set size?

51

IS someone success ful to re-load saved model in this competition?

Always getting cuda out of memory…

Whenever i try to do it with learn.unfreeze()

52

Do you have other notebooks running in the background? I’ve had that issue when I had too many notebooks open.

Not on this competition, on others though.

53

None running…
The best part is i have reduced my bs to 8 still…

The memory consumtion was 4gigs before learn.unfreeze() and shoots all the way to close to 11gigs after that line…

Capture

54

Looks like the key in this competition is to be able to unfreeze and train some of the earlier layers. Reducing the batch size to 32 was able to get over my OOM issues.

Now moved to 7th position on the LB :slight_smile:

1 Like
55

They are “青出于蓝而胜于蓝”.

Thanks, Jeremy. This is my first time to put an end-to-end notebook together (from data preparation to submit a csv file… although it is very hacky). :sweat_smile: Now, I can start training the model with full dataset and other basic experiments.

@jeremy Are you going to teach cross-validation and ensemble anytime soon? How can we do it with fastai library?

1 Like
56

You can follow the approach in planet_cv.ipynb, although you’ll need to update the import statements.

1 Like
57

TIL :slight_smile:

58

==> the student surpasses the master :slight_smile:

I’m even learning Chinese

1 Like
59

Still trying to get unfreeze done…

60

It’s a bit more subtle than the google translation though!: https://forum.wordreference.com/threads/青出于蓝而胜于蓝.2706719/

3 Likes
61

took me a couple of attempts but got you too @Jeremy :stuck_out_tongue:

59
Screen Shot 2017-11-26 at 16.12.59.png1974×716 80.2 KB

9 Likes
62

Good to be back participating after a break! @14 now.

63

What is the memory size of the GPU you are using? You can check using “nvidia-smi”.
Possible Solution :- try reducing batch size.

64

Hummm… It still may left room for personnal improvment…![Screenshot at 2017-11-26 00:39:00|690x387]

Screenshot at 2017-11-26 00:39:00
Screenshot at 2017-11-26 00:39:00.png982×600 52.1 KB

1 Like
65

Just Got it done by reducing the image sizes to 400 from 500…

Around .4 gigs was left thereafter…

1 Like
66

Hi,

I am getting target indexes rather than sparse target matrix.

image
image.png1204×908 80.7 KB

It’s because of is_single thing, but don’t we need nhot labels and a softmax at the end ?

def csv_source(folder, csv_file, skip_header=True, suffix='', continuous=False):
    fnames,csv_labels,all_labels,label2idx = parse_csv_labels(csv_file, skip_header)
    full_names = [os.path.join(folder,fn+suffix) for fn in fnames]
    if continuous:
        label_arr = np.array([csv_labels[i] for i in fnames]).astype(np.float32)
    else:
        label_arr = nhot_labels(label2idx, csv_labels, fnames, len(all_labels))
        is_single = np.all(label_arr.sum(axis=1)==1)
        if is_single: label_arr = np.argmax(label_arr, axis=1)
    return full_names, label_arr, all_labels
68

Nice commands! How do you get classes after you execute them?

69

You execute those commands after you have created a CSV containing file,species. So, the CSV contains the classes i.e species.

If you mean getting the classes after you’ve read the data using ImageClassifierData.from_csv use data.classes.

10 PM
Screen Shot 2017-11-26 at 7.57.10 PM.png864×468 36.6 KB

1 Like