Kaggle Comp: Plant Seedlings Classification

(Aditya) #156

tqdm needs to be modified for this i guess
because this isn’t part of fast.ai lib

(Dinkar Juyal) #157

How do I go about doing that?

(Aditya) #158

Not sure but some file needs to be edited here…

(Jeremy Howard) #159

If you restart the kernel that problem will go away.

(Vijay Narayanan Parakimeethal) #160

I need help to understand where I am going wrong here. I used the ImageClassfier from paths method after creating the necessary folders in valid and transferring around 20% of the images there. When I do the learn.TTA(is_test=True) I have the below error:

my data is

data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz),test_name=‘test’)

(Vijay Narayanan Parakimeethal) #161

This got fixed when I restarted and ran the notebook again. I am having an issue with my submission score. My learn.TTA() on the train and validation set is 0.97646. But my submission score on the test is a poor 0.08. Pls let me know as to where I am going wrong on the submission. I used both learn.TTA() and learn.predict separately on the test set but there is no difference whatsoever on the score.

(Ramesh Sampath) #162

There must have been a problem in your Submission File creation (i.e., the file names and predictions are not aligned correctly). Compare the sample submission file with your own. Then look up in this thread (and other threads as well) where people have suggested methods on how they created their submission file.

(Vijay Narayanan Parakimeethal) #163

Thanks Ramesh! I found the error and corrected it. It was as you had pointed a mismatch between my predictions and the file name in the excel file. My score is now 0.96977 and I am in the 40th place in the leaderboard :slight_smile:

(Shubham Singh Tomar) #164

Hi fellows,
I published my notes for this competition.Very similar to what we learned in 1st 2 lessons :stuck_out_tongue: Would appreciate if you’d like to review.

(Jeremy Howard) #165

@shubham24 (and everyone posting on Medium) - be sure to put your twitter handle in your Medium profile, so that you will be credited properly when sharing your post.

(Shubham Singh Tomar) #166

Done. Thanks for the advice :slight_smile:

(Dinkar Juyal) #167

Will do that…thanks!

(Traun Leyden) #168

Is it OK to share my notebook here to ask for some tips? I’m getting 95.6% accuracy at this point, which looks like it would put it fairly low on the leaderboard anyway.

Or would I need to also share it on Kaggle so that all Kaggle participants have access to it?

(sergii makarevych) #169

@tleyden maybe you can ask it in a way non related to this competition? :see_no_evil::hear_no_evil::speak_no_evil:

(Benjamin DeKoven) #170

@shubham24 when I execute

I get the following error:

Perhaps you can share the full python code to generate the submission file?
Your suggestions have been very helpful, thanks,

(Aditya) #171

I doubt that it’s because of the change made to TTA…
Things will change slightly

(sergii makarevych) #172

Maybe this piece of code might be helpful for somebody. Simple model auto optimiser that works like this: train a model, compare its error score to score of a previous epoch:

  • if score improved: save a model and continue training
  • if score decreased: load saved model, reduce lr/=2 and continue training

Might do some work for you if you cant control the process.


def metric(learn, n_aug=4):
    preds, y = learn.TTA(n_aug=n_aug)
    preds = np.mean(np.exp(preds),0)
    return (f1_score(y, np.argmax(preds, axis=1), average='weighted'))

def train_model(learn, lr=1e-2, metric=metric, wd=0, use_wd_sched=False):
    acc = train_fc_layers(learn, lr, metric, attempts=3)
    acc = train_all_layers(learn, lr, metric, acc, wd, use_wd_sched, attempts=5, tolerance=5)

def train_fc_layers(learn, lr, metric, attempts=3):
    acc = 0
    a = 0
    while a < attempts:
        learn.fit(lr, 1)
        new_acc = metric(learn)
        if new_acc > acc:
            acc = new_acc
            print(f'train_fc_layers, attempt {a}, new best metric score is {acc}')
            print(f'train_fc_layers, attempt {a}, metric score was not improved {new_acc}')
            lr /= 2
            a += 1
    return acc

def train_all_layers(learn, lr, metric, acc, wd, use_wd_sched, attempts=10, tolerance=3):
    lrs = np.array([lr/9, lr/3, lr])
    learn.precompute = False

    a = 0
    tol = 0
    while a < attempts:
        learn.fit(lrs, 2, cycle_len=1, cycle_mult=2, 
                  wds=[wd/100, wd/10, wd], use_wd_sched=use_wd_sched) 
        new_acc = metric(learn)
        if new_acc < acc:
            tol += 1
            if tol == tolerance:
                tol = 0
                lr /= 10
                lrs = np.array([lr/9, lr/3, lr])
                a += 1
            print(f'train_all_layers, attempt {a}, metric score was not improved {new_acc} , tol {tol}')
            tol = 0
            acc = new_acc
            print(f'train_all_layers, attempt {a}, new best metric score is {new_acc}')
    return acc

(Shubham Singh Tomar) #173

Hi @bdekoven,

I generated the submission file something like this:

Let me know if this worked for you.

(Benjamin DeKoven) #174

Hi @shubham24,

Yes, this works perfectly.

Thank you,

(James Requa) #175

@sermakarevich I was wondering when you were going to pass me…nice job! :clap: