Hey congrats for pushing it up to 93%. As far as i know, the SOTR accuracy for this dataset is at 94.8% according to here https://paperswithcode.com/sota/fine-grained-image-classification-on-stanford
1 Like
Thanks! Yep I just started playing around with EfficientNet to see how close I can get to their SOTA on stanford cars. EfficientNets seem tricky to train though, have you tried them? will keep playing around 
Thats awesome! Can you describe abt what you mean by “paying more attention to the LR at each stage really helped here!”? What did you observe and then change?
Thanks @bluesky314, just that I trained for a smaller set of epochs (5 or sometimes 10) and then ran lr_find() again to see if it suggested I needed to change the learning rate, I more or less copied how @jianshen92 had done it 
Haven’t tried the EfficientNet, haven’t been working on this project ever since the competition has ended. I admire your perseverance on this!
Recently there is this Radam and LookAhead optimizer that just came out and the community almost says that it guarantees improvement in accuracy and allows us to use a higher learning rate. I think it is worth a try!
Thanks @jianshen92, I think I am just too unimaginative to try something else Have been trying Radam + LookAhead (Ranger) alright, RangerLars looks like it should be better again from the discussion here: https://forums.fast.ai/t/imagenette-woof-leaderboards-guidelines-for-proving-new-high-scores/52714/57:
RangerLars: https://github.com/lessw2020/mish/blob/master/rangerlars.py
Have you seen this new Mish activation too? Going to swap it in the EfficientNet and see how it goes: Meet Mish: New Activation function, possible successor to ReLU?
Crazy time for new techniques!!
@morgan I have read about the links you have posted! Like what Jeremy says, the best practitioner are the one who are persistent.
How is the results with these new techniques?
1 Like
Hmmm the jury is still out. EfficientNet seems to be much more stable using ranger and rangerlars instead of RMSProp, but I’m not sure I can match the accuracy of the paper’s b3 model (93.somthing%).
I’m doing some 80epoch runs with a flat to cosine anneal lr and it seems to get slowly better, but still maxing out around 91.2% with ranger. I think I’ll have to go to 150epochs+ just in case its just a matter of brute force and time.
Just playing around with when the anneal starts as they seem to improve once the annealing period kicks in…also need to figure out what to drop the momentum to.
Once I have a baseline for ranger I’ll try rangerlars, and then Mish
This is my latest run:
1 Like
Why did you choose such scheduling is it suggested by the paper?
@jianshen92 We found in the imagenette/woof experiments that this form of scheduling showed the best results. One cycle was blowing everything up too quickly, and Grankin created this flat cosine annealing function which saw a dramatic increase in accuracy.
1 Like
@muellerzr I found out the thread where you guys talked about it, great stuff! I guess OneCycle will be suitable with only vanilla Adam for now. Might be completely replaced when newer optimiser becomes the standard i guess?
Exactly as @muellerzr said Quick (dumb?) question for you both @muellerzr, @jianshen92 , when you are training are you using the fulling training set for training, with the test set as validation? Or are you splitting your train set into train + validation, keeping the test set only for a final evaluation after training is complete?
@morgan when we’re running the imagenette/woof tests the test is the validation set for us. (It’s how Jeremy set it up in the example notebook)
1 Like
@morgan I asked the exact same question to myself. I think for research purpose it is okay to use test set as validation set. For the stanford car dataset in the competition I entered, i thought it would be “cheating” to use the test set as validation, although not specified.
Is the fit_fc function new in the library? I can’t seem to find it in the current version (1.0.57) library that I am using.
1 Like
@jianshen92 run !pip install git+https://github.com/fastai/fastai.git to grab the most recent version to grab the absolute newest version to use it
1 Like
Thanks both, I had been splitting the train set, but I think I’ll switch to using the test set for validation. I copied it from that crazy thread, but great that its being pushed to fastai, nice!
I think if you want to compare the performance with other researcher (outside of fast.ai), it would be more accurate with an independent test set that is not used to benchmark your training. Being said im not sure how it is done when researcher report their results for benchmark dataset (imagenet etc.). @muellerzr do you have any insight for this?
1 Like
Generally how I do it is I use the labeled test set ‘trick’ that I found and I report two scores. A validation accuracy and a test set accuracy. If you do a search for labeled test sets on the forum and filter to responses from me you should be able to find the source code for my technique
1 Like
I had been wondering the same @jianshen92, I don’t think I recall reading a paper where they specify whether or not they used the test set as validation. So I never new if it was just taken as a given or not…