Dog Breed Identification challenge

(Sanyam Bhutani) #21

I suggest that we could participate individually and keep a friendly competition amongst ourselves on the leaderboard first. :slight_smile:

(satish) #22

I did move it to the Folder with the Test,Train and Valid and uploaded to .

You can just to Wget to get the dataset with the preprocessing. Hope this helps

(sergii makarevych) #23

Yep. Looks like that is the choice.

(K Sreelakshmi) #24

Hey Thanks that worked well.

(satish) #25

I just realized the Test folder was empty :grinning:. Will upload a new dataset folder shortly

(Eric Perbos-Brinck) #26

@jamesrequa I see you started submitting, did you build upon Jeremy’s notebook demo in class ?
If so, how did you move from training to actual prediction & submit with the fastai library ?

(James Requa) #27

Yep I just built on the lesson 1 notebook with a couple of differences, outlined below:

  1. I used from_csv instead of from_paths since this dataset isn’t provided in sub-folders by class. Jeremy covers all of this in Lesson 2, but its pretty much just a matter of passing in the train csv file name and paths to image folders. You also need to assign the validation set. Make sure you also provide the path to the test images here as the test_name argument otherwise it won’t be able to find the test images when you run predictions after training.

  2. When you are finished training and ready to run predictions on the test set you just need to use learn.predict(is_test=True). You can also do TTA the same way as you normally would just making sure to pass in is_test=True since you are using the test set not the validation set.

  3. You’ll also need to convert the raw predictions to probabilities i.e. test_preds = np.exp(test_preds).

  4. From there you just create the submission file using the probabilities. It should already be in the correct shape. One last thing to be careful about is the sorting/ordering of the predictions. When I first submitted to Kaggle, in fact, I had the wrong sorting (I used the default ordering in the sample submission file) and got really bad scores as a result. You can be sure its correct just by using data.test_dl.dataset.fnames as your test image ids.

That’s basically it!

(Eric Perbos-Brinck) #28


I’ve tried to duplicate your notebook, as shown in last night second-half.
It works fine when I use “regular” model names in Cell #3 as arch=resnet50 (or resnet101), though with lower accuracy (0.89 vs 0.92)

But if I enter arch=resnext101_64, when I run the first learn = ConvLearner.pretrained(arch, data, precompute=True), it generates an error as:

Preformatted text`FileNotFoundError: [Errno 2] No such file or directory: ‘/media/eric/SSD500/fastai-master2/courses/dl1/fastai/weights/resnext_101_64x4d.pth’

Same with inception_4

I did a git pull a few minutes ago, no fix.

(James Requa) #29

@EricPB You just need to download the weight files here and put them in the directory.

Dog Breed Challenge Precompute Error
(Eric Perbos-Brinck) #30

Thx !
Downloading now (without the ‘4’ at the end of the url :-))

(James Requa) #31

Oops just edited the link, thanks!

(Eric Perbos-Brinck) #32

Nasty bugger :spy:

(Chan Sooi Loong) #33

One night works and first ever submission got me top15 in the competition!

Thanks @jeremy for this amazing course! and also @jamesrequa @sermakarevich for the all tips in the discussion!

(Chan Sooi Loong) #34

You can use arch = resnext101 without the 64, I assume resnext101_64 is another model. That works for me without having to download the weights.

(James Requa) #35

@jakcycsl Nice work!!

(satish) #36

Hi Guys, Did you resized the images to 224 or 299. I resaw the video and Jeremy didn’t mention to resize. But there is code block which does the resize. Just want to know how you are handling that

(sergii makarevych) #37

@satish860 It’s sz parameter. Try different values to see which is better. Jeremy gave a hint to look at sizes distribution.

@jakcycsl Congrats! I assume 10-th place is a group of students from some other dl course )

Interesting question from @Moody in another thread

(Eric Perbos-Brinck) #39

@jeremy explains why he used the resize trick, from 224 to 299, here.
It’s cutting-edge SOTA secret sauce, don’t tell anyone :sunglasses:

(Rikiya Yamashita) #40

I got this error when I tried to make predictions for test set.
Does anybody help me solve this issue?
Thanks in advance :slight_smile:

(James Requa) #41

@rikiya The same thing happened to me and only with TTA predictions.

To solve it, you should be able to edit it slightly as below:
test_probs = np.exp(test_log_preds[0])