Thank you for your dive into the code, I’m in it myself right now. I agree that the fourth number is the percentage of the true value, but that doesn’t explain why it’s so high right ? Like 1.00. I think that’s only a rounding error isn’t it ? Because of the .2f
I had the same problem earlier. The solution is to update it to the latest version NOT using conda update -c fastai fastai suggested by others (it will update fastai to 1.0.6), but use the dev install option.
A simpler approach I used is to simply install using pip install git+https://github.com/fastai/fastai
Then when you run python -c "import fastai; print(fastai.__version__)" you should see 1.0.12.dev0
If you installed fastai using conda, for some reason it only installs v1.0.6 like the one you have, but that version is at least 4 days behind (if you take a look at your own your_conda_env_path/fastai/vision/models/__init__.py and compare it with the latest one). The one from 1.0.6 doesn’t import any torchvision models. I think it may have been due to Github’s recent server issue and it may be fixed by tomorrow (as of now it hasn’t been fixed).
It is the probability/confidence of the model’s prediction. The question of why it has a confidence of 1.00 and not something less (given that this is a good model) takes understanding of several other concepts including one hot encoding, activation functions, gradient descent to minimize loss, etc. This models confidence is arbitrary. I’m sure Jeremy will go into all of these in a lot more detail as he has in previous courses. It will begin to make sense when you understand the other concepts and is a little bit hard to explain without understanding those other concepts. It is certainly a good question though! Hopefully this gets you pointed in the right direction. This was explained well in previous MOOCs so if you want to go and do some digging you’ll find the information there, but like I said - there are quite a few concepts in play on why this happens.
Just a note so you don’t get frustrated or feel overwhelmed - I found that Jeremy has figured out a really good order in the way he teaches the class and I’m sure this will become more clear later on.
EDIT: I assumed that this model was using softmax as the final layer as I believe was the standard for previous classification default for older fast.ai versions. As pointed out in other posts, it does not use softmax. I’ll be interested to learn why this has changed.
the highest being at indice 24 (Birman class) with a prob of 0.996. But the Ragdoll class (indice 36, prob of 0.9986) gets rounded up to 1.00 with plot_top_losses
That’s a very serious allegation. I hope you have something very specific and detailed to back it up, or that you you make an immediate full retraction.
We have published the full code for both approaches and showed the comparison at the PyTorch developers conference. They are as close as possible to exactly the same.
Your claim is also a straw man. I specifically stated that keras is the “gold standard”. fastai has the benefit of years of additional research. The keras devs made an explicit choice to freeze their API a long time ago.
Awesome lesson! I was playing around with the notebook and can’t quite figure out where the models are being saved when you call learn.save()? The docs say it is in self.model_dir. Is this somewhere in a hidden directory in the jupyter notebook?
Also I was able to get an under 4% (~3.8%) error rate by decreasing the earlier learning rates by 10e-3 and 10e-1 for the later rates: learn.fit_one_cycle(4, max_lr=slice(1e-9,1e-5))