It wasn’t for accuracy, moreso for time. Though I have noticed mabye a 1%-.5% difference between the two. Which at the end of the day is negligible. But I could go further there as well, I just made that briefly this morning. If you would like me to, I can briefly real quick
Going to cross check quickly because I’m submitting and assignment soon. Knowing the real accuracy on test set would matter. I would expect both of them to be the same. Maybe we can share our findings!
Good to know! @sgugger, any thought behind why when we pass in an outside dataloader, our accuracy sky-rockets? We have already determined thanks to your help the proper way to do it, but why would this behavior exist? You can see in my example I get almost 90% doing it the wrong way on the test set. Any thoughts?
I feel the API should be able to support being able to load an ImageDataBunch from just a single folder. Especially when you are able to use ignore_empty…
Also the error it produces is quite unclear. Yeah it is trying to load something from an empty set but it doesnt say it is exactly. Like is it trying to deduce the labels by trying to find a train folder which doesnt exist? Is it still trying to index a folder which doesnt exist (train & valid)?
I feel FastAI is quite good but the error reporting can be significantly improved to make it even easier for new people like me to understand what and why a function fails.
and then you’ll have a databunch with just a train_dl and not valid_dl/test_dl. In general, the factory methods of ImageDataBunch are only suitable for scenarios very similar to what’s in the MOOC for beginners, the data block API is what you should use when you want more flexibility.
As for the error reporting, it can obviously be improved. Any PR for that will be welcome!