I’m new to fastai and don’t quite well grab some concepts.
There are some questions I have :
When using fit_one_cycle, a result table is obtained at the end. How is the error_rate in this case calculated ( train or validation set ) ?
(It looks like it is on the train but I couldn’t find an official description of the content of this table)
If it is calculated on train set, how come a lot of tutorials (another tutorial) use it as measure of performance of the classifier ?
If not, how can I get performance on validation and test sets separately (in a fastai way if it exists) ?
PS: I spent a fair amount of time trying to find answer without stumbling on reliable content so I hope you don’t mind answering a “stupid” question.
It seems you’ve come to fastai through a 2020 tutorial, so I’m guessing you haven’t discovered the 2022 Fastai Course that was released a couple of weeks ago. I suggest you will find your answers & more by watching those videos, and reading the book.
The metric error_rate is calculated on the validation set. The Recorder callback is responsible for tracking the loss & metrics during training by attaching to the Learner. And its default is to calculate the metrics on the validation data as can be seen from the signature of the constructor.
Can you please tell me how to evaluate on test set ?
PS: I am quite puzzled how it is difficult to get answers to such simple questions. I’ve seen the tutorial hinted by @bencoman but that looks like a lot of work for a framework change.
just to clarify - I’m a novice at this, having just completed the Part 1 lessons. I didn’t have a specific answer but a memory of a good answer in the Lessons.
Can you please tell me how to evaluate on test set ?
