How to output valid data metrics with CSVLogger

fastai
1

I’m trying to use CSVLogger to log metrics during training. Although I see loss of both train and valid, I see only the metrics (e.g. accuracy) of train in the saved log file at each epoch:

Capture

Howe can I extract the metrics of valid data at each epoch as well?

1 Like
2

The accuracy score you see there is on the validation set.

3

Thanks. Then how can I get the metrics for train data?

4

It’s not recommended since the metrics on the validation set are more important (you want to see your validation metrics improve, training metrics improving doesn’t tell you if you’re model is overfitting).

If you really want it, you can do learn.validate(ds_idx=0)

1 Like
5

Thanks. It’s beneficial to have both train and valid data together to see their difference and check for example overfitting.

As I understand, learn.validate(ds_idx=0) only give the metrics of the final model (last epoch). Any way I can record the metrics of both train and valid at each epoch during training process?

6

Hi @welloilel @ilovescience
Thanks to both of you! Finally I find a way to output a value of the metrics and loss. Which value is it? The last? The best?

My problem: I need to evaluate the fitting on slightly changed randomized datasets.
I have a coupe of questions:

1- where I can find some documentation on learn.validate or ?

  1. learn.validate should also contain the best value of accuracy…? Or do I need to use a callback? Because it is not compatible with EarlyStoppingCallBack apparently.
7

Sorry: I also would like CVSlogger. Where is it? How is it saved? Is the Recorder callback which controls this file?
Thanks from now for your help!!