The screenshot below is from Lesson 06_multicat. It is discussing the thresholds and illustrating the difference between thresholds 0.2 and 0.1 by calling the learn.validate().
What do the learn.validate() results mean? I cannot find a documentation that explains it.
Thank you!
Maria
Does the first value pertain to the last epoch’s validation loss, and the second value to the accuracy_multi at the specified threshold level?
Yes, that is correct. You can see that the first number matches the epoch 2’s valid_loss.
Thank you!
The learn.validate() method returns a tuple of two elements. The first element is the validation loss and the second element is the accuracy (or the chosen metric). If we do not pass any argument in the method then it calculates these things on the validation set, which corresponds to the ds_idx=1.
We can also choose to calculate these parameters on the train set as well as test set also e.g. learn.validate(ds_idx=0), learn.validate(ds_idx=2) repectively.
There is an other method learn.get_pred() which calculates the prediction on the chosen ds_idx value, which defaults to validation set(ds_idx=1). It returns a tuple of preds and targs, where the dimension of preds is bs*n_classes and the dimension of targs is bs*1. The value of targs can also be obtained by learn.dls.valid_ds.tensors as shown below and preds can also be obtained as learn.model(x)
preds,targs = learn.get_preds()
x,y = learn.dls.valid_ds.tensors
test_close(preds, learn.model(x))