Hi everyone, I would like to be able to retrieve my losses and metrics at the end of my training. When calling
learner.fit(...) a look in basic_train shows that for each element of callback_fns, a callback is generated and passed to fit. Since Recorder is in callback_fns, I would like to extract the Recorder at the end of training to be able to access the loss/metrics. However, after doing
learn.callbacks doesn’t contain the Recorder.
Indeed, when you look at the source code of fit, you see
callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics, callbacks=self.callbacks+callbacks)
which means the instance of Recorder is created when fit is called, but from what I see, is not saved in Learner.callbacks.
I found a solution to my problem. I created a fit2 method for Learner in which I return the Callbacks at the end of training(basically copy the fit method and add a
return callbacks line at the end), but this seems a little convoluted. Can you think of a better solution to get the Recorder (or any Callback generated from Callback_fns) at the end of Training?
Any callback function you pass in
callback_fn becomes an attribute of your learner (lower case, with _ separation). For instance the recorder is in
That’s great! Thank you very much for your answer. Could you point me to where learn gets an attribute for each callback_fns in the source code? I feel like I’m asking ‘trivial’ questions that could be answered by thorough look in the source code. Before asking this question, I tried to check learn attributes to find the callback, and checked the source of learn.fit to see where the callback associated to callbacks_fn were saved, but without success. I also checked the doc of fit and callbacks but couldn’t find info on where callbacks_fn callbacks were saved. Do you have any advice on how to better check fastai source/doc while looking for this kind of information? I would love to answer those questions by myself and not waste your time.
In this case, it’s defined here, in the class
LearneCallback. By definition,
callback_fns are all
LearnerCallback since they take a learner when you create them, which is why it’s defined here.
Note that any other
LearnerCallback you define, even outside of
callback_fns becomes an attribute of the associated
I tried to access
LearnerTensorboardWriter callback with
learner.learner_tensorboard_writer but it said:
tai/script.py", line 40, in call_parse
File "examples/train_penn.py", line 104, in main
AttributeError: 'LanguageLearner' object has no attribute 'learner_tensorboard_writer'