Hi!
I’m writing a custom callback and I want it to print some informations about training alongside the losses and metrics after each epoch.
On fastai_v1 it works just fine by using this line of code to add the values uniq
, val
and novel
def on_epoch_end(self, last_metrics, **kwargs):
`return add_metrics(last_metrics, [len(val)/self.num_samples, len(uniq)/self.num_samples, len(novel)/self.num_samples])`
But I’m struggling to do the samething using fastai_v2. Is there a similar function to add_metrics
?
I was able to modify the metrics_names attribute, but it’s only printing and not showing on the progress table.
def before_fit(self):
assert hasattr(self.learn, 'recorder')
self.recorder.metric_names = L('novel') + self.recorder.metric_names
print(self.recorder.metric_names)
(#7 ) [‘novel’,‘epoch’,‘train_loss’,‘valid_loss’,‘accuracy’,‘perplexity’,‘time’]
epoch
train_loss
valid_loss
accuracy
perplexity
time
0
3.285669
2.951922
0.216797
19.142704
00:08
Edit:
I think I’m getting closer!
I created a variable run_before = ProgressCallback right after the class definition. Now my result table looks like this:
epoch
train_loss
valid_loss
accuracy
perplexity
time
novel
0
2.824590
2.717227
0.311523
15.138282
00:09
1 Like
Kornel
(Kornel Dylski)
August 27, 2020, 5:13pm
2
metrics are stored in learner where are defined seter and getter for them
store_attr(self, "dls,model,opt_func,lr,splitter,model_dir,wd,wd_bn_bias,train_bn,metrics,moms")
self.training,self.create_mbar,self.logger,self.opt,self.cbs = False,True,print,None,L()
if loss_func is None:
loss_func = getattr(dls.train_ds, 'loss_func', None)
assert loss_func is not None, "Could not infer loss function from the data, please pass a loss function."
self.loss_func = loss_func
self.path = Path(path) if path is not None else getattr(dls, 'path', Path('.'))
self.add_cbs([(cb() if isinstance(cb, type) else cb) for cb in L(defaults.callbacks)+L(cbs)])
self.epoch,self.n_epoch,self.loss = 0,1,tensor(0.)
@property
def metrics(self): return self._metrics
@metrics.setter
def metrics(self,v): self._metrics = L(v).map(mk_metric)
def _grab_cbs(self, cb_cls): return L(cb for cb in self.cbs if isinstance(cb, cb_cls))
def add_cbs(self, cbs): L(cbs).map(self.add_cb)
def remove_cbs(self, cbs): L(cbs).map(self.remove_cb)
def add_cb(self, cb):
old = getattr(self, cb.name, None)
assert not old or isinstance(old, type(cb)), f"self.{cb.name} already registered"
so you only have to modify learn.metrics
property e.g. learn.metrics += your_metric_func
dont modify metric_names
, these are inited in recorder.before_fit()
method using learn.metric
(so you also should modify metrics somewhere before recorder.before_fit()
if you want recorder to show them properly)
1 Like