Trying to use IceVision together with FastAI for my object detection project using labeled bounding boxes.
IceVision has a COCOMetric Wrapper around COCO API to capture one of the metrics and returns a dictionary i.e. { “mAP”: mAP_score }
I thought I can just use SaveModelCallback(monitor=‘COCOMetric’) or SaveModelCallback(monitor=‘mAP’) to save the best performing model but I encounter these exception:
== when using monitor=‘COCOMetric’ ==
fastai/callback/tracker.py...
---> 40 if self.comp(val - self.min_delta, self.best):
...
TypeError: unsupported operand type(s) for -: 'dict' and 'float'
== when using monitor=‘mAP’ ==
fastai/callback/tracker.py in before_fit(self)
...
---> 34 assert self.monitor in self.recorder.metric_names[1:]
...
AssertionError:
How should I get at the mAP number inside the COCOMetric dict? Is my only option to write a wrapper metric around COCOMetric e.g. COCOmAPMetric which just retrieve and return the value from inside the dict?