Metric dependant gpu memory utilization

I noticed that gpu memory consumption is different depending on whether I use acc_camvid or dice as metric.
in fact with acc_camvid I run into a OOM error.
Should the metric (which I believe is calculated only for display purpose) be moved to cpu()?

I am trying to optimize the gpu memory usage (sacrifice speed a little if necessary)

Any comments?