Plotting train/validation loss & accuracy while training model?

rohitc · April 16, 2018, 6:06am

Hi,
When using the Pytorch-based fastai library, is it possible to plot the training and validation losses and accuracy while the model is being trained? e.g., this would be similar to the live graphs in tensorboard that plot the training, validation loss and accuracy. I’ve found it very helpful to view the graphs during long running model training.

Thanks,
Rohit

Vishucyrus · April 16, 2018, 6:15am

There is a github repo I have come across VisualDL.

MicPie · May 25, 2018, 6:09pm

Hello,

is there an simple way to just have another curve for the validation loss with ‘learn.sched.plot_loss()’ to show train & val loss simultanously?

Or maybe even the metrics next to both losses?

I couldn’t find anything in the forums and the fastai source code.

Best regards
Michael

AlForML · June 26, 2018, 2:02pm

Same here! Still looking for a way to create these plots. Will make sure to post here if I figure it out

adrian · June 26, 2018, 2:25pm

I posted a way to do this in another thread you can find herehttp://forums.fast.ai/t/how-do-i-access-training-loss-when-running-learn-fit/17821

KarlH · June 26, 2018, 5:08pm

This notebook shows plotting of train/val loss and accuracy after running learn.fit.

github.com

sgugger/Deep-Learning/blob/master/Cyclical LR and momentums.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook uses the [fastai](https://github.com/fastai/fastai) library. Images from cifar 10 are assumed to be in the folder where PATH points, with a train and a valid subfolders, both of which contains the images from their classes in ten subfolders."
   ]
  },

This file has been truncated. show original

fig,ax = plt.subplots(2,1,figsize=(8,12))
ax[0].plot(list(range(95)),learn.sched.val_losses, label='Validation loss')
ax[0].plot(list(range(95)),[learn.sched.losses[i] for i in range(97,95*98,98)], label='Training 
loss')
ax[0].set_xlabel('Epoch')
ax[0].set_ylabel('Loss')
ax[0].legend(loc='upper right')
ax[1].plot(list(range(95)),learn.sched.rec_metrics)
ax[1].set_xlabel('Epoch')
ax[1].set_ylabel('Accuracy')

maxunterwegs · March 26, 2019, 10:53am

No one has posted fastAI’s dynamic ShowGraph callback function so far. I have been using it for the purpose of observing losses during training.
See: https://docs.fast.ai/train.html#ShowGraph

I have not plotted my accuracy during training, but you can certainly do that with callbacks. You may have to write your own.