The idea of the colorful dimension is to express with colors the mean and standard deviation of activations for each batch during training. Vertical axis represents a group (bin) of activation values. Each column in the horizontal axis is a batch. The colours represent how many activations for that batch have a value in that bin.
NB: Actual plot shown by Jeremy in the lesson are much more curated and focused. This post is about the original idea and my attempt to implement it.
DISCLAIMER: in the chart above I’ve manually superimposed the mean on the colorful dimension, to give an intuition of how the charts works.
Looking at the plots of notebook 06:
Jeremy asked us to find a “fancy” chart able to capture the distribution of activations during training.
Use the colorful dimension to visualize the distribution of activations during training.
The final plot for each layer is made by stacking the histogram of the activations from each batch along the horizontal axis. So each vertical slice in the visualisation represents the histogram of activations for a single batch. The color intensity corresponds to the height of the histogram, in other words the number of activations in each histogram bin.
CODE & EXPERIMENTS
The notebook is the first attempt to solve the problem
Try to have an understanding of what is going to happen looking at the charts.
Adapt the original callback to current fast.ai version to make it available.
Publish all the flash cards