More on the colorful dimension:
The following picture shows the steps involved in the computation of histograms. All these steps are needed to create a **single chart column", such as the first column of the chart is the histogram of the first batch and so on.
- GET A BATCH OF ACTIVATIONS: we usually analyze the activations after a specific layer/transformation. The generic shape is [BS,??]. For example the result of a 2d convolution is [BS,NF,XX,YY], and the result of a Linear layer is [BS,NF] where:
- BS: batch size.
- NF: number of features/filters/channels.
- XX,YY: size of “image”.
- COMPUTE HISTOGRAM: we compute the histogram of the whole activations batch. The parameters of our histogram are:
- hMin,hMax: minimum and maximum values reported in our histogram.
- nBins: number of bins, the “resolution” of the histogram.
- LOG THE HISTOGRAM: Note that this transformation is made only for visual purpose and not applied to the stored histogram values.
Original Idea:
This was my very first sketch made to visualize the initial idea of the “colorful dimension”.
Some anticipations on the next post:
I’m working on a complete rewrite/improvement of the colorful dimension chart.
Here are some anticipation (this picture usually means bad training, but it’s cool 
)
