Visualizing your network in PyTorch

[radek]

I read the awesome write up from @groverpr on Taxi Trajectory Winner Solution. I also seem to encounter this a lot myself when working on posts.

Using text or code to describe an architecture is great, but having a diagram would be extremely beneficial to the readers. But how do you create those diagrams?

I tried tensorboard for pytorch but it is not easy to get it to work with the fastai library (had some issue passing the data to add_graph of SummaryWriter). Also, some of the graphs from tensorboard (the ones in the demo for example) look really poor.

What do people use for papers? I was tempted to learn to use graphviz or networkx but not sure how much mileage I would get out of either and also don’t have time to look into this atm. Has anyone given this a try?

Would be great if there existed a solution that was easy to pick up and produced nice results

I love the graphics from Jeremy in lesson 6 + 7 for the RNN architecture (copied them into a notebook I am planning on sharing if Jeremy doesn’t mind , ofc with proper attribution) but I think they were created in Excel? I do not have access to Windows let alone MS Office so that is not an option for me…

[machinethink (Matthijs)]

I use this code (but tweaked a little): https://github.com/szagoruyko/functional-zoo/blob/master/visualize.py

Since PyTorch is a dynamic framework there isn’t really a graph like in TensorFlow / Keras. There is only the graph that was created when you did some computation. In the case of a neural network, that is the computations for when you did a forward pass.

The make_dot() function from that source code takes the output of your NN (such as the loss) and then draws the graph that was used to compute that loss. However, it only knows about the low-level PyTorch operations, so you may get more details than you want.

[groverpr (Prince Grover)]

That’s an interesting topic. I would definitely explore various options for making diagrams.

You are right. We should at-least incorporate diagrams made in the MS Word, if not from any library, like one in below picture.

[rikiya (Rikiya Yamashita)]

I just recently found this page helpful. Ian Goodfellow used OmniGraffle for his Deep Learning book. Edraw Max also looks great to me. For now I’m wondering whether to take, OmniGraffle or Edraw Max (not for free though ). I would like to even try Graphviz or TikZ/LaTeX when I have time in the near future.

[waleedka (Waleed)]

I researched this a lot, and all the tools I found generate huge graphs because they draw every single node, which gets big fast if you visualize a large network like ResNet. I ended up writing my own tool to solve this. It detects repeated blocks and combines them into one node, and also merges layers that tend to come together into one box as well (e.g. Conv/RELU/BatchNorm). I hope others find it useful. See it here: HiddenLayer.

[angelinayy]

this is SO COOL! apologize for not researching enough but throw a question: I suppose I can use this lib with my fastai leaner generated network? Can I?
Thank you!

[waleedka (Waleed)]

Yes, it’s under MIT license, which means you can use it or modify it for any purpose, personal or commercial. No restrictions at all.

[parthi2929 (Parthiban)]

Learn and use Tikz. Had been a standard choice especially for academic papers for many years. When tikz becomes too difficult, use matplotlib/plotly/graphviz/networkx depending on need basis. Start small.

[iNLyze (Constantin)]

Alternative to Tikz for plotting CNNs in LATEX: PlotNeuralNet