Why is passing a mini-batch to a model "really important for debugging"

Part 1 (2020)
1

Hi everyone,

Chapter 6 of the book says:

“note: Getting Model Activations: Knowing how to manually get a mini-batch and pass it into a model, and look at the activations and loss, is really important for debugging your model. It is also very helpful for learning, so that you can see exactly what is going on.”

Q. Why is passing a mini-batch to a model “really important for debugging”? Would you describe some production examples to show this?

Best,

Tom

2

I think the point is you know how to get a batch from your DataLoader and running your model from there.

Like one_batch in fastai or x, y = next(iter(dataloader)) in general.

then you run model(x) to understand what is exactly going on in your network. For example, you have a bug in your model, by doing that, you can run each layer separately ( model.resblock(x) ) and know at which layer the bug locate.

Or the problem can be more subtle, after a layer, you get all activations is zeros. If you can control everything in your pipeline, you have more chance to see where is the problems.

Hope it helps

1 Like