On documenting the shapes/size of arrays/tensors being passed to methods of FastAI

Part 2 & Alumni (2018)
1

I know we already have some docs for the FastAI methods. But some of the methods manage to drive me spinning eternally.

So one of my suggestion is to document, whenever possible, at least the two pieces of information in all relevant methods:

  • Type of the collection being passed: Numpy array OR PyTorch Tensor
  • Shape if numpy.array, and size. if Pytorch.Tensor

In trying to develop and/or understand any neural-network architecture, knowing exactly the shape (size) of tensors being passed around have always helped me crucially. Much of the other integer/string parameters are simple to understand. The collections, on the other hand, can get really confusing with operation such as:

np.tile(np.stack(np.linspace(0,5,0.1)[-1, :3, :-2], tensor_no_idea_what_shape))

Take for example, a method from pascal-multi.ipynb:

def actn_to_bb(actn, anchors):
    actn_bbs = torch.tanh(actn)
    actn_centers = (actn_bbs[:,:2]/2 * grid_sizes) + anchors[:,:2]
    actn_hw = (actn_bbs[:,2:]/2+1) * anchors[:,2:]
    return hw2corners(actn_centers, actn_hw)

If this method is moved into a library python class, then it’d be really hard to know what this method was doing, without knowing, for instance (I don’t really know what they are, so kind of illustrating to the point) that

  • anchors was a numpy array of shape (num_x_grids, num_y_grids)
  • actn was a tensor of shape (batch_size, num_classes)

@jeremy I realize that it is easy to just instruct, and I apologize for that, but I think adding the two pieces of information for every method that you introduce will be a BIG help to everyone. I think it’s just a really good documentation model, and just doing so going forward would set a great precedence for all documentation forthcoming.

Thanks … and I do genuinely appreciate the course material.

3 Likes
2

Seems like you have some good ideas on documentation. Jeremy and Rachel are kicking off the new documentation project and your contributions will be very helpful.

3

Yes I think it would be very helpful too. The information is missing simply because no-one has gotten around to adding it yet. :slight_smile:

4

I appreciate your taking it kindly … thanks again :slight_smile:

I think FastAI is quickly in track of becoming an industry application, and not just an educational material. Good docs., some rigor in code-review, code-push, and testing etc. will be vital to its wide adoption and viability.

5

Thought experiment: What could dynamic tooling look like that would visually render and give intuition to these objects?

Perhaps in the form of an extension to Jupyter Notebook, etc …

6

I am not sure I fully understand - and I suspect your question might go further than this - but I think some of the functionality you are after lives in model.py in model_summary on line 157.

You can do learn.summary() and it should render dimension information.

7

@britt
I’m not sure if you mean the ability to inspect these elements on the fly, as the code is executing. That would mean setting a debug breakpoint, and pausing while doing a manual inspection.

I don’t think a static approach would work though. By its nature, Python is dynamically typed, and the shape as well as the dimensions of some of these tensors (or np.arrays) could greatly vary.

8

Yes! I find math best represented visually so I imagine some sort of augmented debugger that allows one to hover over elements as they run and see the geometric representation, or augment summary() to output something akin to what @chloews did with her visualization: http://forums.fast.ai/t/part-2-lesson-9-in-class/14028/375

9

I’m a very visual thinker, but I don’t see how a geom representation of the shape would be helpful. I guess maybe just seeing that it’s tall or wide or deep visually is helpful? But perhaps more important is just having those shapes shown in a more compact way so you can see them all at once? One obvious way to do that is to have a method that only prints out the layers where the shape changes. I think that could be nice.

10

That’s the plan. With all the great software engineers working through the program, I think we’ll have plenty of strong contributions to all these areas.

1 Like