Indeed it was a great meeting! Thank you all for participating…
Now, let me ask each of you to share (here, in brief) one or more interesting things that you
heard at the Monday session.
We did not “take notes” of the round table, so it will be great if we can share here…
I recall a few things:
- Soumith question to us about what are the patterns of development/use that we follow.
- The (future?) support in Pytorch of “throw-away” part of the computation on the pass forward, and recompute it as needed in the backward pass, in order to release memory to enable bigger models.
- There are only 2 DL frameworks with momentum today, TensorFlow and PyTorch.
- How using an NVIDIA NVLink motherboard enables fast GPU-CPU memory exchanges by-passing PCI-e bandwidth bottleneck.
What other topics do you recall that we covered?