Lesson 10 Discussion & Wiki (2019)

Part 2 (2019)
1

(This is a wiki post - please edit!)

Errata

  • The layer and instance norm code in the video use std instead of var. This is fixed in the notebook
  • I said binomial when I meant binary. Also shown incorrectly in the XL spreadsheet (now fixed).
  • The variance of a batch of one calculates to 0, not infinity. (With some technical exceptions.) Therefore BatchNorm would attempt to scale the filter to infinity.

Lesson resources

Papers mentioned this week

Notes and other resources

Other relevant papers

Papers for next week

12 Likes
2
5

pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html

replace cu100 with cu90 if you’re on cuda-9.0

All options are here: https://pytorch.org/get-started/locally/

2 Likes
16

Also, I guess the nvidia-smi shows some information about GPU drivers. (In case if you use some other library or framework).

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.39       Driver Version: 418.39       CUDA Version: 10.1     |
+-------------------------------+----------------------+----------------------+
1 Like
17

Any good resource for start reading about software engineering? Frankly, I find it bit overwhelming.

4 Likes
19

Even the slides are feeling a bit grey today with all this overwhelming content! :slight_smile:

4 Likes
22

I was actually thinking of proposing to do fastai.audio but I felt my skills are not enough to help implement it so I didn’t.

23

We are going to need a bigger model


1 Like
25

Is it advisable to get familiar with swift before lesson 13?

5 Likes
27

You don’t need to, the first lesson should be understandable without since Jeremy and Chris will cover the basics. If you have time to, it certainly won’t do any harm
 but there is also a lot of material in those lessons in PyTorch :wink:

3 Likes
28

The next course is very cutting edge and fun, great! But I am worried if we have enough time to finish all this

2 Likes
29

Thanks! Expanding on that, do you have any good ressources to start learning swift? Maybe there’s a thread about that on harebrain, haven’t been there a lot yet.

1 Like
31

Here’s a discussion of some resources Best book, podcast, tutorial a.o. for swift architecture & design.

4 Likes
32

The official book, more specifically its introduction is really great to discover all the basic functionality in Swift. Bonus: it can be opened in playgrounds on an ipad/mac if you have one and be interactive.

3 Likes
34

It is possible to have both PyTorch’s prepackaged CUDA and “manually” installed one? Is there a chance for a conflict between these two?

35

The tutorial notebooks on the GitHub page are a pretty good intro after you’ve gone through the Swift book.

5 Likes
37

If you try to use JIT (see later in the lesson) with a manually installed CUDA different than your PyTorch CUDA, there is going to be hell to pay.

2 Likes
38

Yes.

Is there a chance for a conflict between these two?

No. Assuming you’re using a prebuilt-pytorch.

If you’re building from source you will need the system-wide cuda

1 Like
40

I had some actually, but maybe it has been fixed by PyTorch since then.

41

Why does the callback function f used for widget takes an argument ‘o’?