Lesson 10 Discussion & Wiki (2019)

(Jeremy Howard (Admin)) #1

(This is a wiki post - please edit!)

Errata

  • The layer and instance norm code in the video use std instead of var. This is fixed in the notebook
  • I said binomial when I meant binary. Also shown incorrectly in the XL spreadsheet (now fixed).
  • The variance of a batch of one calculates to 0, not infinity. (With some technical exceptions.) Therefore BatchNorm would attempt to scale the filter to infinity.

Lesson resources

Papers mentioned this week

Notes and other resources

Other relevant papers

Papers for next week

11 Likes

Lesson 8 (2019) discussion & wiki
2019 Part 2 Lessons, Links and Updates
Lesson 12 (2019) discussion and wiki
Issue with 07_batchnorm.ipynb
(Jeremy Howard (Admin)) pinned #2
0 Likes

(Stas Bekman) #5

pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html

replace cu100 with cu90 if you’re on cuda-9.0

All options are here: https://pytorch.org/get-started/locally/

2 Likes

(Ilia) #16

Also, I guess the nvidia-smi shows some information about GPU drivers. (In case if you use some other library or framework).

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.39       Driver Version: 418.39       CUDA Version: 10.1     |
+-------------------------------+----------------------+----------------------+
1 Like

#17

Any good resource for start reading about software engineering? Frankly, I find it bit overwhelming.

5 Likes

(Nalini) #19

Even the slides are feeling a bit grey today with all this overwhelming content! :slight_smile:

3 Likes

(Maxim Pechyonkin) #22

I was actually thinking of proposing to do fastai.audio but I felt my skills are not enough to help implement it so I didn’t.

0 Likes

(Ariel Gamiño) #23

We are going to need a bigger model…

1 Like

(Pierre Ouannes) #25

Is it advisable to get familiar with swift before lesson 13?

5 Likes

#27

You don’t need to, the first lesson should be understandable without since Jeremy and Chris will cover the basics. If you have time to, it certainly won’t do any harm… but there is also a lot of material in those lessons in PyTorch :wink:

3 Likes

(Charm) #28

The next course is very cutting edge and fun, great! But I am worried if we have enough time to finish all this?

2 Likes

(Pierre Ouannes) #29

Thanks! Expanding on that, do you have any good ressources to start learning swift? Maybe there’s a thread about that on harebrain, haven’t been there a lot yet.

1 Like

(Zarak) #31

Here’s a discussion of some resources Best book, podcast, tutorial a.o. for swift architecture & design.

4 Likes

#32

The official book, more specifically its introduction is really great to discover all the basic functionality in Swift. Bonus: it can be opened in playgrounds on an ipad/mac if you have one and be interactive.

3 Likes

(Ilia) #34

It is possible to have both PyTorch’s prepackaged CUDA and “manually” installed one? Is there a chance for a conflict between these two?

0 Likes

(Walter Wiggins) #35

The tutorial notebooks on the GitHub page are a pretty good intro after you’ve gone through the Swift book.

5 Likes

#37

If you try to use JIT (see later in the lesson) with a manually installed CUDA different than your PyTorch CUDA, there is going to be hell to pay.

2 Likes

(Stas Bekman) #38

Yes.

Is there a chance for a conflict between these two?

No. Assuming you’re using a prebuilt-pytorch.

If you’re building from source you will need the system-wide cuda

1 Like

#40

I had some actually, but maybe it has been fixed by PyTorch since then.

0 Likes

(Manan Sanghi) #41

Why does the callback function f used for widget takes an argument ‘o’?

0 Likes