Lesson 10 Discussion & Wiki (2019)

We are going to need a bigger model…

1 Like

Is it advisable to get familiar with swift before lesson 13?

5 Likes

You don’t need to, the first lesson should be understandable without since Jeremy and Chris will cover the basics. If you have time to, it certainly won’t do any harm… but there is also a lot of material in those lessons in PyTorch :wink:

3 Likes

The next course is very cutting edge and fun, great! But I am worried if we have enough time to finish all this?

2 Likes

Thanks! Expanding on that, do you have any good ressources to start learning swift? Maybe there’s a thread about that on harebrain, haven’t been there a lot yet.

1 Like

Here’s a discussion of some resources Best book, podcast, tutorial a.o. for swift architecture & design.

4 Likes

The official book, more specifically its introduction is really great to discover all the basic functionality in Swift. Bonus: it can be opened in playgrounds on an ipad/mac if you have one and be interactive.

3 Likes

It is possible to have both PyTorch’s prepackaged CUDA and “manually” installed one? Is there a chance for a conflict between these two?

The tutorial notebooks on the GitHub page are a pretty good intro after you’ve gone through the Swift book.

5 Likes

If you try to use JIT (see later in the lesson) with a manually installed CUDA different than your PyTorch CUDA, there is going to be hell to pay.

2 Likes

Yes.

Is there a chance for a conflict between these two?

No. Assuming you’re using a prebuilt-pytorch.

If you’re building from source you will need the system-wide cuda

1 Like

I had some actually, but maybe it has been fixed by PyTorch since then.

Why does the callback function f used for widget takes an argument ‘o’?

good opportunity for using python partials

1 Like

That’s a great idea to start from simple things and models, and datasets at first. It is so overwhelming to write new code with huge datasets and cutting edge models. Starting simple gives you a feeling of progress and helps to make things work. And, this approach with callbacks allows one to do exactly this: gradually build the training code and models from basic to sophisticated.

3 Likes

In looking at these Python callbacks, I’m curious how much the inclusion of lots of unused callbacks can slow down training. Is it negligible to check all these if statements on every batch?

1 Like

@tamhash
If you’re looking for a book The Pragmatic Programmer is a solid choice that is easy to walk through

5 Likes

Most conflicts happen when loading other python packages built against conflicting versions of MKL and alikes. e.g. currently OSX has a problem with numpy and pytorch pip packages built against different MKL builds that conflict.

pytorch will loads its own cuda+cudnn libraries, so the conflict can only happen if you have some other python component loaded that is linked against a different version of the same library.

1 Like

The Clean Code is also an interesting thing, I would say. The classic books :smile:

5 Likes

I think that’s just how the callback there is structured. If you modify f to also print out o, you’ll see that you get a Button object: <class 'ipywidgets.widgets.widget_button.Button'>. Maybe this is in case you want f to depend on button attributes.