We are going to need a bigger model…
Is it advisable to get familiar with swift before lesson 13?
You don’t need to, the first lesson should be understandable without since Jeremy and Chris will cover the basics. If you have time to, it certainly won’t do any harm… but there is also a lot of material in those lessons in PyTorch
The next course is very cutting edge and fun, great! But I am worried if we have enough time to finish all this?
Thanks! Expanding on that, do you have any good ressources to start learning swift? Maybe there’s a thread about that on harebrain, haven’t been there a lot yet.
Here’s a discussion of some resources Best book, podcast, tutorial a.o. for swift architecture & design.
The official book, more specifically its introduction is really great to discover all the basic functionality in Swift. Bonus: it can be opened in playgrounds on an ipad/mac if you have one and be interactive.
It is possible to have both PyTorch’s prepackaged CUDA and “manually” installed one? Is there a chance for a conflict between these two?
The tutorial notebooks on the GitHub page are a pretty good intro after you’ve gone through the Swift book.
If you try to use JIT (see later in the lesson) with a manually installed CUDA different than your PyTorch CUDA, there is going to be hell to pay.
Yes.
Is there a chance for a conflict between these two?
No. Assuming you’re using a prebuilt-pytorch.
If you’re building from source you will need the system-wide cuda
I had some actually, but maybe it has been fixed by PyTorch since then.
Why does the callback function f used for widget takes an argument ‘o’?
good opportunity for using python partials
That’s a great idea to start from simple things and models, and datasets at first. It is so overwhelming to write new code with huge datasets and cutting edge models. Starting simple gives you a feeling of progress and helps to make things work. And, this approach with callbacks allows one to do exactly this: gradually build the training code and models from basic to sophisticated.
In looking at these Python callbacks, I’m curious how much the inclusion of lots of unused callbacks can slow down training. Is it negligible to check all these if statements on every batch?
@tamhash
If you’re looking for a book The Pragmatic Programmer is a solid choice that is easy to walk through
Most conflicts happen when loading other python packages built against conflicting versions of MKL and alikes. e.g. currently OSX has a problem with numpy and pytorch pip packages built against different MKL builds that conflict.
pytorch will loads its own cuda+cudnn libraries, so the conflict can only happen if you have some other python component loaded that is linked against a different version of the same library.
I think that’s just how the callback there is structured. If you modify f
to also print out o
, you’ll see that you get a Button object: <class 'ipywidgets.widgets.widget_button.Button'>
. Maybe this is in case you want f
to depend on button attributes.