Beginner: Basics of fastai, PyTorch, numpy, etc ✅

Thanks :slight_smile:

In addition to using documentation website you can also type:

URLs??

in Jupyter nnotebook. This will print docstring and sourcecode for the URLs. Can be useful for fast exploration.

3 Likes

Very true, thanks for the tip!!

I think the datablock api bit confusing, and get_items splitter get_y get_x are not part of the constructor and an intellisense does not show those input parameters.

I don’t follow. They are part of the constructor, they appear in doc(), they’re shown in the docs, and they appear in the intellisense for me (using Jupyter Notebook).

What are you finding confusing about these parameters? Have you tried the tutorial?

2 Likes

Thank you for the response. I did not use doc function I was using control+space to call the intellisens:
image
I am going to use doc function going forward.

That’s interesting - the platform you’re using isn’t using our delegates functionality correctly, so the kwargs aren’t being shown. Instead it’s just showing **kwargs. What platform is that which you’re using there?

1 Like

n/m I figured it out - it’s Colab. I just tested, and you’re right, at first it doesn’t show all the arguments. But I noticed if I wait a couple of seconds then it does, at the bottom:

1 Like

Shravan Kumar:
​when we are running the finetuning using fine_tune(x) . is it necessary that last epoch specified should have less error metric? Until we reach this best epoch; do we need to keep on running?

So essentially you are asking when is the model best fit. The idea is the valid loss should decrease along with the training loss. The best model would be the model with the least valid loss

Generally you want to keep training until your validation loss stops improving or until you run out of time. Sometimes the last epoch does not contain the best validation loss or metric and there are methods for saving the model as training is happening automatically as long as the validation loss is improving using the SaveModelCallback.

6 Likes

Is #export app which jeremy showed just now, part of nbdev?

2 Likes

yes it is.

1 Like

Yes, when specified, will export the cells to python file

1 Like

Can some experienced users please list strategies to browse through the fastai code? I’m looking for suggestions on where to start and what are the best tools to use?

Do you recommend just cloning the repo and then grepping around (but these are ipynb files) or would it be better to load the cloned repo directory in VSCODE and use some extensions which would help in explorations (like I see some @function and I want to know what it does but instead of looking at all the imports, I just click on it and it takes me to that function?)

I’m sorry if there are obvious solutions to this, I don’t have a developer background and as much as I want to dig through the code, I’m find it a little bit overwhelming to figure out where to start as I feel I need to go deeper into every definition as I know nothing about the framework at all.

2 Likes

@mike.moloch

Few steps that worked for me during different scenarios

When I am experimenting and reproducing the notebook, I try to stick to the notebook environment itself and before using a fastai function, I run doc(FUNC_NAME) and try to understand what each parameter does. As Jeremey mentioned in the lecture ?? FUNC_NAME is your friend to read through the code.

Below is a recent discovery from rich (wrapper over python inspect module) , gives a nice summary of a module/object etc

from rich import inspect as rinspect
rinspect(OBJECT) 

When I need to understand a module, I typically use docs.fast.ai and it has source linked from the docs. I also clone fastai repo to my local and use vscode to jump around the fastai module. Make sure you also clone fastcore since lots of goodies from this library are used in fastai library.

9 Likes

Thanks @msivanes , I really like the rinspect() function, this is exactly the kind of thing I was looking for.

I do try to lookup functions used in a lesson notebook via doc() and ?? , but sometimes there’s this rabbithole you need to descend into and it’s hard to figure out at what abstraction level one should remain. I thought maybe I’d start with the tutorials section but I’ve also seen advice on reading through the library code to understand what might actually be going on. So the rich.inspect() functionality would come in super handy!

Cheers!

Multiple ways to go about this, so you’ll end up getting a lot of different-but-similar advice from different people. So, I’m just going to write down how I personally approach this. (WALL OF TEXT ALERT, sorry ! this is usually easier to demo than to write down about.)

Fastai + Jupyter live inspection method

One thing Jeremy has already demo-ed is the usage of ?, ?? & doc in the Jupyter environment. These provide a pretty good place to start, so a good idea to get comfortable with those. Calling the doc on a python object can actually point you to the source in fastai repositories directly. Try running the following in your jupyter environment. ( after importing from fastai.vision.all import * )

  • ?get_image_files
  • ??get_image_files
  • doc(get_image_files) ← This points you to the exact lines in the fastai repo

Searching in a codebase via editor / command line method

The other method (in my case) is to approach the codebase like I do with any other one. There’s usually two main stages (and appropriate tools for them)

Stage 1. Searching for a keyword / idea

In this phase, I use tools that can quickly find all entries of a text/pattern I’m looking for. I use a tool called rg, but the idea is similar to other tools like ag & grep, or search built-in to the editor you use.

You use this tool to narrow down your search space and find things you might be looking for. If you don’t know the codebase at all, there’ll be some guessing involved. (eg. rg adam, rg Adam, rg optimizer, rg opt and so on…)

Your editor might already have built in support for searching for keywords. So, you can just start with that, no need for external tools really. As long as it’s fast, it’ll work fine. For eg. Search across files in vscode.

NOTE : For the fastai repo, you want your searches to be done not at the top level, but inside the fastai folder. That way you exclude results from other than the pure python files.

Stage 2. Load the file in your editor and start “jumping” to definitions

In this phase I open the corresponding file in my editor, get to the line/col where I was searching the above term. Now I just try to “walk” the codebase.

When there’s a new function/variable that I want to know more about, I try to “jump to” where it was defined. Once I understand it, I “jump back” to where I came from. This has to be supported by your editor. If you’re using vscode, Go To Definition & Go Back is what you’re looking for. I don’t use VSCode a lot yet, so hopefully somebody else can chime in more on that.

Either ways, taking the time to
a) learn how to find the keywords, and
b) learn how to “jump in” and “jump back out” using your editor+language
can be really useful if you plan to have an easier time browsing a codebase.

I hope this was somewhat helpful. :raised_hands:

12 Likes

Actually, I really appreciate you taking the time to answer this in detail (as usual) , so I welcome this (and I’m sure a lot of other beginner types like me would appreciate your answer) :smiley:

I am more comfortable with linux commandline tools so rg/ag would be great to play with. I use grep extensively for my day to day work anyway.

I think I’ll use either a tutorial or a chapter notebook as the starting point and dig through code that way instead of starting with 00_torch_core.ipynb and going down the list because that might just be way over my head.

Thanks!

4 Likes

I’d recommend finishing reading the book before diving too far into the library code, because in the book you’ll recreate many of the key classes in fastai from scratch.

Once you’ve done that the fastai code will make much more sense since you’ll know what each bit is doing.

13 Likes

It’s great @suvash! I add just one technique that I found super useful for experimenting, Installing an editable version of a library with pip install -e then add the auto-reload in notebook with %load_ext autoreload %autoreload 2. With this you are free to add your code in the library, or add break point, …

6 Likes