SOURCE CODE: Mid-Level API

Yup. That’s kind of what my goal is too!

1 Like

Also, one bit on this is I may step on your toes a little bit. I plan on doing an article on dblock.summary() (probably like an article and a half from now?) because it’s so powerful, and I’ll be introducing how to use it to debug your medium level stuff as well, hope that’s okay :slight_smile:

1 Like

It’s fine @muellerz. I’m not really worried all that much about who’d write about what. All I really want is to understand as much of the library as possible, and in the process possibly write what hasn’t been written about. It doesn’t matter if I don’t get to write about some particular aspect of it. :sweat_smile:

2 Likes

Great, I’m will surely read this. I have a suggestion: the font size is bit smaller and makes it harder to read the text on a laptop screen (but looks great on mobile), could you make it a bit larger?.

how do you all dig through the code? in notebooks and ?? each method? My first idea was to use VSCode and use the debugger / breakpoints, but I can’t get fastai to work on my MacBook. I think it would be way easier to set breakpoints and step through the code instead of doing it manually in the notebooks.

any advice on how to explore the code?

I personally use VS code on my MacBook only which I found to be the best way of all. Maybe vim is a better option but too much of a learning curve and thus I stuck to VS code.

Most of the tips are by following Jeremy’s advice itself: https://youtu.be/Z0ssNAbe81M?t=3809

In summary, here are the shortcuts most useful:

  • Search: cmd + t
  • Navigate to sub-command: f12
  • Returning back to where you originally were: ctrl + - (or ctrl + shift + - for going back forward)

Here’s a related topic on the forum as well: Examining source code with Visual Studio

3 Likes

Must be the FakeLoader bit.

I’ll join you. Can we maybe have a call tomorrow? I very thoroughly looked into DataLoaders and Transforms and also have a write up on TypedList from around 2-3 months ago. Would love to jump on a call and get back into this.

I use a combination of vim & tmux to step through code and Jupyter to interact with code.

Checkout lesson-3 here! https://missing.csail.mit.edu/

It’s a wonderful course by MIT on command line/ git/ vim/ editors etc :slight_smile:

3 Likes

Hey @arora_aman, thanks for the resource! Always wanted to learn vim properly but didn’t invest the time mainly due to lack of good suggestions on learning source for it! Will definitely check theirs out!

I’m playing around with it, it’s from the fastpages template so gotta adjust that css (and learn how to make it adaptable :wink: )

1 Like

Thanks for the video!

CMD + t doesn’t work unfortunately … I always get “no symbols found”. is there anything I have to do to get it to work?

Are you in the fastai2 directory in VS code?

yes, it doesn’t even find symbols in the same file …

That’s strange. Could you try doing a pip install of the fastai2 in vs code again since I believe you’re using its virtual environment where fastai2 isn’t available…

Forgot how great this is. Thanks for putting it here :slight_smile:

i had to enable workspace symbols in VSCode settings. Now it’s working :slight_smile: Thanks!

3 Likes

@florianl
Here’s what I personally do:

  1. Open up the fastai2 and fastcore folders in sublime text. (could be VS Code based on your preference)
  2. Open up a Jupyter notebook.
  3. Decide what to work on/understand and copy it’s source code into Jupyter.
  4. Read every line, put comments based on what I understand, and wade through the source code using sublime text (any IDE works) when needed.
  5. For the bits I find weird, I’d try and run something similar in the Jupyter notebook.
  6. I don’t always do a set_trace and follow through the source code. For me personally, I’ve had a lot of practice with just looking at code with parameters in my head and keeping a sort-of mental stack trace of where things are at the moment and I find this better for my purposes. Also, given the fact that there’s an insane amount of loops all over the place, I wouldn’t want to trouble myself with stepping through them in a debugger.

This ends up working very well for me. Of course, it can be different for different people.

6 Likes

Yes, very accurately guessed.

Sure, whenever you’re free.

1 Like

That’s definitely a neat approach of wading through the library’s code! :slight_smile:

One advantage of keeping a mental stack trace is that I don’t need to be on the server or Colab and be running things on a GPU. I have a fair idea of how arrays/tensors work on a GPU so I can just do this exercise on my MacBook.