Eight ways to debug in fastai

Part 1 (2020)
21

Thanks for taking the time to do these videos @slawekbiel I found them instructive and will be recommending them to my team as a quick intro to python debugging!

@Antoine.C I watched the whole video on 2x speed, stopping here and there for taking notes. Great stuff and totally worth it.

22

@butchland @slawekbiel @farid
Late to the party, but for future reference, I followed Farid Hussainiaā€™s blog:

3 ways to pip install a package - fastai2 use case

One difference (but not the only) with the recommended install is the use of

pip install -e .

instead of
pip install -e ".[dev]"

Farid explains why in this section of his post.

2 Likes
23

Good to know, Iā€™ll keep that in mind if I run into issues with nbdev

24

Thank you for your kind words! It makes me happy that people find it useful.

1 Like
25

Thank you @Antoine.C for sharing the blog post.

As a best practice, itā€™s good to install the editable version of both fastai2 and fastcore, and update them at the same time in order to have both packages aligned.

@slawekbiel, Thank you for sharing your videos. Iā€™m a heavy user of VS Code in both windows and Linux, I find it super useful despite the fact that its Jupyter notebook version is not on par with the official one. As a matter of fact, I wrote the entire timeseries package (see description here) using VS Code and the Python extension. I find it more practical to both be able working on a Jupyter notebook and easily navigate the source code. I have a fastai2 workspace that I always keep open so I can easily jump from my package to the fastai2 one.

26

Iā€™d like to hijack the thread and ask if anyone knows of a way to achieve what I think of as the ā€œholy grailā€ of debugging in datascienceā€¦ attaching the VSCode debugger onto a running jupyter/IPython kernel and following the call stack down for any particular RunCell.

To do this I have to copy a notebookā€™s content into a .py file and then launch in debug mode in vscode. This is suboptimal when Iā€™ve run cells out of order, or there is some object, like a learn.fit(), that takes a long time to run before reaching the breakpoint of interest.

27

I agree with you. Unfortunately, it looks like that option doesnā€™t exist yet based on my recent search: I hope Iā€™m wrong because I canā€™t wait to see that happen: That will be a game changer.

1 Like
28

HI. Do we have a way to put an pdb.set_trace() to fastai library without modifying its source code (the lib is not installed by pip -e). I find in fastcore the trace() function and its source code as below.

def trace(f):
    "Add `set_trace` to an existing function `f`"
    def _inner(*args,**kwargs):
        set_trace()
        return f(*args,**kwargs)
    return _inner

So Iā€™m thinking about kind of monkey-patching the original source code by putting trace as a decorator.

@trace
def func_to_debug(*args, **kwargs):
ā€¦

But really donā€™t know how to do it. Is it approach possible ? Thanks

29

With that you can just do:
ImageDataLoaders.from_dblock = trace(ImageDataLoaders.from_dblock)
and now this will break inside the function I substituted.

Alternatively you can invoke pdb earlier and then set a breakpoint with the b command. Like this:

(Pdb) b ImageDataLoaders.from_dblock
Breakpoint 1 at /notebooks/fastai2/fastai2/data/core.py:157
(Pdb) c
> /notebooks/fastai2/fastai2/data/core.py(159)from_dblock()
3 Likes
30

Great! Thanks, I will try it

31

I specially registered an account to clike a like! Thanks for the video, it really helps me a lot and improve me to debug deeper! Almost Iā€™m just ready to give upļ¼Œ but find someone try a lot to do the same thing and do better.