Patch_to code walkthrough

fastai fastai dev
1

When I initially saw patch_to in the 01_core notebook, it seems super daunting and I didn’t understand what it was doing so I decided to write a post to say what it is doing and hopefully have others that understand it better, correct me if I misunderstand any of it.

Here is the code we will be breaking down in this post:

#export
def patch_to(cls, as_prop=False):
    "Decorator: add `f` to `cls`"
    def _inner(f):
        nf = copy(f)
        # `functools.update_wrapper` when passing patched function to `Pipeline`, so we do it manually
        for o in functools.WRAPPER_ASSIGNMENTS: setattr(nf, o, getattr(f,o))
        nf.__qualname__ = f"{cls.__name__}.{f.__name__}"
        setattr(cls, f.__name__, property(nf) if as_prop else nf)
        return f
    return _inner

This creates what is called a decorator. It can be referred to using an @ and will run code before and after the function it is decorating runs.

The purpose of patch_to is to take a class and add a new function to the class. So to use it, first identify what class you want to add a function to. Then, pass that class to the decorator and put a function directly below the decorator like so:

@patch_to(_T3)
def func1(self,x): return x+2

Now, remember a decorator runs on the code that follows it and it has the ability to process code before and after the code

So to orient ourselves with the actual code, _T3 now is cls in the code from 01_core and the function func1 is f.

So first, a copy of func1 (f) is created called nf.

Next, the variable WRAPPER_ASSIGNEMENTS from the library functools is iterated through.

This is what WRAPPER ASSIGNEMNTS contains:

('__module__', '__name__', '__qualname__', '__doc__', '__annotations__')

So each of these is going to be put into o and looped through setattr.

This is copying all of the attributes from f using getattr(f,o) and putting them in the copy of f, nf.

Then, __qualname__ is updated to be {class.__name__}.{function.__name__} in this case: _T3.func1

Lastly, this function is added to the class using setattr. If as_prop is set to True, a property function wraps nf, otherwise, it is passed directly.

I think I finally have a decent idea how decorators work and especially how this specific decorator works to add a function to a class.

I’m hoping that by sharing my thought process going through this, somebody else will gain a deeper understanding of this code and also I will get feedback on any issue I may have had in my explanation.

10 Likes
2

Great! I’ll add a link to this in the core topic.

1 Like
3

In practice we almost never use patch_to, but instead use patch - so you might want to add an explanation of that to your post too. You may also want to mention functools.update_wrapper since we’re basically copying its functionality in the setattr loop. Finally, I’d suggest describing what as_prop is doing in practice, and why.

Feel free to ask if you have any questions, of course!

4

Great initiative!
I have the same question, thats why I get to this post
But I cant access the link (the one attached in Here is the code we will be breaking down in this post).
@KevinB Do you have an updated link available?

5

It looks like it is at this location now:

1 Like
6

@KevinB
Awesome thanks! It helps
Additionally going back to patch_to, I am a bit confused as in why f is not put as an argument in patch_to because from my understanding a decorator should take in the function to be wrapped as an argument too

For example, the example you provided should be implicitly work like this:
func1 = patch_to(cls = _T3, f = func1)

but instead it seems to work like this:
func1 = patch_to(cls = _T3)(func1)

which contradicts to my understanding of how a decorator should work

7

patch_to returns a func, it still needs to be applied (that return _inner)

8

Could I say that patch returns the function that has been altered by the wrapper (arguments are same as f's), while patch_to returns the wrapper that is going to alter the function (argument is f itself)?

9

Thats how I understand it from the source code too. And the wrapper _inner returned from patch_to doesnt have the same arguments as f but it takes f itself as an argument. So _inner has to be additionally applied to f in order to register f into the target class.

However I encountered a scenario that seems to be at odd with this (see this thread)
In the thread above, create is decorated by patch_to(PILBase, cls_method=True) and then it works as if the newly defined create has been registered in PILBase.

So it seems to me @patch_to(PILBase, cls_method=True) is doing this underneath the hood:
(i.e. the decorator operation @patch_to doesnt only return the wrapper but also do one more step to apply it on the concerned function):

create = patch_to(PILBase, cls_method=True)(create)

Shouldnt a decorator operation works like the following instead?

create = patch_to(PILBase, cls_method=True)
10

OK, I finally resolved the confusion here. I was confused in how a decorator WITHOUT argument operates v.s. a decorator WITH arguments. Turns out they operate differently, in summary:

when a decorator doesnt have argument. It operates like this:

foo = decorator(foo)

However when a decorator has arguments, it operates like this:

foo = decorator_with_args(args)(foo)
# I mistakenly thought it works like this:
foo = decorator_with_args(args)

so a decorator with arguments serves like a factory that returns a decorator, and then the returned decorator consequently applies on the function.

The following reference helps resolves the confusion:

3 Likes