This may answer to your question
The result of the cb.begin_fit(learn) call has been accumulated on this res variable. As you can notice res and cb.begin_fit(learn) is boolean operation and can be seen as res = ( res and cb.begin_fit(learn) ).
You can print, save, or do some manipulations inside callback. It is easy was to get values or terminate operation without changing library.
I didn’t study the callback system in detail yet so this answer is just based on intuition and might be completely wrong :):
My guess is that if someone wants to implement a different kind of fit() function via a callback, they could set cb.begin_fit(learn) to return True to avoid starting the standard fitting.
can someone help please understand this?
what is the purpose of super().__setattr__(k, v) ? If I remove it code crashes…
thanks
__setattr__ is a method that automatically ‘exists’ in a Python class when it is created. It is used by Python to set attributes, so something like self.name = name is done via the __setattr__ method.
When you override this method by explicitly defining it in your class, you need to call that original Python implementation too.
This is done by writing super().__setattr__(k, v).
If you don’t include this, Python has no way anymore to set an attribute in the class, so something like self.name = name will not work - that’s why your code crashes when you remove it.
If a real Python expert knows that this is the wrong explanation, please tell me 
Apparently one more way to achieve broadcasting
def log_softmax(x): return (x.exp()/(x.exp().sum(-1,keepdim=True))).log()
is equivalent of
def log_softmax(x): return (x.exp()/(x.exp().sum(-1).unsqueeze(-1))).log()
Is one way better over other?
The first one is probably marginally faster, since the 2nd one does the extra ops of squeeze+unsqueeze
https://pytorch.org/docs/stable/torch.html#torch.sum
torch.sum( input , dim , keepdim=False , dtype=None ) → TensorReturns the sum of each row of the
inputtensor in the given dimensiondim. Ifdimis a list of dimensions, reduce over all of them.If
keepdimisTrue, the output tensor is of the same size asinputexcept in the dimension(s)dimwhere it is of size 1. Otherwise,dimis squeezed (seetorch.squeeze()), resulting in the output tensor having 1 (orlen(dim)) fewer dimension(s).
Jeremy showed how LRScheduler function is working and I tried to make own function which multiply learning rate by two (I know that it is stupid but I did it to understand callbacks better) but there was a problem:
class MultiplyLR(Callback):
def begin_epoch(self, epoch, **kwargs):
self.learn.opt.lr = self.learn.opt.lr * 2
print("lr:",self.learn.opt.lr)
print("epoch:",epoch)
It said that
AttributeError: ‘SGD’ object has no attribute ‘lr’
I tried to look what parameters learn keeps inside but none of them seemed to be learning rate. What I’m doing wrong?
Also is lambda function a callback?
This is the newer version which Jeremy made. He said that in case something returns true it can be stopped right away but isn’t that causing a problem if later callbacks for example calculate stats? If first callback cause stop then the second callback (stats) is not called.
where are the notebook from the lesson stored?
Saw some weird behavior and couldn’t understand. I have two similar pieces of code doing parameter update. I see error in one and not in another. Can anyone help me understand why we observe this behavior
Can you try to replace the following line:
l.weight = l.weight - l.weight.grad*lr
with
l.weight.sub_(l.weight.grad*lr)
It might help.
As shown in the screenshots even
l.weight -= l.weight.gradlr
seems to work. However i’m confused on why that would work and not
l.weight = l.weight - l.weight.gradlr
It doesn’t block me or anything, but it is just weird
Thanks for the response.
weight is an nn.Parameter. So you need to assign it with an nn.Parameter. To avoid this properly, either use the data attribute (which is a tensor) or do inplace mutation.
@jeremy, if you’d like to replace the hardcoded cells in each dev_nbs such as:
!./notebook2script.py 02_fully_connected.ipynb
with just:
nb_auto_export()
add this into the first nb export, so that it’s imported everywhere.
from IPython.display import display, Javascript
def nb_auto_export():
display(Javascript("if (IPython.notebook.kernel) { IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name )}"))
or w/o needing to import anything, just this js magic cell:
%%javascript
if (IPython.notebook.kernel) {
IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name)
}
More details are here.
edit: had to add if (IPython.notebook.kernel) {} or it’d fail on nb load.
A simplest analogy to a callback in real life is an emergency phone number. If something happens an emergency phone number is used to notify someone who cares about the situation or can do something about it. There might be different phone numbers for different situations. If there is fire we call a fire fighter. If there is a water leak there is a plumber’s phone number. Phone number enables notification about the situation, but in any case, someone should make the call and someone should answer it. This way the caller can make things happen without knowing details about what would happen as the result of the call.
In our case, think about the function’s name as a phone number. When we start training, Fast.ai library
makes a courtesy call to a function named begin_fit. Boilerplate begin_fit function implemented in the library does nothing at all. If you want to do something special though, you can implement your own function called begin_fit and in the body of the function put all the custom code that you want to be executed before fitting begins. Fast.ai does many ‘courtesy calls’ at different points in the training loop. These courtesy calls enable flexibility and customization of the training loop.
I will do that!
I have annotated notebooks discussed in the Lesson 9 with hyperlinks to the time spots in YouTube video. They are available in my GitHub repo
Meanwhile I am trying to create a pull request for the annotated notebooks against the fastai_docs repository. I get an error while syncing with the original repository (see below).
Could someone help me getting past this error?
serge@mybox: git remote -v
origin https://github.com/sergeman/fastai_docs.git (fetch)
origin https://github.com/sergeman/fastai_docs.git (push)
upstream git@github.com:fastai/fastai_docs.git (fetch)
upstream git@github.com:fastai/fastai_docs.git (push)
serge@mybox:~/annotations/fastai_docs-fork$ git fetch upstream
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I get an error while syncing with the original repository (see below).
Please follow this guide: https://docs.fast.ai/dev/git.html#how-to-make-a-pull-request-pr
Most likely it’s because your protocols don’t match, one of them is https the other git, normally via ssh it looks like:
git remote -v
origin git@github.com:stas00/pytorch.git (fetch)
origin git@github.com:stas00/pytorch.git (push)
upstream git@github.com:pytorch/pytorch.git (fetch)
upstream git@github.com:pytorch/pytorch.git (push)
used pytorch example as I don’t have fastai_docs handy. but you can see the difference.
or perhaps make both of them https, I haven’t tried that way.