Lesson 9 Discussion & Wiki (2019)

Hi mods. May I know what is Jeremy’s typical process in debugging the issues that he encounters during his experimentation? Would love to see the process step by step.

I have a recollection that he mentioned this sometime before - but can’t find the reference somehow.

Thanks in advance.

6 Likes

… and LINQ.

I think this is where something like S4TF can be a big win.

I asked about it here, and I was indeed meaning to ask an update on it :slight_smile: so +1 on that!

1 Like

Why must the begin_fit method in the CallbackHandler class stop executing subsequent callbacks in the list if one of the prior ones returns False?

The execution will not stop, since the loop goes like for cb in self.cbs. We just record the fact that one of the functions returned False.

Also, I’m having trouble wrapping my mind around the concept of a Callback as a set of callbacks – I naturally think of “begin_fit”, “after_fit”, etc each as a separate callback. Why would you ever need to pass in two "begin_fit"s?

We may need to do more than one things at, say, begin_fit (i.e. when fit is called). Thus, you may want multiple callbacks per event.

3 Likes

That is the main reason we develop in Jupyter Notebook. Debugging or checking your outputs is super easy and that really boosts our productivity.
And the ipython debugger is great if needed!

2 Likes

Sounds like a fun exercise :slight_smile: . I still underestimate the power of Jupyter notebooks as documentation :).

I’m not sure about Jeremy … but one thing I do frequently is copy the pertinent fastai code into my own notebook, rename the methods/classes so it doesn’t conflict with the fastai code, and then I sprinkle pdb.set_trace() all over and look at the shape of things in particular as code runs.

5 Likes

What does Jeremy mean by “having the first batch right means everything is good”?

Did he mean that the first batch must have expected behaviour in terms of mean, std, loss, etc.? Or something more implicit that I’m mising?

He means that in the first batches of training, it’s crucial to have the beginning of training go well. That is why we begin with a low learning rate and warm up in 1cycle.

5 Likes

As far as I know, so far the schedules are set manually. Are you aware of any research that create a schedule based on the current state of the model? In order word, that determine the ‘phase’ the model is in by looking at it, and sets the annealing accordingly?

3 Likes

Is there a list of the available callbacks?

That would be cool wouldn’t it? That’s an area we’re researching, but no results yet.

3 Likes

Would we use a callback if we want to be setting schedules that depend on current or recent values/statistics of different layers, or is that something that requires a different approach?

Read the docs :wink: here

1 Like

Is there a way to write tests to test all these callbacks (using TDD)? Or you just try and hope it will work as expected for edge cases?

Not sure what you mean. It’s easy to write a test for a Callback to make sure it did what it was supposed to do.

Shameless plugin: if you’re a bit rusty with decorators, I’ve written a blog post on them :slight_smile:

11 Likes

This lecture is the most dense lecture of all fastai courses. I reckon it will take weeks to get a good grasp of all practices taught in the lesson. Pure gold!

8 Likes

And it’s only lesson 2 :wink:

3 Likes

In fastai do you write automated tests?