Using the callback system in fastai

Part 2 & Alumni (2018)
1

Hi everyone,

I’ve put down a notebook to explore the system of callbacks in the fastai library, which I think is pretty amazing. It lets you customize your training completely as you want.

In particular, I show how to do three different kind of things:

  • record some data linked to the training (here validation loss at each epoch)
  • change the hyper-parameters when some condition is met (here dividing the learning rate by 10 each time the validation loss doesn’t get lower) I do not endorse that specific training policy, I just chose that example because it’s a very popular one.
  • make some parameters change as the training goes (here have the p of a dropout layer vary through training)

I hope you find it useful!

28 Likes
2

I’m going through this now… so that’s what callbacks are, this is great!

3

Great! Great! Great! Thank you!

4

It’s really great… Thanks Sylvian

5

Thanks for the knowledge, this is insightful. Now figuring out how to use this myself…

6

This is awesome. Callbacks were on my list of things to study. Looking forward to diving into the notebook.

1 Like
7

Hi @sgugger,

Is there a way to stop the learner if the learner’s metric (e.g. accuracy) is below or above a threshold value?

BR, Kimmo

8

Yes, if the on_epoch_end function of a Callback returns true, it will stop the training. So a very simple Callback with a test in this function should work just fine.

1 Like
9

I tried it and it works!

Thanks!!!

10

great