Lesson 11 discussion and wiki

(Jeremy Howard (Admin)) #1

Lesson resources

Papers

Resources

Notes

12 Likes

2019 Part 2 Lessons, Links and Updates
Great article on the interaction between batchnorm and l2 regularization discussed in lesson 11
#19

Could I get a refresher on what callbacks and hooks were?

0 Likes

#20

This is the focus of the last lesson. The video is accessible in the general announcements thread.

0 Likes

(hector) #21

How does lsuv work on test/val data? do we still adjust mean and std parameters on test data?

0 Likes

(Kevin Bird) #22

Is there a reason we use standard deviation instead of Mean Absolute Deviation?

1 Like

(benjmann) #23

So just to confirm, LSUV is something you run on all the layers once at the beginning, not during training? What if your batch size is small, could you overfit to that batch?

5 Likes

(hari rajeev) #25

possible to give an overview of lsuv again ?(high level)

1 Like

(Maxim Pechyonkin) #26

I want to hear Jeremy pronounce Imagenette in French accent!

3 Likes

#27

It’s only the initialization of your network. Then, you train it.

3 Likes

(Brian Smith) #28
6 Likes

#29

Wait for it :wink:

2 Likes

(Josh Varty) #30

Even though it’s live you can always go back in time on the video if you missed something.

2 Likes

(Tanya Roosta) #31

but is it better than say Kaiming?

2 Likes

#32

Yes, it trains better, especially for a deeper network.

8 Likes

(Andrea La Rosa) #33

Interesting about the size of the images affecting what works and what doesn’t. What’s the intuition behind?
Does it mean that some of the operations don’t really “scale” with size?

0 Likes

(Ilia) #35

Probably the resolution is too low to learn good filters for full-size images?

1 Like

#36

The network sees less (or more) details, so it can’t get to the same results.

2 Likes

(Tanya Roosta) #37

you got it :slight_smile:

0 Likes

(Andrea La Rosa) #39

Oh ok - so it’s really the size of the objects, rather than the size of the images per se

0 Likes

(hector) #40

Ah! got it now…thought the readjustment will done after every epoch/few iterations…thanks

0 Likes