Lesson 8 (2019) discussion & wiki

Part 2 (2019)
253

Why is yhat a result of mse? I thought yhat was the output of lin2, and mse was only a way of measuring how good our predictor is.

254

and those propagate back to outside the function? (so, classes are passed by reference?)

255

What is .clone() exactly doing?

1 Like
256

That’s python for you, most objects are mutable.

2 Likes
257

I’ve run into this elsewhere in python (think it was scikit-learn, but can’t remember specifically now) and was totally confused by the transposition. I still don’t understand it, so if someone can explain, that’d be awesome.

258

the model has layers referenced in its constructor. if we mutate the referenced class, we can access the value we’ve added to it (as .g) from within the main class (model)

3 Likes
259

From what I read, looks like the idea is to head toward a language that integrates deep learning / numerical computation / differentiable programming. So the idea is that Swift will grow out of the “… for TF” part. I think. Anyone chime in who knows more.

260

It’s copying your tensor during assignment. Otherwise it would just copy the reference to that tensor and not the actual object.

5 Likes
261

Just to confirm, the “.clamp_min(0.)-0.5” is part of the tweaking?

1 Like
262

I believe so, yes

263

I hadn’t noticed this fixup initialization paper until Jeremy mentioned it today and it looks very interesting. I haven’t fully grasped that paper yet, but does anyone know if some of these ideas or related ideas could apply to RNNs as well? Are there any other ways to improve over the LSTM/GRU (or antisymmetric RNN)?

264

Are __call__ and __init__ completely independent? For example if I have a __init__ variable, would it be available in a __call__

265

Yes, regular ReLU doesn’t have the -0.5

2 Likes
266

So it’s like copy.deepcopy()?

267

But for PyTorch’s tensors.

268

What’s the intuition behind it?

269

Rewatch the video tomorrow, Jeremy explained it :wink:

1 Like
270

init is called when you instantiate a variable (which you would have to do before calling it)

1 Like
271

Module should have bwd() shouldn’t it?

272

shouldn’t einsum be written
as: ib bj–> ij (not bi bj–> ij)? was that a typo?