But⌠we did show that conv is just a matrix multiply, with some tied weights and zeros, and weâve already done that from scratch; so I figured we donât gain much doing conv from scratch too. And it would be soooooo slooooow.
But for folks still feeling a little unsure about what a conv does - you absolutely should write it yourself!
Itâs fine to have a negative class for a binary problem (NLP, vision, or anything else) since itâs simply sigmoid activation and we donât have this same issue.
But we donât have a negative class for multi-class NLP problems IIRCâŚ
Yes, if you know you have one and exactly one class represented in each data item, then softmax is best, since youâre helping the model by giving it one less thing to learn.
nano doesnât really do enough to be useful. I wouldnât suggest spending time learning it. Use vim or emacs. Emacs is a little easy to get started with, although vim is better for manipulating datasets (although there are emacs extensions to help there).
Itâs negligible. But you can check for yourself - use %timeit to see how long an if statement takes in python. Then compare that to the number of batches we do to train a model, and see what you think.
Is there a specific reason why we continue using standard deviation in our convnet model, after Jeremy explains that mean absolute deviation is often better? Or does this statement not apply at all to Batchnorm etc. somehow? (Maybe that is one of those âtry blah and seeâ experiments? )
Because thatâs what everyone has always done, so I made something I knew would work to show in class. It would be interesting to try abs instead. My guess is would work about equally well. Let me know what you find if you try it!
Why is the Runner not part of learner? If intention is to keep the learner free of code then why not just have the runner as a member of learner like model, data, loss? It seems weird to call runner.fit(1, learn) rather than just learn.fit(1).
If you donât feel ready to take on Vim or Emacs yet, I suggest you download VS code as an intermediate step. As Jeremy mentioned, you can hover or right click on, say an object that is inherited by a class and it will take you to the source code.
Note that it is important to be in the correct Python environment for VScode. You have to select an interpreter, see instructions here. Folders are also important. I can show you what I know so far - text me some times when youâre available.
I liked Jeremyâs mantra from lesson 10:
âactivations are things we calculate
parameters are things we learnâ