just so i understand this. to detect overfitting we monitor accuracy or validation error?
in the response above, Jeremy states that we want to track accuracy, then in the lesson 8 video, Jeremy says it’s the validation error we want to track:
so overfit means what? it means that your training loss is lower than your validation loss. no. no it doesn’t mean that. remember it doesn’t mean that. a well fit model will almost always have training loss lower than the validation loss. remember that overfit means you have actually personally seen your validation error getting worse okay. until you see that happening you’re not overfitting.
or maybe this is just definition thing, we have:
validation loss
training loss
validation error
accuracy
error_rate (introduced in pets notebook from part 1)
so i understand that validation loss is not the same as validation error
and that validation error (or error rate) = 1 - accuracy
and that to detect overfitting we track accuracy or validation error (error rate).
Going through the recording of the lecture 8 I annotated lesson notebooks with YouTube video links. Added annotations link position in the notebook to the corresponding location of the video where Jeremy discusses the few next cells in the notebook. I found it useful for myself, and I think it might be useful for others. I would like to share it, but I am not sure how to do that. What would be the right way to contribute the annotated .ipynb files?
I wanted to force myself to not “cheat” by looking at Jeremy’s code while re-creating what we did in lesson 8, so I made a notebook with more or less only the instructions. Not sure if it will be helpful to anyone else, but I posted it here
@jeremy, I think there is another buglet to fix in the video.
In broadcasting matmult (video 1:04:25) you suggest that these 2 are the same:
for i in range(ar):
c[i] = (a[i].unsqueeze(-1)*b).sum(dim=0)
c[i] = (a[i,None]*b).sum(dim=0)
but the second one gives an error.
RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 1
The only way I can make it work is via a temp assignment:
for i in range(ar):
t = a[i]
c[i] = (t[:,None]*b).sum(dim=0)
All your unsqueeze using None examples were using : but as soon as you replace : with an actual index like 0, those things no longer work. Here is an example:
Glad I did it since, I definitely needed to refresh some concepts, but it is pretty straightforward, and combining it with the videos you should have a clear idea on what this method does vs. the Xavier/Godot Initialization.
If you see something that is not clear or wrong, just reach out.
PS: Shoutout to @PierreO, saw your post this morning, really nice job man.
Thanks! I’ve created an Errata section in the top post and added this there. Please add any other bugs you notice (except those already mentioned in the video).
I’m glad I searched before posting the question
I tried to see if there are ways to achieve this using None but didn’t find one until I saw Jeremy’s post on top (and no matter how long I play with it, it still feels strange).
Often you want to write code that can handle varying rank - in which case ... can be used to ensure you don’t have to change anything when rank changes.
I use Serveo.net (free) to create an SSH tunnel which exposes my Jupyter notebook (port 8888) to the public internet (port 80/443). Caution, security is only as good as the security of Jupyter’s login screen. This command will allow you to access your server’s Jupyter notebook, running on your intra-net, using your own (sub-)domain (e.g. https://n.u.example.com). If ssh crashes, the script will try to reestablish connection once per second. I find it quite reliable. See serveo.net for more info. The beauty of this method is you can access your notebooks anywhere, optionally using your own custom domain. No need to open ports on your router or firewall. An alternative is to use remote access software such as VNC or TeamViewer (free, recommended).
date && until ssh -o ServerAliveInterval=60 -R n.u.example.com:443:localhost:8888 serveo.net -i ~/OneDrive/Documents/u.example.com; do date && sleep 1 && echo "Restarting..."; done