What are some of the salient characteristics of a loss function: e.g., do the have to be differentiable in the parameter space for SGD to work ? Anything else ? How do you choose a loss function for a task ?
Yep I’ve seen some of the differences but on the same dataset it looks the same most of the time training on resnet34 was faster
I’m curious about the pacing of this course. While I do appreciate the information that Jeremy shares about coronavirus I’m concerned that all the material for the course may not be covered.
10 Likes
Parameters are the “weights” of your neural network. They’re the individual “neurons” of your neural network. Their values are chosen/learned automatically during training.
Hyperparameters (such as batch size, learning rate etc.) are things that we have to choose ourselves and are typically not learned automatically.
4 Likes
There are canonical loss functions that, on top of being differentiable, have some nice properties that play well with the last activation function (we will see what that means soon). Usually you have three general ones at your disposal, and depending on your problem you will pick one of them. We will teach you all about it in the following lessons 
2 Likes
I am wondering about trying to make a dogs/cats/other classifier. The specific problem is that users (imagine the input is eg. photos from a mobile phone) frequently try to confuse the model, so INTENTIONALLY take a picture of for example cars and it really helps for perception of the quality of the model to be able to filter it. How would I tell the model to detect that, so as to be able to tell the user to ‘stop that,this is not a cat or dog’? Obviously, I cant just assume humans will not be humans and behave. 
3 Likes
We will cover this problem when we look at a multi-label task in the next chapters. (This is a great question btw).
2 Likes
Exploratory data analysis for image datasets, is it still relevant or necessary in deep learning, especially when we are using transfer learning?
3 Likes
How would I tell the model to detect that, so as to be able to tell the user to ‘stop that,this is not a cat or dog’?
What you are describing is the issue of out-of-distribution predictions. In general, neural networks cannot be used for completely different domains from the ones that they are trained in. There exist sophisticated ways to detect if your example image is not from the same distribution as the training set, e.g., using self-supervised learning or other out-of-distribution detection methods.
3 Likes
Link to Zeiler’s paper
Visualizing and Understanding Convolutional Networks
A post was merged into an existing topic: Lesson 2 - Non-beginner discussion
Is there any pretrained weights available other than the ones from Imagenet that we can use? If yes, when should we use others and when Imagenet? Thanks!
7 Likes
How is the size of the training set determined to be able reasonable accuracy while testing ?
I get skips in between the video. Not sure if the issue is in my end. Is someone else feeling the same…
The pretrained models (in fastai) come from PyTorch’s Model Zoo, and generally if they are pretrained it’s on ImageNet.
2 Likes
You could try introducing a third class neither cat nor dog, and include lots of examples of these in your training set.
Can we merge different pre-trained models?
3 Likes
could you please talk a bit about ways to handle and measure “uncertainty” in deep learning.
3 Likes
The issue with that is you get into a problem of infinite possibilities of what should my third class be. It’s a great idea just it has some weaknesses. The idea with multi-label (I imagine) will be similar to this
A post was merged into an existing topic: Lesson 2 - Non-beginner discussion