Lesson 3 In-Class Discussion

Part 1 (2018)
101

I don’t understand. Jeremy mentioned that multi label encoding doesn’t use Softmax. Then when he says how the labels are being compared against the activation function he says softmax. So is multi-label classification using Softmax or not using softmax?

102

How does Jeremy decide what early, middle, and last layers are?

2 Likes
103

Does it take only jpeg image formats?

104

As Jeremy first said, Multilabel shouldn’t use softmax. Softmax tries to prioritize a single output.

105

I think the difference is between multi-label and multi-class. When you have a multi-label problem you use sigmoid but with multi-class you would use softmax. With multi-class you want the class with the highest probability and with multi-label it means you can have multiple labels i.e. with planet you can have clear and primary labels both. But with dog breeds you can only have one breed of dog at a time out of 120 possibilities.

Another point to add is that with multi-label you need to come up with some threshold that the sigmoid values need to be in order to be considered. For example if you set a threshold of 0.2 then any of the sigmoid values that are over 0.2 would be in the list of labels that were assigned to that image.

11 Likes
106

Can custom metric functions accept inputs other than the preds and targs (e.g. some attribute from the input dataset)?

107

Are there any heuristics on what size to try and use if it’s not the same as imagenet?

108

Sigmoid Function

1 Like
109

got it. It’s using sigmoid. I thought I heard he said he used softmax.

110

%?? before the function

111

SoftMax is for single-label, Sigmoid is for multi-label.

EDITED

112

are there any other activations you would want to use at the end for any other reasons besides Softmax or Sigmoid?

113

not multi class but its multi label.

114

tanh was popular at one point.

115

not multi-class but multi label.

116

So what would be a good reason to use that? Or have these two phased out tanh?

117

The microphone sounds kind of muffled tonight to me. Maybe it’s just my ears, but it doesn’t sound great to me

118

Tanhs work well with Gans. It depends on the model you are using and the task.

1 Like
119

It’s complex, but I think it’s explained well here: https://stats.stackexchange.com/questions/101560/tanh-activation-function-vs-sigmoid-activation-function

4 Likes
120

Can the learning rate finder function be adapted to suggest the differential (intermediate) learning rates?

1 Like