Lesson 3 In-Class Discussion

Part 1 (2018)
182

Not sure :slight_smile: away from computer - maybe 4 or -1. -1 means use max.

BTW to see this play out I would recommend installing htop and running it from terminal, really nice way to visualize what your CPU cores are doing.

183

:+1: Okay

184

It’s 8. You can always get the default value by looking at Source code…type ??ImageClassifierData.from_csv or ??ImageClassifierData.from_paths depending on what you are using.

1 Like
185

Jeremy mentioned earlier that 20% of the training data is good for validation set. I have a dataset just like in the dogscats format - train and test only. This is how we do it with the csv file:
label_csv = f'{PATH}train_v2.csv
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n)

how do i get the validation set from train when i don’t have a csv labels file?

186

How about create such a file? os.listdir , split, to_csv etc ?

187

Take a look at this post to how you can shuffle some data from train and move to validation directory - Faster experimentation for better learning

188

I didn’t get that. Are you saying:

  1. create csv file?
  2. split training set?
189

This really helps. Thank you!

190

You would like to split train/valid using from_csv method but dont have csv with labels. I assumed you should have then train with labels encoded in images filenames (dog12345.jpg) or already stored in specific folder names (dog/12345.jpg). It is possible to create necessary csv file from both these types of train set.

With from_csv method you should not move files between folders train > valid > train.

191

The from_csv method is more effective i feel. The images are in specific folders (A/A-train001.jpg). So should i be writing a py script to get the filenames to a csv file?

192

I think so. Something like:

labels = {}
for c in os.listdir(train):
    for f in os.listdir(os.path.join(train, c)):
       labels[f.split(".jpg")[0]] = c
193

How do someone access the types of pretrained models present e.g - resnet18, resnet34 - how many models are out there ?

194

in conv_learner.py you can find

model_meta = {
    resnet18:[8,6], resnet34:[8,6], resnet50:[8,6], resnet101:[8,6], resnet152:[8,6],
    vgg16:[0,22], vgg19:[0,22],
    resnext50:[8,6], resnext101:[8,6], resnext101_64:[8,6],
    wrn:[8,6], inceptionresnet_2:[-2,9], inception_4:[-1,9],
    dn121:[0,6], dn161:[0,6], dn169:[0,6], dn201:[0,6],
}
1 Like
195

Thanks.

196

Thanks.

197

Is it possible to know what size of input they accept.

198

Jeremy told they can accept any input. On what image size they were trained you can find here https://keras.io/applications/#xception. Typically it is 224 for old models and 299 for new.

1 Like
199

Hi @vikbehal,

Let me give a try and @jeremy can correct me.
I think what you are asking is actually about the efficiency of “binary encoding” v.s. “set encoding” . Assuming you have a universe of classes as U={A,B,C,D,E}, for a subset S={A,D} (as true classes belongs to a particular instance). For example, a picture contains a dog (A) and a tree (D). One can either represents it as a binary encoding “10010” or a set encoding “{A,D}”. I would say both are useful. If |U| is small, binary encoding is more efficient, but when |U| is large, like in word2vec with >1million classes (word tokens), “set encoding” saves a lot of computation space and time.

In pytorch, the loss layer of cross-entropy takes a “set encoding” for the target, but a “binary encoding” for the prediction. You can learn more about it here . We can dig much deeper if needed, but I think Jeremy is going to talk about loss functions in the next lecture. Many things should make more sense afterward.

In case you want to learn them right now, I would recommend you to learn about “sigmoid”, “entropy”, “cross-entropy”, “KL-divergence”, “NCE_loss” (in the order from fundamental to advanced) and any other interesting subjects/terms you encounter along the way for computing loss functions of multi-class.

In essence, one wants to find a measure between the “distribution” of prediction and the truth. The measure (loss function) has to be differentiable and convex.

I hope this help.
Bests,
Ray

3 Likes
200

Take a look at the source from freeze_to - it’s only 3 lines of code IIRC. You could just loop through children(learn.model) and use the same basic code.

201

AFAIK - The issue is, the pre-trained models are defined as three Sequential Layers and they don’t have any name parameter. So not sure how to go about unfreezing only parts of the layers in there. But freeze_to is an reasnable way to just unfreeze the immediate upper layer. So going to leave it for now.