Lesson 2 In-Class Discussion

Part 1 (2018)
104

Could @jeremy go over the cycle_len and cycle_mult once again

3 Likes
105

For differential learning rates, do all three defined rates decrease at same rate?

106

So does cycle_mult=2 basically mean run 2x more epochs than the previous “cycle”?

2 Likes
107

Is there any way to visualize differences in datasets the way that Jeremy is drawing? I know that image data is highly dimensional and not 2d, but is there a way to know the generalizability or quality of the minima that the model is converging to?

108

Would something akin to TTA help in inference ? By giving different augmentations of test data, we may get some of them to be found by the model thereby increasing the chances of better predictions.

109

Can we do TTA on the actual test data set and not validation data set to get better prediction on the test data?

2 Likes
110

https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

1 Like
111

Instead of taking the average of the TTA samples, could we instead weight it by the prediction confidence?

4 Likes
112

Yes, you can do it on test data.

2 Likes
113

@ravivijay Sorry what is TTA ? I must have missed it in lesson.

114

Keras to Tensorflow == Fastai to Pytorch :upside_down_face:

11 Likes
115

Test Time Augmentation.

116

So this is what stops us from using augmentation

learn = ConvLearner.pretrained(arch, data, precompute=True)

What if data pointed at an augmented set of data, rather than the un-augmented originals? Would there be harm in precomputing activations on the augmented set? Is it not worth the additional compute required?

117

The short answer is that it is not worth.

1 Like
118

Would we also eventually peel back fastai gradually, and see how things are done directly on PyTorch itself? Which would be useful for custom implementations, etc that may not exist in the fastai library

4 Likes
119

Can the pytorch models be used in mobile, in an Android and/or iOS app? Or, do we have to use something like Keras/TF?

120

@yinterian For the goup selection, what do since I am not attending physically today? I was group 5 on Monday but I am not able to attend in person today but I would like to be part of a group.

121

Tensorflow is a bit more mature in terms of mobile support, but I think PyTorch is getting around to it with recent introduction of ONNX format and ability to transfer it to Cafe2. This may be a useful resource: http://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html

2 Likes
122

TTA needs to be run everytime when we move this model to production right?

123

Probably not something possible yet but you can keep track on https://github.com/ysh329/deep-learning-model-convertor

1 Like