A walk with fastai2 - Vision - Study Group and Online Lectures Megathread

barnacl · February 19, 2020, 6:11pm

so not related to this?
The reason i ask is what if (rather when)google fixes that bug then how do we run it ?

muellerzr · February 19, 2020, 6:15pm

I’m unsure. Probably choose a smaller image size for your transfer image. Or shrink it down via a transform (since it is a TensorImage).

To the no_grad and eval, no. See above discussion where I said why their different.

No_grad never updates for evaluation weights. Only gradients. And vice versa

barnacl · February 19, 2020, 6:24pm

oh shoot yeah.

muellerzr · February 19, 2020, 6:25pm

That’s why I brought it into further discussion, very easy to get confused

barnacl · February 19, 2020, 6:36pm

So to lay this out:
We are using vgg19 to calc the feature loss. We are using a pre-trained model so all the weights are calc.
We are using vgg19 in inference mode ie .eval (to calc the activation at different layers).
The purpose of .eval is to appropriately change the behaviour of bn and dropout layers. But our vgg19 doesn’t have any of those? (so why is .eval being used ? ) is .eval effecting something else ?
Because by default a model is in model.train mode so we use .eval to change the mode.

no_grad can be used with .eval for saving space. (they CAN"T be used interchangeably)
model.train() and model.eval() do not change any behavior of the gradient calculations.

model.eval()
for batch in val_loader:
    #some code
vs
model.eval()
with torch.no_grad():
    for batch in val_loader:
        #some code

The first approach is enough to get correct results. The second approach will additionally save some memory.

@muellerzr am i missing anything? please edit it if i am
Thanks.

s.s.o · February 19, 2020, 6:47pm

I’m not sure, but eval already disables grad. Thats’ the major diffence between modes. So, no need to additionally disable it - mentioned either on forums or lectures…

muellerzr · February 19, 2020, 6:52pm

It doesn’t. (See the very last post on that thread). It’s still calculated. We can disable it to save memory.

muellerzr · February 19, 2020, 6:57pm

Now for a bigger question which could help us all explore the code:

What does get_preds in fastai2 default to doing? For where to start, check Learner.py

barnacl · February 19, 2020, 7:01pm

Still don’t understand a use case where you would want to calc gradients in the eval mode though

muellerzr · February 19, 2020, 7:07pm

The answer (Atleast for learn.validate()) is we don’t calculate the gradients:

So we can see on GatherPredsCallback during begin_validate we see the model set to eval:

github.com

fastai/fastai2/blob/master/fastai2/learner.py#L60


        self.learn.train_iter += 1


    def begin_train(self):
        "Set the model in training mode"
        self.learn.pct_train=self.epoch/self.n_epoch
        self.model.train()
        self.learn.training=True


    def begin_validate(self):
        "Set the model in validation mode"
        self.model.eval()
        self.learn.training=False


# Cell
#TODO: save_targs and save_preds only handle preds/targets that have one tensor, not tuples of tensors.
class GatherPredsCallback(Callback):
    "`Callback` that saves the predictions and targets, optionally `with_loss`"
    def __init__(self, with_input=False, with_loss=False, save_preds=None, save_targs=None, concat_dim=0):
        store_attr(self, "with_input,with_loss,save_preds,save_targs,concat_dim")


    def begin_batch(self):

Later, if we go look at _do_epoch_validate (on Learner), we find our with torch.no_grad():

github.com

fastai/fastai2/blob/master/fastai2/learner.py#L285


        self.all_batches()
    except CancelTrainException:                         self('after_cancel_train')
    finally:                                             self('after_train')


def _do_epoch_validate(self, ds_idx=1, dl=None):
    if dl is None: dl = self.dls[ds_idx]
    names = ['shuffle', 'drop_last']
    try:
        dl,old,has = change_attrs(dl, names, [False,False])
        self.dl = dl;                                    self('begin_validate')
        with torch.no_grad(): self.all_batches()
    except CancelValidException:                         self('after_cancel_validate')
    finally:
        dl,*_ = change_attrs(dl, names, old, has);       self('after_validate')


def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False):
    with self.added_cbs(cbs):
        if reset_opt or not self.opt: self.create_opt()
        self.opt.set_hypers(wd=self.wd if wd is None else wd, lr=self.lr if lr is None else lr)


        try:

barnacl · February 19, 2020, 7:09pm

Just to clarify-

You mean during evaluation we set it in eval mode and turn off grad calculations right ?

muellerzr · February 19, 2020, 7:10pm

Correct I’ll put that in real quick

Edited the post before this

muellerzr · February 19, 2020, 10:40pm

Video for tonight

mgloria · February 19, 2020, 11:38pm

Yes, the category ids are not continuous so sorting them out as @muellerzr will unfortunately not work. Right now I am just ignoring the category ids - getting the mask - mapping back the mask to the correct category id… it works but it feels very inefficient…

muellerzr · February 19, 2020, 11:59pm

Thanks guys for joining!

A few notes from today:

Jeremy’s Object Detection Lesson: here
RetinaNet Paper: here
Relevant source code for predictions (may break, have not looked at this for a few days): here
Single Point Regression (headpose): here

(remind me if there’s anything else I missed to link)

muellerzr · February 20, 2020, 2:16am

One thing not mentioned was the heatmap that I’ve spoken about. This is due to I was not able to successfully train the model well. However, if it is of interest I can release the source code and do a quick walk through video describing the technique and how I went about the dataloader if people would be interested. Let me know

lgvaz · February 20, 2020, 2:31am

Replying to let you know I’m interested

vijayabhaskar · February 20, 2020, 12:03pm

@muellerzr I don’t understand how fastai2 decides to apply transforms, I mean for Keypoint regression if input is transformed the output should also be transformed, For example, If Resize is applied as item_tfms, the output is transformed by the same function?, or how is that handled?, lets say if my input is a rotated image and output is always a straight image, In this case I just want to apply rotation only to my inputs, how would I do that?

muellerzr · February 20, 2020, 2:05pm

Yes. If we think about keypoints we need to adjust our y’s along with it, remember our KeypointScalar transform we made a few lessons back. The transforms are run by TypeDispatch, where if we use a transform, if our input is what that transform is expecting, it will run it. IE Resize for instance looks like so (not exact but close):

class Resize():
def encodes(x:[TensorPoint, TensorImage])

vijayabhaskar · February 20, 2020, 2:57pm

Got it! But what if I need my input images to be transformed but not the output image, how would I do that?