Fastai v2 vision

Can we use BBoxBlock independently without BBoxLblBlock?

I tried doing so but it throws some error with bb_pad. I tried removing that method from pipeline but then it started complaining about invalid shapes despite of all being same.

Collating items in a batch
Error! It's not possible to collate your items in a batch
Could not collate the 0-th members of your tuples because got the following shapes
torch.Size([3, 224, 224]),torch.Size([3, 224, 224]),torch.Size([3, 224, 224]),torch.Size([3, 224, 224])

I can post whole stacktrace if you need.

EDIT: Passing in list of dummy labels of required length passes the pipeline, is that how it should be done?

If you use it on its own, you have to write your own padding functions. bb_pad is meant to work with labeled bouding boxes, so you need to adapt it.

1 Like

I did try to adapt bb_pad by passing in empty string as a label. Iā€™ve also modified the L1Loss to work with multiple outputs. Now Iā€™m getting bool value of Tensor with more than one value is ambiguous error. Iā€™ve posted more about it under similar topic here

Just a general question, would you guys (devs) or others in general be interested in a PR that includes a utility function for transferring weights from similar architectures? IE transfering a modelā€™s weights trained on ImageWoof to a model being used on the PETs dataset?

See this post for a general idea: Loading pretrained weights that are not from ImageNet

2 Likes

There is a function that does the same somewhere in text.models.

Ah that actually makes sense, probably load_encoder? (Iā€™ll look in a little bit) If so, would it be okay moving it to a level higher? (Learner) or would you rather keep itā€™s functionality there

We can have the utility in torch_core and some access in Learner, yes.

1 Like

Hey guys, just wanted to point out this very cool behavior with get_preds and multiple models!

Let me build a scenario for you, I have two separate models that receive the same image, and I want to do inference with both models. One model does say regression and the other model does classification. Now our presumption would first be I need to make two new dataloaders for each and then I need to run any decoding separately, but thatā€™s unnecessary! Even though I have a dataloader built from the classification model, if I do decode_preds on the regression model, it will work!

Example code snippet that wonā€™t break:

imgs = ['img1.jpg', ...]
dl = classify_learner.dls.test_dl(imgs)
a,b,c = classify_learner.get_preds(dl=dl, with_decoded=True)
a,b,c = reg_learner.get_preds(dl=dl, with_decoded=True)

Very surprised by this (but a welcome one!) I presume it has to do with decodes looks at the type of output and then does the decode_batch, which is a very cool behavior :slight_smile:

1 Like

The decoding is only the one from the loss function, that is why it works :wink:

1 Like

I had similar requirement in one of my project, I was wanted to export the weights of encoder of Unet and use that for classification problem. One additional requirement was that I had few additional bottleneck layers after the encoder, which should also be exported in the same .pth file. I feel the way I did it is somewhat hairy and would be great if you could suggest a better of doing the same :slightly_smiling_face:

  1. Save weights:
encoder = nn.Sequential(*learn.model.layers[:4]) // For additional bottleneck layers
checkpoint = {
    "arch": encoder,
    "model": encoder.state_dict()
}
torch.save(checkpoint, encoder.pth')

(If your requirement is to export only the encoder, you could do simply learn.model[0])

  1. Load weights for different task
ckpt = torch.load('encoder.pth')
encoder = ckpt['arch']
encoder.load_state_dict(ckpt['model'])
# output:
# <All keys matched successfully>

One caveat though: It need any custom layer/layer block definitions to be imported.(eg. ResBlock)

It would be great if this functionality becomes the part of library :smiley:

@sgugger @jeremy Hello! Have you heard by chance of this project: https://github.com/microsoft/unilm/tree/master/layoutlm ?

Not able to reproduce the results using learn.load

saved_model = SaveModelCallback(fname=exp_name)

Iā€™m using SaveModelCallback to save best model while training. After done with the training, I tried deleting and re-instantiating the learner with random weights and then ran learn.validate() (results went down as expected)

Then I performed learn.load(exp_name) and ran learn.validate(), still the results are as worse as the random weights.

EDIT: Saving it explicitly using learn.save and then loading it back reproduces the results

deep and deeper versions of xresnet and xse_resnext produce following error when used with pool=MaxPool.

RuntimeError                              Traceback (most recent call last)

<ipython-input-44-0aadd6b2d7d1> in <module>()
----> 1 body(torch.randn(8,3,288,288)).shape

5 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in forward(self, input)
     98     def forward(self, input):
     99         for module in self:
--> 100             input = module(input)
    101         return input
    102 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in forward(self, input)
     98     def forward(self, input):
     99         for module in self:
--> 100             input = module(input)
    101         return input
    102 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/fastai2/layers.py in forward(self, x)
    552         self.act = defaults.activation(inplace=True) if act_cls is defaults.activation else act_cls()
    553 
--> 554     def forward(self, x): return self.act(self.convpath(x) + self.idpath(x))
    555 
    556 # Cell

RuntimeError: The size of tensor a (5) must match the size of tensor b (4) at non-singleton dimension 3

Code to reproduce the results:

arch = partial(xresnet34_deep, pool=MaxPool)
body = create_body(arch,cut=-4)
body(torch.randn(8,3,288,288))

I have a short question: does the Resize parameter in item-tfms apply to just the images, or does it also resize the bounding boxes? Thanks for your advice

The vision transforms use type dispatching to handle the various types (bounding boxes, keypoints, segmentation, etc) so yes :slight_smile: however you should use a padding method IIRC to not lose any data

Edit: yes you should use ResizeMethod.Squish

Abysmal performance by xresnet34 over vanilla resnet34 on ā€˜Stanford-dogsā€™ dataset.

xresnet34:

resnet34:

Any reason why this is happening?

Colab link: https://colab.research.google.com/drive/1zRk0nRMdtLZdiuKpEzJAymsZUgWZg9-b

I think itā€™s due to the fact that thereā€™s no pre-trained version. See this comment.

1 Like

So atleast xresnet50 should work with pretrained weights or that might fail as well? I my previous experiments, I tried training xresnet34 from scratch, it went from 3% to 26% and loss went ā€˜nanā€™ after 20 epochs. (I was using ranger optimizer with lr=1e-4)

In general, is it safe to start with xresnet models while working on new dataset/augmentation/loss function?

What are the recommended optimizers and lr_schedulers while training for large no. of epochs (50+)

Looking at the log it seems it tried to download xresnet50 insteadā€¦
You can clearly add in your learner pretrained=False and it may work (though training will be much longer and it wonā€™t benefit from pretrained networks).

What are the advanced techniques for UNet right now??? What does the parameter norm from unet_config??