Fastai v2 vision

kshitijpatil09 · May 25, 2020, 5:37pm

I confirm this is happening due to SaveModelCallback. I tried removing that callback and calling learn.validate which indeed got rid of the error. @sgugger should I open an issue on fastai2 repo about this?

Code to reproduce the error:

path = untar_data(URLs.PETS)/'images'
def is_cat(x): return x[0].isupper()
dls = ImageDataLoaders.from_name_func(
    path, get_image_files(path), valid_pct=0.2, seed=21,
    label_func=is_cat, item_tfms=Resize(224))
learn = cnn_learner(dls, resnet34, metrics=error_rate, cbs=SaveModelCallback(monitor='error_rate',fname='test'))
learn.fine_tune(1)
learn.load('test')
learn.validate()

sgugger · May 25, 2020, 5:57pm

You can, but in general, callbacks that are only relevant for training should be passed in your finet_tune/fit/fit_one_cycle call. I wouldn’t say this is a bug in the library per se.

kshitijpatil09 · May 25, 2020, 7:47pm

Now this makes sense but still, if possible, try to fix this one. I often call the fit method multiple times instead of calling it once for large no. of epochs, and passing SaveModelCallback to every call seems inefficient.

rsomani95 · May 25, 2020, 8:04pm

ClassificationInterpretation.plot_confusion_matrix() doesn’t have the return_fig option anymore, unlike v1. I was wondering if this is a deliberate choice?
I found it quite useful as it made saving the matrices to disk trivial.

kshitijpatil09 · May 26, 2020, 8:25am

Using dls.valid.show_batch() on imagewoof shows images from same class despite being unique=False (default), how can I get shuffled examples ?

kshitijpatil09 · May 26, 2020, 2:17pm

I learned a neat little trick for this

After exploring what’s different with train and valid DataLoader, I found shuffle is one of them.

So dls.train.shuffle is True while dls.valid.shuffle is False. So if you need a random batch from validation dataset, you just set it True like so:

dls.valid.shuffle=True
xb,yb = dls.valid.one_batch()
dls.valid.shuffle=False

rsomani95 · May 28, 2020, 4:48am

I’d like to train two models that have the same body, then combine them together at the end of training for inference only i.e. two separate heads on two separate datasets with a common body.

Is it possible to shuffle fitting one minibatch from dataset1, then one minibatch from dataset2?

I know how to make this work in PyTorch; sample code is available on their forums: https://discuss.pytorch.org/t/combining-multiple-models-and-datasets/82623/2

WaterKnight · May 28, 2020, 6:55pm

What is the easiest way of drawing the segmentation mask predicted by unet_learner over the input image?

kshitijpatil09 · June 16, 2020, 5:11pm

Is there any way I could decode a batch of images down to filenames? I’m looking for ways to access a particular image from learn.show_results output and try evaluating the same image on some different model architecture to compare the outcomes.

muellerzr · June 16, 2020, 5:17pm

Sure (this should work, I didn’t double check, it’s just from my CC code):


    img, lbl = x.dataset[idx]
    fn = x.items[idx]
    fn = re.search('([^\/]\d+.*$)', str(fn)).group(0)

(I use this in ClassConfusion). x is a DataLoader. (You don’t have to have the re.search, I just didn’t want the full thing just the filename)

kshitijpatil09 · June 16, 2020, 5:31pm

what does idx here represent? Let’s say I got a batch from dataloader and showed the same using:

xb,yb = dl.one_batch()
dl.show_batch((xb, yb))

Now, is it possible to find the filenames of images in this batch?

muellerzr · June 16, 2020, 5:39pm

I don’t believe so, no. Because at that point the only thing you can get back is the PIL image. If you really want to keep track of that, you should do something beforehand to know that. For mine it came from the top_losses. If you worry about shuffling and dropping you should disable those so you can grab it’s index.

Else the other method is you can look inside dl.get_idxs() and this will return the list of indexs into the Dataset for you to look at and grab. So for instance one_batch returns the first n (n=bs) from your index’s.

(again, DL is a dataloader)

kshitijpatil09 · June 16, 2020, 5:43pm

This looks great! Will try that out and let you know how it works

leaf · June 22, 2020, 1:48pm

Hi I thought I’d check here for whether this kind of approach is the right one:

I have been trying to train a semantic segmentation model using v2, but where the masks are ~2x size of the actual images.

I thought that the natural place to start would be item_tfms, but unfortunately the results are not as expected. I load the data like so:

dls = SegmentationDataLoaders.from_label_func(path, bs=1,
        fnames = imgs, 
        item_tfms=[Resize(564, 1024),RandomResizedCrop(512)],
        label_func = label_fcn,
        codes = codes,                      
        batch_tfms=[*aug_transforms(size=(512,512), flip_vert=True), 
        Normalize.from_stats(*imagenet_stats)])

But for some reason, when I run learn.dls.show_batch(figsize=(10,10),max_n=20) the sem-seg map is still misaligned. IMHO, the leading hypothesis is that the resize does not happen before the RandomResizedCrop. Is that possible? Any other thoughts?

muellerzr · June 22, 2020, 1:51pm

A quick way to check if it is going in that order or not is to use the DataBlock API and call dblock.summary() to see what is called. Considering they have the same order resize should be going first.

Also since they are 2x the size, there could be downsampling issues that may be happening as well. Do you want your transform to only make the image itself 2x larger for every 1x the mask size? (So that they are the same size)

leaf · June 22, 2020, 2:05pm

Awesome, thank you for your input.

Re: dlock. Thank you! Very handy and surprisingly easy to convert! Here’s the output. I have not yet worked with this API so I have a bit off difficult time parsing it. Very handy info, just a lot of it

Setting-up type transforms pipelines
Collecting items from /efs/public_data/images
Found 5000 items
2 datasets of sizes 4000,1000
Setting up Pipeline: PILBase.create
Setting up Pipeline: label_fcn -> PILBase.create

Building one sample
  Pipeline: PILBase.create
    starting from
      /efs/public_data/images/22346.png
    applying PILBase.create gives
      PILImage mode=RGB size=1024x560
  Pipeline: label_fcn -> PILBase.create
    starting from
      /efs/public_data/images/22346.png
    applying label_fcn gives
      /efs/public_data/inst_ids/22346.png
    applying PILBase.create gives
      PILMask mode=L size=1914x1046

Final sample: (PILImage mode=RGB size=1024x560, PILMask mode=L size=1914x1046)


Setting up after_item: Pipeline: AddMaskCodes -> RandomResizedCrop -> Resize -> ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -> AffineCoordTfm -> LightingTfm -> Normalize

Building one batch
Applying item_tfms to the first sample:
  Pipeline: AddMaskCodes -> RandomResizedCrop -> Resize -> ToTensor
    starting from
      (PILImage mode=RGB size=1024x560, PILMask mode=L size=1914x1046)
    applying AddMaskCodes gives
      (PILImage mode=RGB size=1024x560, PILMask mode=L size=1914x1046)
    applying RandomResizedCrop gives
      (PILImage mode=RGB size=512x512, PILMask mode=L size=512x512)
    applying Resize gives
      (PILImage mode=RGB size=564x564, PILMask mode=L size=564x564)
    applying ToTensor gives
      (TensorImage of size 3x564x564, TensorMask of size 564x564)

Adding the next 3 samples

No before_batch transform to apply

Collating items in a batch

Applying batch_tfms to the batch built
  Pipeline: IntToFloatTensor -> AffineCoordTfm -> LightingTfm -> Normalize
    starting from
      (TensorImage of size 4x3x564x564, TensorMask of size 4x564x564)
    applying IntToFloatTensor gives
      (TensorImage of size 4x3x564x564, TensorMask of size 4x564x564)
    applying AffineCoordTfm gives
      (TensorImage of size 4x3x512x512, TensorMask of size 4x512x512)
    applying LightingTfm gives
      (TensorImage of size 4x3x512x512, TensorMask of size 4x512x512)
    applying Normalize gives
      (TensorImage of size 4x3x512x512, TensorMask of size 4x512x512)

So my interpretation of the above is that, as hypothesized, RandomResizeCrop does come before the Resize, yielding misalignment. Is there some sort of after_item or before_batch I can access from here? These don’t seem to valid arguments for DBlock.

Re: Downsampling. yes, but the masks are in this case completely off. I’d be fine with some minor fuzzing around the edges. (I’d also consider downsampling to be easier than upsampling.)

muellerzr · June 22, 2020, 2:21pm

after_item = item_tfms
after_batch = batch_tfms

The best thing I would do is perhaps make an instance of your transform and change the .order attribute of RRC to 2 and Resize keep the same, this will let you change their order

leaf · June 22, 2020, 2:26pm

Actually, it seems I can just put RandomResizedCrop in batch_tfms as it always takes precedence. and that seems to fix it. Thank you so much!

muellerzr · June 22, 2020, 2:28pm

In that case you may instead wish to use RandomResizedCropGPU instead (it’s designed for the batch transforms)

bowenroom · June 26, 2020, 10:19am

Hi mueller @muellerzr, I have a question. If the original size of the images in the dataset is 6000*6000 and the amount is small, I use RandomCrop method in fastaiv2 to have augmentation. How could I do an inference on the original 6000*6000 images? appreciate for the help!

the original images are like these: