Dynamic Unet

Part 2 & Alumni (2018)
21

Kerem, thanks for the PR, it is a great idea.

I am trying to use it in Lesson 14’s Carvana example.

When I try to learn.lr_find(), it gives this error message:

ValueError: Target size (torch.Size([8, 256, 256])) must be the same as input size (torch.Size([8, 3, 256, 256]))

This is quite odd. Target is the ground truth. So, it is a black and white image, just one channel. Input is a RGB image. This is how Unet34 from Lesson 14 works as well.

I tried to follow your code from github and the only thing I didn’t copy was the loss function (which is exactly where the error happens). My loss is a simple BCEWithLogitsLoss.

22

Hi Fred,

I have replied to your previous problem in DM, you can check it out :slight_smile:

For this problem can you try to see where exactly your code fails by using pdb package. It’s widely mentioned throughout the course. A better approach on seeking help from the forums would be sharing your notebook.

Hope this will help.

Best

23

Thanks a lot

1 Like
24

This looks great. I tried it out and am getting the following when I use DynamicUnet.
Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same

I get it even when running the notebook in the OP:

in my notebook i get it as soon as I do learn.summary()

25

Hi Peter,

I don’t have GPU resources to test it but you may follow this notebook and let me know if it still doesn’t work:

All the Best

26

IIRC, when I last tried to use fastai’s dynamic U-net, I had to .cpu() the initial m = DynamicUnet creation before moving it across to the gpu.

2 Likes
27

yes putting on cpu then gpu makes the difference:

init upsample on cpu

inp = torch.ones(1, 3, 256, 256)
out = m(V(inp).cpu())

put model to gpu if desired

m = m.cuda(0)

thanks guys

1 Like
28

I am hitting the same problem with target size and input size.
@kcturgutlu what was in that DM you sent him :slight_smile:

1 Like
29

I am replicating the message here for the greater good :slight_smile:

Hi Fred,

I believe you are not putting newly created part of the model into gpu. I add a bounding box to the part that might help you. It’s from https://github.com/KeremTurgutlu/deeplearning/blob/master/datasciencebowl2018/Dynamic%20Unet%20Tutorial.ipynb.

http://forums.fast.ai/uploads/default/original/2X/1/188f6d623c391d500f23f561485e10855e8dca3a.png

This should solve it.

Your new code chunk should look something like this.

f = vgg16
cut, cut_lr = model_meta[f]
encoder = get_encoder(f, 30)
m = DynamicUnet(encoder)

# init upsample on cpu
inp = torch.ones(1, 3, 256, 256)
out = m(V(inp).cpu())

# put model to gpu if desired
m = m.cuda(0)

models = UpsampleModel(m, cut_lr=20)
learn = ConvLearner(md, models)
learn.opt_fn=optim.Adam
learn.crit=nn.BCEWithLogitsLoss()
learn.metrics=[accuracy_thresh(0.5),dice, soft_jaccard]
1 Like
30

Hi Fred and @kcturgutlu

I am running into the exact problem you mentioned above.

ValueError: Target size (torch.Size([64, 128, 128])) must be the same as input size (torch.Size([64, 3, 128, 128]))

I am following the exact steps but the error persists. I am using BCEWithLogitLoss() with ResNet. I get the same error if I switch to vgg16. When I use cpu only, I get an error related to ‘tensors on different GPU’.

Any hints on how you managed to resolve this? Thanks.

Solution: I managed to fix this by modifying the channel outputs to match the masks (grayscale). Very neat implementation Kerem! I didn’t know you already have a parameter for n_classes output. Cheers!

31

I am having similar problem when using resnet152 and I get the error:

RuntimeError: Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same

where running learn.summary()

Here is the piece of code that I am running:

def get_encoder(f, cut):
base_model = (cut_model(f(True), cut))
return nn.Sequential(*base_model)

f = resnet152
cut,lr_cut = model_meta[f]
encoder = get_encoder(f,cut)
encoder = encoder.cuda(1)
m = DynamicUnet(encoder).cuda(1)
models = UnetModel(m)
learn = ConvLearner(md, models) # md is my model data

any clue what should I do?

32

I ran into the same issue. Passing n_classes = 1 got the right output but it was delivered as a [64, 1, 128, 128] tensor rather than the [64, 128, 128] tensor the model expected. I fixed this by adding

if self.n_classes == 1:
    out = out.squeeze(1)

before the return

4 Likes
33

try putting on cpu first then moving to gpu

1 Like
34

I’ve been playing around with a few different kinds of Unets and I’m having issues using progressive resizing. I’m training on 256 x 256 images, then 512 x 512.

When I use a resnet34 backbone, I can get a dice score of 0.81 on 256 x 256 and 0.84 on 512 x 512.

When I use a VGG16 backbone, I get a dice score of 0.9 on 256 x 256, which is a great improvement over resnet34. But when I scale up to 512 x 512, the model falls apart. BCE loss jumps about 10x, dice score drops to 0.71, and the model doesn’t seem to train - it just oscillates. Behavior is the same over a range of learning rates.

Any thoughts on why this might be the case?

1 Like
35

There had been lot of requests around this issue. If anyone interested in how to get dynamic unet up and running for a real example I’ve created a repo for an ongoing kaggle competition. You may check it.

It essentially demonstrates:

This is the notebook https://github.com/KeremTurgutlu/airbus/blob/master/start%20-%20detection_segmentation.ipynb. This is just to show, it might not work start to end without preparing data as required. So you may check the other notebooks in the same repo.

Next I will look into custom transforms for this task.

I hope this helps

2 Likes
36

I’ve been trying to use your U-Net implementation with my own dataset and I am facing some problems, hope someone can help!

My dataset is composed by a train and a test set. Inside the training set folder I have 2 other folders, one with the original image and other with the segmentation groundtruth (the original and mask share the same file name). The same is true for the test set folder.

The problem is that I cannot find a way to load the data and feed it to the model with the fastai library.

Thanks :slight_smile:

EDIT: Solved it, my problem was that I was not passing the mask images.

1 Like
37

@kcturgutlu Is it possible to load MobileNet as the encoder with your Dynamic U-Net and load my own wheigts from another task other than segmentation?

38

Thanks for this!

39

I haven’t tested it yet but should be very much possible.

1 Like
40

i’m having the same problem