Yup this worked! Thanks
1 Like
That error bothered me for a while. It works now. Thanks
Did anyone try to give a single class because because it is binary(it supposed to predict 0 or 1 for each pixel)?
hello
I am trying to do binary as well.
I am getting this error…Thanks
src = (ImageFileList.from_folder(path_img)
.label_from_func(get_y_fn))
data = (src.datasets(SegmentationDataset, classes=codes)
.transform(get_transforms(), size=size, tfm_y=True)
.databunch(bs=bs)
.normalize(imagenet_stats))
AttributeError Traceback (most recent call last)
in
----> 1 data = (src.datasets(SegmentationDataset, classes=codes)
2 .transform(get_transforms(), size=size, tfm_y=True)
3 .databunch(bs=bs)
4 .normalize(imagenet_stats))
AttributeError: ‘ImageLabelList’ object has no attribute ‘datasets’
1 Like
@sgugger My mask has 0 and 255 as its values(binary segmentation). How do i use div = True for open_mask with SegmentationItemList for that 255 is converted to 1.
1 Like
Just slightly change the source code of SegmentationLabelList in your custom SegmentationLL
class SegmentationLL(ImageItemList):
def __init__(self, items:Iterator, classes:Collection=None, **kwargs):
super().__init__(items, **kwargs)
self.classes,self.loss_func,self.create_func = classes,CrossEntropyFlat(),partial(open_mask, div=True)
self.c = len(self.classes)
def new(self, items, classes=None, **kwargs):
return self.__class__(items, ifnone(classes, self.classes), **kwargs)
Or even shorter:
class SegmentationLL(SegmentationItemList):
def __init__(self, items:Iterator, classes:Collection=None, **kwargs):
super().__init__(items, classes, **kwargs)
self.create_func = partial(open_mask, div=True)
3 Likes
@sgugger I defined a new class in my code as suggested. But i am getting name 'kwarg' is not defined error. Where can i find the definition of kwarg ? What should i do?
It should be kwargs instead kwarg
1 Like
With a double **, I edited my post.
1 Like
@sgugger here is my code used for segmentation. Weirdly i am getting very high validation loss in few epochs while training and dice accuracy are not bad. What could be the reason for this?
class SegmentationLabelList(ImageItemList):
def __init__(self, items:Iterator, classes:Collection=None, **kwargs):
super().__init__(items, **kwargs)
self.classes,self.loss_func,self.create_func = classes,CrossEntropyFlat(),partial(open_mask, div=True)
self.c = len(self.classes)
def new(self, items, classes=None, **kwargs):
return self.__class__(items, ifnone(classes, self.classes), **kwargs)
class SegmentationItemList(ImageItemList):
def __post_init__(self):
super().__post_init__()
self._label_cls = SegmentationLabelList
src = (SegmentationItemList.from_folder(path_img)
.random_split_by_pct(0.2)
.label_from_func(get_y_fn, classes=codes))
data = (src.transform(get_transforms(), size=size, tfm_y=True)
.databunch(bs=bs)
.normalize(imagenet_stats))
def dice(input:Tensor, targs:Tensor, iou:bool=True)->Rank0Tensor:
"Dice coefficient metric for binary target. If iou=True, returns iou metric, classic for segmentation problems."
n = targs.shape[0]
input = input.argmax(dim=1).view(n,-1)
targs = targs.view(n,-1)
intersect = (input*targs).sum().float()
union = (input+targs).sum().float()
if not iou: return 2. * intersect / union
else: return intersect / (union-intersect+1.0)
learn = Learner.create_unet(data, models.resnet34, metrics=dice)
2 Likes
Lowering the learning rate solved the problem.
I have also been working on applying the camvid lesson to binary segment classification - specifically trying to implement carvana with fastaiV1. I am getting the cuda runtime error (59) : device-side assert triggered error and believe I need to add a fix for mask opening.
Since the newer API instead of Segment Dataset believe we now use SegmentationItemList which does not accept set_attr( ...) or have div=True as an option.
would love a pointer as to how to appropriately set the maskopener in the new setup to account for binary masks of 0 and 255?
For reference my src/data at the moment:
src = (SegmentationItemList.from_folder(train_128_path, True)
.split_by_idx(valid_idx=range(4065,5087))
.label_from_func(get_y_fn, classes=codes))
data = (src.transform(get_transforms(), size=size, tfm_y=True)
.databunch(bs=bs)
.normalize(imagenet_stats))
1 Like
You can see my code above which helps solve the problem. I am custom defined the segmentationitemlist and segmentationlablelist to handle the issue. We need to pass div=True to open_mask function
@jeremy In above training cycle i got abnormally high validation error. Is it something similar to what you mentioned in the talk today? Reducing the training error kind of avoided the issue for me
Thanks, am using it - I realize that I assumed that there would be a hook like assert or some such added in the 1.024 version (vs still requiring redefining the function).
Hi, I had a similar issue with new changes and I little dig in the code went this way:
src = (SegmentationItemList.from_folder(train_128_path, True)
.split_by_idx(valid_idx=range(4065,5087))
.label_from_func(get_y_fn, classes=codes))
# changes open_mask for target value
src.train.y.create_func = partial(open_mask, div=True)
src.valid.y.create_func = partial(open_mask, div=True)
data = (src.transform(get_transforms(), size=size, tfm_y=True)
.databunch(bs=bs)
.normalize(imagenet_stats))
Hope this will help.
3 Likes
Hello, can you help be more specific please?
For the new API I am getting the same error with
data = (SegmentationItemList.from_folder(path_img,div=True)
.random_split_by_pct()
.label_from_func(get_y_fn, classes=codes,div=True) #This method is primarly intended for inputs that are filenames, but could work in other settings.
.transform(get_transforms(), tfm_y=True, size=128)
.databunch())
I have passed in div in two places, for the images and masks yet this is happening. What is the fix now?
Fixed: I tried @Mirodil’s code and it gave me TypeError: ‘bool’ object is not callable , so use that code but pass div=True not just True in SegmentationItemList.from_folder.
My model is training but can view more than a few examples in the example viewer
what results do you get?
i have also obtained these empty images every now and then. i do not know what causes it, or how to fix it.
I can share the notebook, but basically my loss goes to about 0.25 but my valid and dice explode… What abt you?