Images getting cropped at an angle

dtrik · October 29, 2018, 12:18pm

I am trying to create a classifier for raven vs crow and observed that my input images are getting cropped like this:

Is this expected?

ramesh · October 29, 2018, 12:42pm

Can you also show your transforms step (How you created your data = ImageDataBunch?) and also show how your image looks -

from PIL import Image
Image.open(<your image filename>)

dtrik · October 29, 2018, 12:55pm

gist.github.com

https://gist.github.com/dtrik/0a1821701f7e26703c44cdf8b3355125

load_images.ipynb

np.random.seed(25)
data = ImageDataBunch.from_folder(PATH, train=".", valid_pct=0.2, ds_tfms=get_transforms(), size=224, num_workers=4)
data =data.normalize(imagenet_stats)
data.show_batch(figsize=(7,8),rows=3)

jeremy · October 29, 2018, 1:14pm

Your original images have white borders on them. Our data augmentation (which we’ll learn about soon) is augmenting the white borders too.

dtrik · October 29, 2018, 1:20pm

Oh, will the images being in this format result in loss of accuracy during training? I couldn’t get my model to be better than 33% error rate. I tried running more epochs/unfreezing and retraining/setting a learning rate from lr_find/all of the above on resnet50.

ramesh · October 29, 2018, 1:25pm

You could also try turning off some of the Data augmentation step in get_transforms call from their default values, particularly the warp (set to zero) -

github.com

fastai/fastai/blob/5425d762a243225f69eabc75dad68fe4131f5f7f/fastai/vision/transform.py#L252


elif direction == 1: targ_pts = [[-1,-1-magnitude], [-1,1], [1,-1], [1,1]]
elif direction == 2: targ_pts = [[-1,-1], [-1-magnitude,1], [1,-1], [1,1]]
elif direction == 3: targ_pts = [[-1,-1], [-1,1+magnitude], [1,-1], [1,1]]
elif direction == 4: targ_pts = [[-1,-1], [-1,1], [1+magnitude,-1], [1,1]]
elif direction == 5: targ_pts = [[-1,-1], [-1,1], [1,-1-magnitude], [1,1]]
elif direction == 6: targ_pts = [[-1,-1], [-1,1], [1,-1], [1+magnitude,1]]
elif direction == 7: targ_pts = [[-1,-1], [-1,1], [1,-1], [1,1+magnitude]]
coeffs = _find_coeffs(targ_pts, _orig_pts) if invert else _find_coeffs(_orig_pts, targ_pts)
return _apply_perspective(c, coeffs)


def get_transforms(do_flip:bool=True, flip_vert:bool=False, max_rotate:float=10., max_zoom:float=1.1,
               max_lighting:float=0.2, max_warp:float=0.2, p_affine:float=0.75,
               p_lighting:float=0.75, xtra_tfms:float=None)->Collection[Transform]:
"Utility func to easily create a list of flip, rotate, `zoom`, warp, lighting transforms."
res = [rand_crop()]
if do_flip:    res.append(dihedral_affine() if flip_vert else flip_affine(p=0.5))
if max_warp:   res.append(symmetric_warp(magnitude=(-max_warp,max_warp), p=p_affine))
if max_rotate: res.append(rotate(degrees=(-max_rotate,max_rotate), p=p_affine))
if max_zoom>1: res.append(rand_zoom(scale=(1.,max_zoom), p=p_affine))
if max_lighting:
    res.append(brightness(change=(0.5*(1-max_lighting), 0.5*(1+max_lighting)), p=p_lighting))

ramesh · October 29, 2018, 3:04pm

Because Raven and Crow are close to each other unless seen up close, it might be hard to distinguish them even for ourselves. For example in the top row middle image, I would not been able to distinguish them. You may need lot of examples to distinguish them.

Also, you could try to get a Human Level Accuracy (ideally from someone other than yourself) to see how best someone can distinguish them manually from these same pictures.

dtrik · October 29, 2018, 3:22pm

Ah thanks, guess I should start with an easier classification problem.