Any Ideas on how to modify Resnet with Images that have 2 channels?

Part 1 (2018)
41

You are just training the last layer. You can “unfreeze” other layers and train those as well. See this example.

freeze_to is the function that unfreezes layers.

1 Like
42

I tried unfreeze, just did not include it into notebook as it did not work for me as well. I have added experiments with freez_to now:

https://github.com/sermakarevich/fastai/blob/master/lesson1%20code%20for%20dog%20breeds%20id.ipynb

Even though I was not training other layers for resnet, I did the same for inception/xception. Actually I only extracted features - those are predictions from keras models with include_top=False and GlobalAveragePooling2D at the end. And then I trained some network on those features.

43

Training a neural net is a bit tricky. In this case you first have to train the last layer and then you can unfreeze. When you unfreeze you could do all of them at the same time or in different steps. If you unfreeze all layers you may need to use different learning rates for different layers.

2 Likes
44

Okay, thanks @yinterian, tomorrow will try these variants as well . I was just confused by the fact that doing exactly the same things for resnet and inception I achieve significantly different results - so I wondered maybe there are some well known pitfalls.

45

I don’t have an answer to that. It may be that inception works better with this dataset.

46

@yinterian I was just thinking of trying this very first RSNA-challenge dataset because I am a radiologist. That’s really helpful. I’ll reference this later :slight_smile:

47

ps=[0.4, 0.9] regularisation improved score from 0.4 to 0.34 and accuracy from 84 to 89% and picture size gave me 0.3< and >90%.

48

How much data do you have? You should try resnet18 as well.

49

this is Dogs Breeds competition - a subset of 120 breeds of dogs from Imagenet data set. I assume no intense tuning of conv layers is needed.

classes 120, train 8200, valid 2000.
resnet18 - 0.5 logloss, 82% accuracy (checked img size, xtra_fc, ps, aug_tfms, freeze_to)

Also I have faced with some errors, not sure if somobody else highlighted them:

  • inception/wrn models are not auto - loadable
  • I manage to load inception4 weights and import a model into notebook, but receive this error (same error for dn121)
~/fastai/courses/dl1/fastai/model.py in num_features(m)
     17 def num_features(m):
     18     c=children(m)
---> 19     if hasattr(c[-1], 'num_features'): return c[-1].num_features
  • densnet and resnext are not in the list model_meta, so:
~/fastai/courses/dl1/fastai/conv_learner.py in __init__(self, f, c, is_multi, is_reg, ps, xtra_fc, xtra_cut)
     28         self.xtra_fc = xtra_fc or [512]
     29 
---> 30         cut,self.lr_cut = self.model_meta[self.f]
     31         cut-=xtra_cut
     32         layers = cut_model(self.f(True), cut)

KeyError: <function densenet121 at 0x7f0c91814730>
  • error with resnet50 if precompute=True

~/fastai/courses/dl1/fastai/dataset.py in init(self, x, y, transform)
162 def init(self, x, y, transform):
163 self.x,self.y=x,y
–> 164 assert(len(x)==len(y))
165 super().init(transform)
166 def get_x(self, i):

AssertionError:

  • all errors when execute

data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(
resnet50, sz ,aug_tfms=transforms_top_down + [RandomFlipXY()], max_zoom=1.2))
learn = ConvLearner.pretrained(resnet50, data, ps=[0.2, 0.8], xtra_fc=[512], precompute=True)

@yinterian hope you don`t mind that I steal “hand” avatar from notebook you shared :wink:

50

All of these models should be very good at this since it is very close to the ImageNet data they were trained on.

51

To start just run it like this

learn = ConvLearner.pretrained(resnet34, data, precompute=True)
1 Like
52

Update on Iceberg challenge, here is benchmarking after running my color composite transformation:

Those who are doing the challenge can find my kernel to get the transformations here:
https://www.kaggle.com/keremt/getting-color-composites

These results may seem low compared to leaderboard, because I didn’t do much data augmentation or strict tuning. These results are ran by equal setups but just with and without transformations. You can see the substantial improvement.

Previous epoch no, train loss, val loss, val accuracy

[ 0. 0.1901 0.48927 0.7523 ]
[ 1. 0.27997 0.4605 0.74965]
[ 2. 0.33238 0.46812 0.75965]
[ 3. 0.35789 0.45681 0.75379]
[ 4. 0.36752 0.45284 0.76919]
[ 5. 0.36609 0.45696 0.76942]
[ 6. 0.35876 0.46293 0.76919]

Current epoch no, train loss, val loss, val accuracy (after color composites)

[ 0. 0.19271 0.44538 0.77941]
[ 1. 0.27228 0.38289 0.80802]
[ 2. 0.31324 0.37949 0.81583]
[ 3. 0.32731 0.38963 0.80779]
[ 4. 0.33954 0.37099 0.82537]
[ 5. 0.34608 0.37832 0.81537]
[ 6. 0.35207 0.37523 0.831 ]
[ 7. 0.34778 0.3716 0.83318]
[ 8. 0.33806 0.36026 0.84295]

Edit: LB 0.46 -> 0.33

3 Likes
53

@sermakarevich I’ll try to fix those bugs - can you provide a link/gist for a notebook that shows each of these bugs, so I can replicate them?

54

They are - they’re called dn121 and resnext50 (and others); see torch_imports.py for the function definitions.

55

@jeremy sure, will prepare it. Actually all these bugs I see when run two lines of code with mentioned models:

data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(model, sz )
learn = ConvLearner.pretrained(model, data, ps=[0.2, 0.8], xtra_fc=[512], precompute=True)

Right, these are there. I realise that in order to use any new model we need to put it into meta_model dict first.

56

If you’re transfer learning like we’re doing with the imagenet trained version of Resnet34 then it’s important. Think of it this way, the model has only ever seen RGB images so it has no way of understand YUV.

If you’re training a model from scratch then there’s no reason why it wouldn’t work in that format though.

57

Aaah, good point; thanks, Even!

58

Hello,

I have questions about resizing images before running pretrained networks.

1) When I set sz=224 in tfms_from_model this code runs:

ScaleXY(CoordTransform)

A transformation that scales the min size to sz.

Given that I have a 75x75 image if I set sz=224 will it become a 224x224 image as an input to be feed to the network ?

I couldn’t be sure because after creating data=… with sz=224 when I checked data.dataset.trn_dl.get_x() it’s shape was still 75, 75, 3.

2) I am using resnet as my pretrained model and it’s first convolution has kernel size = (7,7). Someone asked me on kaggle that whether I had resized my images to 224x224. Given 7x7 kernel size wouldn’t network be able to capture features well enough ? or should I resize it to it’s own size 224 to capture more local features ? or sz is another hyperparameter given my unique dataset ?

Thanks

59

I am trying to find image transformation options for data augmentation so I went searching like this: tfms_from_model -> image_gen -> Transforms -> compose and I am stuck right now.

Any suggestions where to find all the arguments to give to tfms like transforms_side_on , transforms_top_down, transforms_basic…

Thanks

60

You can find them all in transforms.py

1 Like