Issues Using EFFICIENTNET

Deep Learning
1

I created an efficient net b1 model using the following

model_effnetb1 =  EfficientNet.from_pretrained('efficientnet-b1', num_classes=data.c)

However when i tried to use the model in a learner :

learn = cnn_learner(data, base_arch=model_effnetb1, metrics = [acc_02, f_score], callback_fns=[partial(EarlyStoppingCallback, monitor='acc_02', min_delta=0.01, patience=3)], path = '/kaggle/working', model_dir = '/kaggle/working' )

i got the following error:

    AttributeError                            Traceback (most recent call last)
<ipython-input-30-76764e213374> in <module>
----> 1 learn = cnn_learner(data, base_arch=model_effnetb1, metrics = [acc_02, f_score], callback_fns=[partial(EarlyStoppingCallback, monitor='acc_02', min_delta=0.01, patience=3)], path = '/kaggle/working', model_dir = '/kaggle/working' )

/opt/conda/lib/python3.6/site-packages/fastai/vision/learner.py in cnn_learner(data, base_arch, cut, pretrained, lin_ftrs, ps, custom_head, split_on, bn_final, init, concat_pool, **kwargs)
     96     meta = cnn_config(base_arch)
     97     model = create_cnn_model(base_arch, data.c, cut, pretrained, lin_ftrs, ps=ps, custom_head=custom_head,
---> 98         bn_final=bn_final, concat_pool=concat_pool)
     99     learn = Learner(data, model, **kwargs)
    100     learn.split(split_on or meta['split'])

/opt/conda/lib/python3.6/site-packages/fastai/vision/learner.py in create_cnn_model(base_arch, nc, cut, pretrained, lin_ftrs, ps, custom_head, bn_final, concat_pool)
     82                      bn_final:bool=False, concat_pool:bool=True):
     83     "Create custom convnet architecture"
---> 84     body = create_body(base_arch, pretrained, cut)
     85     if custom_head is None:
     86         nf = num_features_model(nn.Sequential(*body.children())) * (2 if concat_pool else 1)

/opt/conda/lib/python3.6/site-packages/fastai/vision/learner.py in create_body(arch, pretrained, cut)
     54 def create_body(arch:Callable, pretrained:bool=True, cut:Optional[Union[int, Callable]]=None):
     55     "Cut off the body of a typically pretrained `model` at `cut` (int) or cut the model as specified by `cut(model)` (function)."
---> 56     model = arch(pretrained)
     57     cut = ifnone(cut, cnn_config(arch)['cut'])
     58     if cut is None:

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/conda/lib/python3.6/site-packages/efficientnet_pytorch/model.py in forward(self, inputs)
    176 
    177         # Convolution layers
--> 178         x = self.extract_features(inputs)
    179 
    180         # Pooling and final linear layer

/opt/conda/lib/python3.6/site-packages/efficientnet_pytorch/model.py in extract_features(self, inputs)
    158 
    159         # Stem
--> 160         x = relu_fn(self._bn0(self._conv_stem(inputs)))
    161 
    162         # Blocks

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/conda/lib/python3.6/site-packages/efficientnet_pytorch/utils.py in forward(self, x)
    123 
    124     def forward(self, x):
--> 125         x = self.static_padding(x)
    126         x = F.conv2d(x, self.weight, self.bias, self.stride, self.padding, self.dilation, self.groups)
    127         return x

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/padding.py in forward(self, input)
     15 
     16     def forward(self, input):
---> 17         return F.pad(input, self.padding, 'constant', self.value)
     18 
     19     def extra_repr(self):

/opt/conda/lib/python3.6/site-packages/torch/nn/functional.py in pad(input, pad, mode, value)
   2732     """
   2733     assert len(pad) % 2 == 0, 'Padding length must be divisible by 2'
-> 2734     assert len(pad) // 2 <= input.dim(), 'Padding length too large'
   2735     if mode == 'constant':
   2736         ret = _VF.constant_pad_nd(input, pad, value)

AttributeError: 'bool' object has no attribute 'dim'

how can i fix this?

1 Like
2

Don’t use cnn_learner. It is expecting a function to create a model, not a model, so it tried to pass pretrained to the function it expected to create a model and instead that ran the model.
The purpose of cnn_learner is to take a pretrained model and adapt it to a different number of classes. The EfficientNet.from_pretrained seems to be already doing that as you passed it data.c. So just create a learner from the model like learn=Learner(data, model_effnetb1, ...).

3 Likes
3
  1. md_ef = EfficientNet.from_pretrained('efficientnet-b3', num_classes=data.c)
  2. use Learner instead of ‘cnn_learner’

e.g

learn = Learner(data,
                md_ef,
                wd=1e-3)
2 Likes
4

Having now used efficientnet, I also got the splits working so you can use differential learning rates and freezing. To do this run:

learn = learn.split([learn.model._conv_stem,learn.model._blocks,learn.model._conv_head])

This creates groups for the initial convolution/bn, the group of MBConvBlocks and the head. A standard learn.freeze() should then freeze all but the head. Tested on efficientnet-b0 but I think those layer names should be the same across all variants.

7 Likes