How to split EfficientNet

fastai
1

image
image.png923×223 4.73 KB

EfficientNet(
(_conv_stem): Conv2dStaticSamePadding(
3, 40, kernel_size=(3, 3), stride=(2, 2), bias=False
(static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
)
(_bn0): BatchNorm2d(40, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)

(_blocks): ModuleList( 26 MBConvBlock)

(_conv_head): Conv2dStaticSamePadding(
384, 1536, kernel_size=(1, 1), stride=(1, 1), bias=False
(static_padding): Identity()
)
(_bn1): BatchNorm2d(1536, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
(_fc): Linear(in_features=1536, out_features=1, bias=True)
)

image
image.png913×171 2.18 KB

When i try default split just like the source code in https://github.com/fastai/fastai/blob/master/fastai/vision/learner.py , it says: ‘EfficientNet’ object does not support indexing.

Can someone help me out? Thank in advance!

2

This is one way of doing it

    model = EfficientNet.from_name('efficientnet-b2') 
    eff = list(children(model)[:-1])
    eff[2] = nn.Sequential(*children(eff[2]))
    eff = nn.Sequential(*eff)
    return eff

class MyEfficientNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.eff = get_efficientnet()
        self.classi = nn.Sequential(nn.AdaptiveAvgPool2d(1),Flatten(),nn.Linear(1408,1))
    def forward(self,x):
        x = self.eff(x)
        x = self.classi(x)
        return x
    
learn = Learner(data,MyEfficientNet(),metrics=[quadratic_kappa])
learn.split([learn.model.eff,learn.model.classi])
1 Like
4

Thanks for your code sharing! It helps me a lot!
Would i ask a naive question: Is there any trick for training EfficientNet?

5

I guess image size with which the variant of efficent-net is trained is important

6

EfficientNet-B2 pretrained on size 260, but if i set my image size to 260,and batch size=32, my gpu ran out of memory! Does it help to decrease batch size and remain the image size 260, although it will slow down training.

7

You can try with to_fp16().

8

check this link EfficientNet. He has better answer.

9

When i remove AdaptiveAvgPool2d in self.classi = nn.Sequential(nn.AdaptiveAvgPool2d(1),Flatten(),nn.Linear(1408,1)) ,
there is an error : RuntimeError: size mismatch, m1: [64 x 62720], m2: [1280 x 1] at /opt/conda/conda-bld/pytorch_1556653183467/work/aten/src/THC/generic/THCTensorMathBlas.cu:268
If i just want to change the output feature of the net, how to do that, i try several times but failed

10

Nice …
Thanks For the Code Sharing its really help me a lot firstbankcard.

1 Like
11

Yeah I have the same problem with zzgong. May I know how you calculated to have the number (1408, 1) in the last Linear Layer?
Thank you.