Building a custom pytorch model

Part 2 (2019)
1

Hello Everyone,

Since we have learned more about what’s under the hood of fastai I was interested in building a model from scratch instead of using one of the preset like models.resnet34.

This has been harder then I expected.

I’m trying to recreate this model and then use it with the fastai cnn_learner - https://github.com/pytorch/examples/blob/master/mnist/main.py

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5, 1)
        self.conv2 = nn.Conv2d(20, 50, 5, 1)
        self.fc1 = nn.Linear(4*4*50, 500)
        self.fc2 = nn.Linear(500, 10)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2, 2)
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2, 2)
        x = x.view(-1, 4*4*50)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

But I keep getting different errors depending on how I try and add it. Here is my Kaggle example. https://www.kaggle.com/pascalnoble/minst-fastai-3ab05b

learn = cnn_learner(data, Net,pretrained=False, metrics=error_rate)

Here is the error
TypeError: __init__() takes 1 positional argument but 2 were given

If added like this
learn = cnn_learner(data, Net(),pretrained=False, metrics=error_rate)
then the error is
TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not bool

If anyone has any ideas or solutions let me know.

It looks like someone else had a similar question that went unanswered -Customize Densenet model

2

cnn_learner only takes pre-defined networks as input. You have to use Learner here.

1 Like
3

Hi @kushaj ,

I just tried your suggestion but I’m still having issues. Do you have any example code? Also, I’ve used cnn_learner with other models using touchvision.models - https://pytorch.org/docs/stable/torchvision/models.html

How would you make a custom predefined network that would work with fastai?

4

You want to train yout model Net using fastai, if I am correct. Then

learn = Learner(data, Net)
5

Yup, you are correct that is the goal.

I made the changes as you suggested (thank you). I think it will work but right now I’m getting a new error which I think might have to do with kaggle

AttributeError: 'torch.device' object has no attribute '_apply'
https://www.kaggle.com/pascalnoble/minst-fastai-3ab05b

1 Like
6

Have you enabled GPU in kernel?

7

Yes and I ran
torch.cuda.is_available() it returned True

8

Sorry man, I do not know about that.

9

Need to initialize arch2 before passing to Learner
learn = Learner(data, arch2(False),model_dir='/tmp/models')

 class Net1(nn.Module):
     def __init__(self,pretrained):
         super(Net1, self).__init__()
         self.conv1 = nn.Conv2d(3, 20, 5, 1)

You inputs have three channels, so your first layer needs three channels. After this everything should “work.” Will need to play around with your architecture though.

1 Like
10

Okay, went ahead and rewrote your Net1 using the same layers, but now it will train better because pytorch’s autograd should work correctly for you now:

class Net1(nn.Module):
    def __init__(self,pretrained):
        super(Net1, self).__init__()
        self.body=nn.Sequential(
            nn.Conv2d(3, 20, 5, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(20, 50, 5, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),

        )
        self.head=nn.Sequential(
            nn.Linear(4*4*50, 500),
            nn.ReLU(),
            nn.Linear(500, 10),
            nn.LogSoftmax(dim=1)
        )

    def forward(self, x):
        x=self.body(x)
        x = x.view(-1, 4*4*50)
        return self.head(x)

The parameters of your model should be contained in some variable attached to your module.

1 Like
11

@marii that was it everything is working now. I’m not sure what happened to the _apply error but it is gone.

I’ve updated the Kaggle notebook.

@kushaj thanks to both of you! Hopefully someone finds this helpful in the future.

2 Likes