Building a custom pytorch model

Hello Everyone,

Since we have learned more about what’s under the hood of fastai I was interested in building a model from scratch instead of using one of the preset like models.resnet34.

This has been harder then I expected.

I’m trying to recreate this model and then use it with the fastai cnn_learner - https://github.com/pytorch/examples/blob/master/mnist/main.py

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5, 1)
        self.conv2 = nn.Conv2d(20, 50, 5, 1)
        self.fc1 = nn.Linear(4*4*50, 500)
        self.fc2 = nn.Linear(500, 10)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2, 2)
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2, 2)
        x = x.view(-1, 4*4*50)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

But I keep getting different errors depending on how I try and add it. Here is my Kaggle example. https://www.kaggle.com/pascalnoble/minst-fastai-3ab05b

learn = cnn_learner(data, Net,pretrained=False, metrics=error_rate)

Here is the error
TypeError: __init__() takes 1 positional argument but 2 were given

If added like this
learn = cnn_learner(data, Net(),pretrained=False, metrics=error_rate)
then the error is
TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not bool

If anyone has any ideas or solutions let me know.

It looks like someone else had a similar question that went unanswered -Customize Densenet model

cnn_learner only takes pre-defined networks as input. You have to use Learner here.

1 Like

Hi @kushaj ,

I just tried your suggestion but I’m still having issues. Do you have any example code? Also, I’ve used cnn_learner with other models using touchvision.models - https://pytorch.org/docs/stable/torchvision/models.html

How would you make a custom predefined network that would work with fastai?

You want to train yout model Net using fastai, if I am correct. Then

learn = Learner(data, Net)

Yup, you are correct that is the goal.

I made the changes as you suggested (thank you). I think it will work but right now I’m getting a new error which I think might have to do with kaggle

AttributeError: 'torch.device' object has no attribute '_apply'
https://www.kaggle.com/pascalnoble/minst-fastai-3ab05b

Have you enabled GPU in kernel?

Yes and I ran
torch.cuda.is_available() it returned True

Sorry man, I do not know about that.

Need to initialize arch2 before passing to Learner
learn = Learner(data, arch2(False),model_dir='/tmp/models')

 class Net1(nn.Module):
     def __init__(self,pretrained):
         super(Net1, self).__init__()
         self.conv1 = nn.Conv2d(3, 20, 5, 1)

You inputs have three channels, so your first layer needs three channels. After this everything should “work.” Will need to play around with your architecture though.

Okay, went ahead and rewrote your Net1 using the same layers, but now it will train better because pytorch’s autograd should work correctly for you now:

class Net1(nn.Module):
    def __init__(self,pretrained):
        super(Net1, self).__init__()
        self.body=nn.Sequential(
            nn.Conv2d(3, 20, 5, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(20, 50, 5, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),

        )
        self.head=nn.Sequential(
            nn.Linear(4*4*50, 500),
            nn.ReLU(),
            nn.Linear(500, 10),
            nn.LogSoftmax(dim=1)
        )

    def forward(self, x):
        x=self.body(x)
        x = x.view(-1, 4*4*50)
        return self.head(x)

The parameters of your model should be contained in some variable attached to your module.

1 Like

@marii that was it everything is working now. I’m not sure what happened to the _apply error but it is gone.

I’ve updated the Kaggle notebook.

@kushaj thanks to both of you! Hopefully someone finds this helpful in the future.

1 Like