Help needed in implementing a BERT-based-uncased followed by a simple lstm model

I am trying to implement a multi-label classification model with BERT model as its primary structure. I successfully implement a model which used BertForSequenceClassification model with multi-label classification and decided to experiment a bit. So I defined the following model:

class BiggerModel(nn.Module):
def init(self , bert):
self.bert = bert
self.rnn = nn.LSTM(786 , 384 ,num_layers = 2, bidirectional = True)
self.dropout = nn.Dropout(0.1)
self.linear = nn.Linear(768,5,bias = True)
def forward(self,input):
x = self.bert(input)
x = self.rnn(x)
return self.linear(x)

bert = BertModel.from_pretrained(‘bert-base-uncased’)
But when I try to use learner.lr_find() I get the following error:

‘tuple’ object has no attribute ‘size’

What should I do to fix it?

1 Like

I have the same problem
Did you find a solution?