My PyTorch model is predicting only zeros


(Lankinen) #1

Hi,
I have again problem with my PyTorch code. I just started learning PyTorch so probably this is easy to solve for pros.

class NeuralNetwork(nn.Module):
    def __init__(self, num_classes=1):
        super(NeuralNetwork, self).__init__()
    
    self.layer1 = nn.Linear(num_input,num_classes)
    
def forward(self, x):
    out = self.layer1(x)
    return out



# Loss and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)


total_step = int(len(x_train) / batch_size)
for epoch in range(epochs):
    for i, (x,y) in enumerate(train_loader):
        # Forward pass
        outputs = model(x.float())
        loss = criterion(outputs, y[:,None].float())
    
        # Backward and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    
        if (i+1) % 100 == 0:
            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}' 
                  .format(epoch+1, num_epochs, i+1, total_step, loss.item()))


from sklearn.metrics import r2_score
# Test the model
model.eval()  # eval mode (batchnorm uses moving mean/variance instead of mini-batch mean/variance)
with torch.no_grad():
    r_squares = []
    for x, y in test_loader:
        outputs = model(Variable(x).float())
        _, predicted = torch.max(outputs.data, 1)
        y = Variable(y).type(torch.LongTensor)
        print("y:",y)
        print("y_hat:",predicted)
        r2_score(y.numpy(),predicted.numpy())
        print("r2:",r2_score)
    

    print('Test Accuracy of the model on the 10000 test images: {} %'.format(np.mean(r_squares)))

So I printed the predictions and all were zeros.


(Ramesh Sampath) #2

You have not said anything about inputs or outputs, so it’s hard to tell. May be your labels (Y) is predominately 0’s with only a few 1’s so its natural for the network to gravitate towards zero.
This is more pronounced because you are using MSELoss here that calculates average squared loss. You can try -

  1. Change loss to nn.CrossEntropyLoss
  2. See the distribution of y labels by np.unique(y, return_counts=True)

There are other things to look at like your Learning Rate and How you losses are changing, but I would suggest you start from above two steps. At somepoint you want to visualize X, Y as well to see if you need a more complex function (Additional layers) to separate the classes.


(Lankinen) #3

Sorry late respond. So ys are log of sale prices so most of them are between 12 and 13. Problem is regression so I try to predict one value. I think if you can’t see any problems with my model the problem might be importing data. I try to find the error and if I can’t find anything I will post more information. It was good to know that the model look like what it should.


(Karl) #4

Print out x and y data from your dataloader. See if they look right.


(Lankinen) #5

It was something wrong with my data importing. I solved it now. Thanks for help Karl and Ramesh.