My PyTorch model is predicting only zeros

Deep Learning
1

Hi,
I have again problem with my PyTorch code. I just started learning PyTorch so probably this is easy to solve for pros.

class NeuralNetwork(nn.Module):
    def __init__(self, num_classes=1):
        super(NeuralNetwork, self).__init__()
    
    self.layer1 = nn.Linear(num_input,num_classes)
    
def forward(self, x):
    out = self.layer1(x)
    return out



# Loss and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)


total_step = int(len(x_train) / batch_size)
for epoch in range(epochs):
    for i, (x,y) in enumerate(train_loader):
        # Forward pass
        outputs = model(x.float())
        loss = criterion(outputs, y[:,None].float())
    
        # Backward and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    
        if (i+1) % 100 == 0:
            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}' 
                  .format(epoch+1, num_epochs, i+1, total_step, loss.item()))


from sklearn.metrics import r2_score
# Test the model
model.eval()  # eval mode (batchnorm uses moving mean/variance instead of mini-batch mean/variance)
with torch.no_grad():
    r_squares = []
    for x, y in test_loader:
        outputs = model(Variable(x).float())
        _, predicted = torch.max(outputs.data, 1)
        y = Variable(y).type(torch.LongTensor)
        print("y:",y)
        print("y_hat:",predicted)
        r2_score(y.numpy(),predicted.numpy())
        print("r2:",r2_score)
    

    print('Test Accuracy of the model on the 10000 test images: {} %'.format(np.mean(r_squares)))

So I printed the predictions and all were zeros.

2

You have not said anything about inputs or outputs, so it’s hard to tell. May be your labels (Y) is predominately 0’s with only a few 1’s so its natural for the network to gravitate towards zero.
This is more pronounced because you are using MSELoss here that calculates average squared loss. You can try -

  1. Change loss to nn.CrossEntropyLoss
  2. See the distribution of y labels by np.unique(y, return_counts=True)

There are other things to look at like your Learning Rate and How you losses are changing, but I would suggest you start from above two steps. At somepoint you want to visualize X, Y as well to see if you need a more complex function (Additional layers) to separate the classes.

1 Like
3

Sorry late respond. So ys are log of sale prices so most of them are between 12 and 13. Problem is regression so I try to predict one value. I think if you can’t see any problems with my model the problem might be importing data. I try to find the error and if I can’t find anything I will post more information. It was good to know that the model look like what it should.

4

Print out x and y data from your dataloader. See if they look right.

5

It was something wrong with my data importing. I solved it now. Thanks for help Karl and Ramesh.

6

Hi, how did you solve it? I am facing same problem.

7

It’s long time since I wrote that. There is no data importing in the code so probably the code you are seeing works but it is missing important pieces. If you are interested of this kind of simple example I recommend reading this https://adventuresinmachinelearning.com/pytorch-tutorial-deep-learning/ or some other blog. Just search “simple pytorch example” and you will find something. In my opinion, having this kind of simple example help a lot when you later build more complex models you can just realize that the structure is still the same.