 # My PyTorch model is predicting only zeros

(Lankinen) #1

Hi,
I have again problem with my PyTorch code. I just started learning PyTorch so probably this is easy to solve for pros.

``````class NeuralNetwork(nn.Module):
def __init__(self, num_classes=1):
super(NeuralNetwork, self).__init__()

self.layer1 = nn.Linear(num_input,num_classes)

def forward(self, x):
out = self.layer1(x)
return out

# Loss and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

total_step = int(len(x_train) / batch_size)
for epoch in range(epochs):
for i, (x,y) in enumerate(train_loader):
# Forward pass
outputs = model(x.float())
loss = criterion(outputs, y[:,None].float())

# Backward and optimize
loss.backward()
optimizer.step()

if (i+1) % 100 == 0:
print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
.format(epoch+1, num_epochs, i+1, total_step, loss.item()))

from sklearn.metrics import r2_score
# Test the model
model.eval()  # eval mode (batchnorm uses moving mean/variance instead of mini-batch mean/variance)
r_squares = []
for x, y in test_loader:
outputs = model(Variable(x).float())
_, predicted = torch.max(outputs.data, 1)
y = Variable(y).type(torch.LongTensor)
print("y:",y)
print("y_hat:",predicted)
r2_score(y.numpy(),predicted.numpy())
print("r2:",r2_score)

print('Test Accuracy of the model on the 10000 test images: {} %'.format(np.mean(r_squares)))
``````

So I printed the predictions and all were zeros.

(Ramesh Sampath) #2

You have not said anything about inputs or outputs, so it’s hard to tell. May be your labels (Y) is predominately 0’s with only a few 1’s so its natural for the network to gravitate towards zero.
This is more pronounced because you are using MSELoss here that calculates average squared loss. You can try -

1. Change loss to nn.CrossEntropyLoss
2. See the distribution of y labels by `np.unique(y, return_counts=True)`

There are other things to look at like your Learning Rate and How you losses are changing, but I would suggest you start from above two steps. At somepoint you want to visualize X, Y as well to see if you need a more complex function (Additional layers) to separate the classes.

(Lankinen) #3

Sorry late respond. So ys are log of sale prices so most of them are between 12 and 13. Problem is regression so I try to predict one value. I think if you can’t see any problems with my model the problem might be importing data. I try to find the error and if I can’t find anything I will post more information. It was good to know that the model look like what it should.

(Karl) #4

Print out x and y data from your dataloader. See if they look right.

(Lankinen) #5

It was something wrong with my data importing. I solved it now. Thanks for help Karl and Ramesh.