Low accuracy on toxic comments dataset with simple LSTM model

I have created a simple LSTM model for the kaggle’s jigsaw-unintended-bias-in-toxicity-classification datset.

But the accuracy is very very low, seems like the model isn’t learning anything at all. Can you help me figure out what is wrong with my model?

Here’s notebook of my work : https://nbviewer.jupyter.org/gist/aashudwivedi/e2e44f7085089743cf5086e23863fe65

You might be using the wrong loss function. It looks like you’re telling the model to run NLLLoss on raw logits.

Thanks @KarlH using CrossEntropyLoss appears to have fixed it. Thanks a ton!