If you mean the IMDB sentiment, given your accuracy is around 0.5 you’re looking at a random accuracy. Most likely the order of your dataset is getting mixed up somewhere and you’re comparing to the wrong prediction.
Yes, IMDB sentiments. I run the notebook without other changes. Just adding the code for the accuracy evaluation at the bottom. Can someone else try this?
Solved - it looks like the problem is in the sampling of the dataset: val_dl = DataLoader(val_ds, bs, transpose=True, num_workers=1, pad_idx=1, sampler=val_samp).
The following code with removed sampler gives the expected results: