Collaborative filtering with cross entropy loss

So i’m trying to solve collaborative filtration problem with categorization approach, as been asked in ‘Further research’ section of Lesson 8:
Create a model for MovieLens that works with cross-entropy loss, and compare it to the model in this chapter.

I made a model and managed to train it.

Here is the notebook:

Questions:

  1. Although it trains without throwing an exception, results below show that it doesn’t actually learn anything. Error rate stays at ~40-60% no matter how many epochs it trains. Why is that?
  2. Another strange thing: if i do learn.get_preds() after training, preds[0].sum() actually does not sum to 1.

The model (please see the notebook for full detailed info):

class CollabNNCE(Module):
    def __init__(self, user_sz, item_sz, n_act=250):
        self.user_factors = Embedding(*user_sz)
        self.item_factors = Embedding(*item_sz)
        self.layers = nn.Sequential(
            nn.Linear(user_sz[1]+item_sz[1], n_act),
            nn.ReLU(),
            nn.Linear(n_act, 5)
        )
        
    def forward(self,user,movie):
        users = self.user_factors(user)
        items = self.item_factors(movie)
        facts = torch.cat((users,items),dim=1)
        return self.layers(facts)

Please give me a hint here, im kinda stuck and exhausted

I figured it out, the thing is, preds[0] should sum to 1 after softmax. But we are taking LogSoftmax, so they do not sum to 1.

As for the training, i tried different approaches and the best seems to have no NN or small NN with 10 neurons on one hidden layer. This way it trains smoother on LR = 0.04. But still error_rate metrics stays too high.
Im going to dive deeper into it. Maybe construct another metric which shows distance to the right answer from the given answer, like MSELossFlat, since we can treat this problem as regression (at least from metrics perspective)