Lesson 4 - Further Research 1

Part 1 (2020)
1

Hello! I’ve completed rewriting parts of Lesson 4 to create my own Learner class. The link is on GitHub here https://github.com/nalhilal/ml_projects/blob/main/LearnerImplementation/Learner.ipynb

I ran into a few problems along the way, but I think I managed to get it to work, or at least it looks like it works.

I implemented my own validate loss function because I wasn’t sure exactly how to get the loss that prints using the fastai Learner so I made a guess in the function _log_validation_loss as so: def _log_validation_loss(self, xb, yb, model):

def _log_validation_loss(self, xb, yb, model):
    preds = model(xb)
    loss = self.loss_func(preds, yb)
    self._vloss = round(loss.item(), 4)
    return self._vloss

The call for the function is run in the training loop in thetrain_epoch() function:

    for xb, yb in self.valid_dl:
    self._log_validation_loss(xb, yb, model)

Does this function do what I want it to? Get the loss of the validation set?

I would really appreciate if someone can take a look over the code I posted for any advice. I’m hoping to use this code for the second research question.

2

Hello @nalhilal
I was working out the Further Research section of Lesson 4 and your Notebook that you linked has been of great help!

Yes, I think it does what you want it to do.

However I have a question. I see you implemented your own version of the DataLoaders class.
Inside it, you have a function called get_valid_dl, that returns shuffled validation data. I was reading the peer reviewed fastai paper and there is a part that mentions:

validation

I think the validation data shouldn’t be shuffled but I may be wrong. What do you think?

3

Hi @jimmiemunyi

Thank for your comment and I’m glad my notebook helped. I believe you’re right, I’m not supposed to use shuffle with the validation retrieval. After going through fastai the documentation I understand that when instantiating or reading from the DataLoaders class the retrieval of the training and validation dataset happens internally (as stated in the extract from the paper you posted) this is a step I didn’t implement at all.

In my example, the validation and training set are defined prior to instantiating the my_dataloader class where ideally and I believe the same applies to fastai, the DataLoaders class takes a dataset and then splits the data into train and valid based on the rules you require.

I can’t test it at the moment, but I’ll see how removing the shuffle affects the results, thanks for that.

1 Like