Higher MSE with categorical variables and 0% accuracy

Hello everybody,

I’m new in the world of Deep Learning and thanks to the fast.ai course I’m building a neural network to estimate the yield for corn crops. My model is based on continuous and categorical variables. I have two problems:

  • the first one is that the accuracy is always equal to 0 but the MSE varies,
  • the second one is that when I add the categorical variables into my model (my categorical variables represent the type of soil and the previous crop which are very important in estimating the yield), it lowers the MSE (I mean my model gets worse). But the logic would be the opposite. So it is probably an error coming from me. Do you have any ideas? Maybe the emb_szs value is not good or the number of layers.

Thank you!

Any ideas?

Thanks.

So you’re using MSE and accuracy? That should be a mistake. MSE is for regression and accuracy is for classification. It sounds like you’re doing regression so you should just use MSE. Also make sure you’re using a y_range on your model equal to the maximum value you expect along with 0 for a minimum. This should help some. (See Rossman example for how we did this)

1 Like

Thank you, I will try to set a y range to see if it improves the MSE.

For the embedding size I used the following rule of thumb: min(600, round(1.6 * n_cat_atxr**0.56)). But is there something similar for the layer size ? I’m using a 500:200 layer size for my model. I have about 7000 rows and 11 independent variables (including 3 that are categorical).

Layer size is pretty much up to you. There’s a post of mine where I found a formula (do a search for finding the right layer size) but I generally try 200,100 if there’s not many variables and 1000,500 if there are

1 Like

One last question. Sometimes, if I do a 5 epochs training, I get a pretty good MSE at the 3rd epoch for example and then it goes back to a higher value. Is it because I have a too high/low learning rate?

Possibly yes. You could use the SaveModelCallback to save the most recent model and evaluate it on your test set to see how it did

1 Like