RMSPE oscillate in Rossmann notebook _ lesson 4

[dhoa (Dien-Hoa)]

I’m learning the lesson 4 with Entity Embedding and there are something that I’m not understand. (I ran the notebook in order until Sample section)

• m.lr_find() to find the appropriate learning rate stop at 59%
  

  

• When I’m fitting the model, the loss and RMSPE are too small and it was not decreasing but oscillating not in any order
  

  

• For dropout parameter, I wonder why it is so small. emb_drop = 0.04 ; drops = [0.001,0.01]. It means that the model is very easy to overfitting because we drop almost nothing.

This is the first time I attack the structure learning. I am very appreciate if someone can help me to clarify these problem. Thank you so much

[dhoa (Dien-Hoa)]

I post the same question at Wiki_lesson4 and get the answer. The reason why the exp_rmspe is so small comparing to the original notebook is because of setting val_idx=[0] before learning. If someone try to run the notebook in order, should comment this line to get the proper results.

@KarlH

The notebook has several different ways of defining the validation indices. If you just run the notebook cells in order, you set val_idx=[0] (what you do before training with the entire dataset) before forming your model. If you ran the model with a single validation index, you would be comparing your rmspe to a single value. See what your validation set is, and if it is a single value, try change it to the last two weeks of data via:

val_idx = np.flatnonzero(
    (df.index<=datetime.datetime(2014,9,17)) & (df.index>=datetime.datetime(2014,8,1)))