RMSPE oscillate in Rossmann notebook _ lesson 4

[dhoa (Dien-Hoa)]

I post the same question at Wiki_lesson4 and get the answer. The reason why the exp_rmspe is so small comparing to the original notebook is because of setting val_idx=[0] before learning. If someone try to run the notebook in order, should comment this line to get the proper results.

@KarlH

The notebook has several different ways of defining the validation indices. If you just run the notebook cells in order, you set val_idx=[0] (what you do before training with the entire dataset) before forming your model. If you ran the model with a single validation index, you would be comparing your rmspe to a single value. See what your validation set is, and if it is a single value, try change it to the last two weeks of data via:

val_idx = np.flatnonzero(
    (df.index<=datetime.datetime(2014,9,17)) & (df.index>=datetime.datetime(2014,8,1)))