Not sure what is going on. The first time I run .fit() it immediately displays the training loss as lower than the validation loss, no matter what numbers I put for the parameters. Any idea why this could be? I understand this means I’m overfitting, but how is it happening the first time I train, and on the first iteration?
I’ve tried drastically increasing the dropout rates to no avail.
A few lines of code that may be useful:
m = md.get_learner(emb_szs, len(df.columns) - len(cat_vars), 0.04, 1, [500,250], [0.04,0.4],
m.fit(lr, 2, wd, cycle_len=1, cycle_mult=2)
Any information would be very helpful!