Regularization for a seq2seq attn model applied for Time Series using fastaiv1

nisar009 · January 24, 2019, 9:18am

Hello,

I am currently trying to train a seq2seq attn model for time series ( stock prices) forecasting. I am using a customized version of this model from fastai part 2.

github.com

fastai/fastai/blob/master/courses/dl2/translate.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Important: This notebook will only work with fastai-0.7.x. Do not try to run any fastai-1.x code from this path in the repository because it will load fastai-0.7.x**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },

This file has been truncated. show original

also created a custom ItemList and DataBunch for Time series.
using the RNNLearner from fastai.text to create the learner and train.

The problem the model looks like it is heavily overfitting.

I have used self.out_drop = nn.Dropout(0.8) from the notebook from part 2. to get this results in the screenshot.

How can i regularize the model more to reduce the overfitting?

Regards,
Nisar