BiLSTM for Text Classification

kaustumbh7 · January 15, 2019, 5:46pm

I want to implement a Bi-LSTM for Text Classification using fastai. I watched the Lesson-10 video but am quite unsure about what changes I have to to do to the Lesson-10 notebook to implement a Bi-LSTM to improve the accuracy on my own dataset.

Can anyone please help me out?

florianl · January 24, 2021, 11:49am

you’ll have to train two models - one forward one backward and then ensemble both models for predictions. See fastai’s nlp course notebooks for an example:

Train language model + enable

github.com

fastai/course-nlp/blob/master/nn-vietnamese.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vietnamese ULMFiT from scratch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "from fastai import *\n",

This file has been truncated. show original

Train backward language model

github.com

fastai/course-nlp/blob/master/nn-vietnamese-bwd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vietnamese ULMFiT from scratch (backwards)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "from fastai import *\n",

This file has been truncated. show original

The notebooks are still fastai v1 - I created a repo to train a German ulmfit language model with fastai2:

Florian