Concatenation of two models

kontrabas · January 28, 2020, 2:31pm

Hello, I’ve trained two neural networks (both are ULMFiT) with two different fine-tunes, first on them is prepared for binary classification and second for multi-label multi classification.
Is there possibility to somehow concatenate outputs from encoders of them and add binary classification on top?

Best regards

Hadus · January 29, 2020, 1:10pm

So do you mean something that takes the output of them as input?

kontrabas · January 29, 2020, 5:01pm

Yes. I’ve made something similar to this:

gist.github.com

https://gist.github.com/joshfp/b62b76eae95e6863cb511997b5a63118

1.tabular.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model based on tabular data only"
   ]
  },
  {

This file has been truncated. show original

2.lm-spanish.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ULMFiT: Train spanish LM"
   ]
  },
  {

This file has been truncated. show original

3.nlp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NLP model to predict from title (ULMFiT)"
   ]
  },
  {

This file has been truncated. show original

There are more than three files. show original

But instead tabular network, I’ve inserted my AWD_LSTM. And in the end I’ve met another issue - I don’t know how to predict single example with .predict(example), because I’ve got this error:
AttributeError: ‘ConcatDataset’ object has no attribute ‘set_item’