Index Error when creating an Interpretation from a saved text classifier

fastai
1

I have trained an ULMFiT text classification model and then saved it like
learn.export(path / ‘models’ / ‘trained_model.pkl’)
Then, I ran

def predict(text):
    path = Path(r'D:/Python/NLP/FatAcceptance/Training/Final/ULMFiT')
    learn = load_learner(path / 'models', 'trained_model.pkl')
    interp = TextClassificationInterpretation.from_learner(learn)
    interp.show_intrinsic_attention(text)

but I get an
IndexError: list index out of range

What am I doing wrong?

2

Alternatively, is there a way to save a TextClassificationInterpretation object?

3

In case anyone in the future is interested in this, I was able to get it to work by doing

def predict(text):
    random_seed()
    path = Path(r'D:/Python/NLP/FatAcceptance/Training/Final/ULMFiT')
    learn = load_learner(path / 'models', 'trained_model.pkl')
    test = pd.DataFrame([text], columns=['text'])
    learn.data.add_test(test['text'])
    preds, y, losses = learn.get_preds(ds_type=DatasetType.Test, with_loss=True)
    classes = ['Support', 'Oppose', 'Unclear']
    print(classes[preds.argmax(dim=1)[0].tolist()])
    interp = TextClassificationInterpretation(learn, preds, y, losses)
    html = interp.html_intrinsic_attention(text)
    print(html)

The reason I use ‘html_intrinsic_attention’ is because ‘show_intrinsic_attention’ only works in jupyter notebooks, and I’m using vscode, so i have to render the html myself

1 Like