Hi,
Just wanted to share a working example of multi-label text classification that is working with Fast AI v1.
Did a quick search and I couldn’t see any clear examples of getting a multi-label classifier working.
This is useful when you have a passage of text/document that can have one of several labels or tags.
For example, a news article could have the tags world-news, political, election.
Your data should be in the form of a passage of text in one column and a string of labels with a separator in another column. For more details, check the dataset below.
After fine-tuning the language model on our documents we create a TextDataBunch
classifier_data = (TextList.from_df(df, path, cols=[‘words’,‘tag_list’], vocab=lm_data.vocab)
.split_by_rand_pct(0.2)
.label_from_df(cols=‘tag_list’, label_delim=’|’)
.databunch(bs=bs))
The key pieces to get the classifier working is providing the columns to import, the correct vocab from your fine-tuned language model, and labelling the data (tags) from a DataFrame column and passing the label delimiter.
Then you can just crate a text classifier as per usual.
learner = text_classifier_learner(classifer_data, AWD_LSTM, drop_mult=0.5, metrics=[fbeta])
Load your fine-tuned encoder.
learner.load_encoder(‘fine_tuned_enc’)
learner.freeze()
train the learner
lr = 1e-1
learner.fit_one_cycle(1,slice(lr/(2.6**4),lr), moms=(0.8,0.7) )
learner.freeze_to(-2)
learner.fit_one_cycle(1,slice(lr/(2.6**4),lr), moms=(0.8,0.7) )
Once you are finished fine-tuning the classifier you can check the predictions against a document from your validation sample.
article = classifer_data.valid_ds[0]
“(Text xxbos xxmaj my friend xxmaj kevin xxmaj taylor ( known to his family as xxmaj gordon )…
…
he had grown to love . xxmaj he is survived by his brothers , xxmaj ben and xxmaj alf , and three nephews . xxunk - news,
MultiCategory world;us-news)”
pred = learner.predict(article); pred
(MultiCategory us-news;world,
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1.]),
Have poped a rough working example notebook on github.
Workbook:
Dataset: