Attempting to use my trained text classifier for inference but I get about .5 for each of the eight labels I’m trying to predict (this is a multi-label problem).
This problem usually indicates data is being shuffled but I don’t see it. Of course, there may be some other critical piece I’m missing, but again, been playing with this for a few hours and no glory.
What am I doing wrong?
# use to get all the pre-processing funcs
data_inf = TextClasDataBunch.load_empty(CLS_PATH)
data_inf.vocab = vocab
data_inf.c = 8
learn = None; gc.collect()
learn = text_classifier_learner(data_inf, drop_mult=0.5, bptt=70, emb_sz=400, nh=1150, nl=3,
lin_ftrs=[50], ps=[0.1], alpha=2., beta=1.)
learn.model.add_module('2', MultiLabelClassifier())
learn.model_dir = 'models'
learn = learn.load(f'{lm_pre}cls_bestmodel{exp_suffix}')
learn.model = learn.model.to(device)
test_data = (TextList
.from_df(test_df_filtered, cols=['AnswerText'], processor=txt_procs)
.label_from_df(cols='labels', classes=LABELS_SENT[1:], sep=' ')
.process()
)
# build a DataLoader => fastai.basic_data.DeviceDataLoader
dl = DeviceDataLoader.create(test_data, bs=80, shuffle=False, collate_fn=pad_collate,
device=device)
b = next(iter(dl))
b[0].shape, b[1].shape
# (torch.Size([191, 80]), torch.Size([80, 8]))
preds = learn.pred_batch(DatasetType.Test, b)
preds[:10]
… and the output is:
tensor([[0.5002, 0.5040, 0.5002, 0.5219, 0.7222, 0.5000, 0.5000, 0.5050],
[0.5044, 0.6624, 0.5118, 0.7054, 0.5604, 0.5025, 0.5001, 0.5001],
[0.5004, 0.5037, 0.5861, 0.7245, 0.5123, 0.5006, 0.5002, 0.5023],
[0.5002, 0.5045, 0.5354, 0.7231, 0.5291, 0.5003, 0.5001, 0.5005],
[0.5000, 0.5014, 0.5434, 0.7277, 0.5032, 0.5003, 0.5052, 0.5013],
[0.5005, 0.5020, 0.5072, 0.6680, 0.7220, 0.5001, 0.5003, 0.5010],
[0.5024, 0.5485, 0.5013, 0.5066, 0.5056, 0.5002, 0.5002, 0.6098],
[0.5000, 0.5001, 0.5001, 0.5863, 0.7296, 0.5000, 0.5000, 0.5000],
[0.5077, 0.7145, 0.5012, 0.5060, 0.5035, 0.5000, 0.5006, 0.5031],
[0.5004, 0.5018, 0.5004, 0.5215, 0.7291, 0.5000, 0.5001, 0.5026]])