How is the threshold set for accuracy and f_score on a tabular model?

I’m trying to change the default threshold of 0.5 tabular model, but cant get it to work

    f1_score =FBeta(average='macro',beta = 1)
    acc_02 = partial(accuracy_thresh, thresh=0.2)
    f_score = partial(fbeta, thresh=0.2)
    learn = tabular_learner(data, layers=[1000, 200, 15], emb_szs=emb_szs, metrics=[acc_02,f1_score],emb_drop=0.1, callback_fns=ShowGraph)

I’m getting the following error:

RuntimeError                              Traceback (most recent call last)
<ipython-input-196-fd2bddb11935> in <module>()
----> 1 learn.fit_one_cycle(2, max_lr=slice(1e-01))

8 frames
/usr/local/lib/python3.6/dist-packages/fastai/ in accuracy_thresh(y_pred, y_true, thresh, sigmoid)
     33     "Computes accuracy when `y_pred` and `y_true` are the same size."
     34     if sigmoid: y_pred = y_pred.sigmoid()
---> 35     return ((y_pred>thresh).byte()==y_true.byte()).float().mean()
     37 def top_k_accuracy(input:Tensor, targs:Tensor, k:int=5)->Rank0Tensor:

RuntimeError: The size of tensor a (2) must match the size of tensor b (64) at non-singleton dimension 1

Furthermore, trying to change thresh at prediction time seems to not be working for me too:

print(learn.predict(train.iloc[99], thresh=0.1) )
print(learn.predict(train.iloc[207], thresh=0.1))

(Category 0, tensor(0), tensor([0.6813, 0.3187]))
(Category 0, tensor(0), tensor([0.7392, 0.2608]))

I have 2 categories 0 and 1. I would expect both predictions to be classified as ‘Category 1’

what I’m doing wrong?

I have a similar question/problem. Did you figure this out?

Did you end up figuring this out? I’m having a similar issue.