Thanks for your response 
Sure, that makes sense. I mentioned the Data Blocks API because if I pass in a DataFrame like the one shown here, I get a KeyError when forming the dlock.dataloaders() since -100 is not part of the vocab for any of the columns.
Ah that would make sense, Iād possibly recommend a custom vocab then possibly if you want to go that route, otherwise no 
(pass said vocab to CategoryBlock(vocab=))
Hmm, I see. That is a bit awkward since if you already have a dedicated NA class, it doesnāt make sense. However, I guess if the loss function will ignore it anyways, it doesnāt matter ā it wonāt take up any part of the probability space in the Softmax activation?
Iāll give it a shot anyways and report back here.
Honestly not sure, thatās out of my realm/scope for right now
Keep us updated
Definitely.
Iām not sure how the library is calling the batches during training, because when I run the following chunk to the notebook, the shape for yb is definitely wrong
I think it would be a super neat addition to the notebook to add an example for how the batch is actually called during training, and an example CombinationLoss(), since given the above, I canāt test the CombinationLoss() function before actually training.
Itās exactly as you have it, so:
xb, yb = next(iter(dl))
Which is also what one_batch does
To further this on any loss function simply have it accept an input and output and print what they are 
(but you may already know this one)
Hah, Iām a bit confused. Iād love to just pop the xs and ys into the loss func, but I think my sizes are off.
To be precise, next(iter(dls)) returns a TfmdDL object and not a batch. Even after I grab a batch, I canāt plug it into the CombinationLoss() straight away.
Hereās the code Iām running:
dls = bengel.dataloaders(train.sample(1000), bs=bs)
batch = dls.one_batch()
xb = batch[0]
yb = batch[1]
# xb.shape ==> torch.Size([128, 1, 128, 128])
# yb.shape ==> torch.Size([128]
I expected yb to be a list of 3 elements with size=128 each, or a stack of 3 tensors of size 128. Looking at the contents of yb, it appears to me that these are only the ground truths for grapheme and not vowel or constant.
If I try running this through CombinationLoss() as such:
loss_func = CombinationLoss()
loss_func(xb,yb)
I run into a ValueError: Expected input batch_size (1) to match target batch_size (128).
To take a step back, am I understanding CombinationLoss()'s forward correct? Iāve jotted my thoughs in the comments
def forward(self, xs, *ys, reduction='mean'):
# xb.shape ==> (batch_size,1,128,128)
# what shape/type should `ys` be?
# list of 3 tensors of size (batch_size, truths)?
for i, w, x, y in zip(range(len(xs)), self.w, xs, ys):
# this loop will run three times, calculate the loss for
# grapheme,vowel,consonant and sum them up
if i == 0: loss = w*self.func(x, y, reduction=reduction)
else: loss += w*self.func(x, y, reduction=reduction)
return loss
dls are both DataLoaders, so this makes sense. You should do dls[0] (train), dls[1] (valid) or .train, etc
How are you building your DataLoaders? Whatās your DataBlock like? (Is it just my example notebook?)
Ahhh gotcha. That makes sense re. dls.
Yeah, Iām using your notebook, nothing changed, just ran these code blocks after the definition of the loss function.
Okay, Iāll look into and debug that notebook tomorrow then and get back with you, as that shouldnāt be whatās happening. Thanks!
Awesome. Thank you!
Actually, I made a silly error 
batch = dls.one_batch()
len(batch) # ==> 4
x = batch[0]
# ys = batch[1] # len ==> 1 (ofc!)
ys = batch[1:] # len ==> 3
On further thinking, this shouldnāt be done, because your model will have one parameter in the final layer that will be making predictions for the ignore_index element.
I think tweaking Categorize to support ignore_index is the correct approach.
Could somebody please remember me how to add the test dataset to the dataloader after the training?, i.e. my training data has been splitted into training and validation (originally in the same folder 80/20) and now I would like to know how well my model works in the test set which is in another folder i.e. get predictions and metrics.
test_dl = learn.dls.test_dl(items)
learn.validate(dl=test_dl)
@muellerzr if you have time can you please look into this? Ignore the code inside show_results, once it gets called I can work on that. Typedispatch system is what causing this error.
Iām not sure there. Sylvain would be more help, I havenāt faced that error before
I am building a simple cat vs dog classifier BUT using multi-cat example so that if the picture is not a dog or a cat I get not output. This is my learner and its threshold:
learn = cnn_learner(dls, resnet34, pretrained=True, metrics=[partial(accuracy_multi, thresh=0.9)]).to_fp16()
I trained the model and trained with a random pic. The prediction is NOT above 0.9 but stills says āDogāā¦ what am I missing in here?
Makes no sense to me
Thatās because thatās done on the loss function, not the metric So you need to adjust the threshold of the BCELossLogits loss function (I had this same question myself last week or so)
Can you explain a bit more? My understanding is that the loss function is what we are trying to minimize, the metric just a proxy for how good our model is working. So I would have said it belongs to the metric, not to the lossā¦
I re-trained the model againā¦ how can I prevent it about being so confident about the wrong class? I find this really concerning
