I am trying to build a sentences siamese network. I have done similar work in computer vision, but very limited understanding of the TextItemList.
I have tried quick hack solution by change the data_loader, but I think the problem is the collate function… Since the data structure I have now is ([seq_1, seq_2),label), collate function doesn’t know what to do with the list.
I am wondering if anyone has any inputs, I can simply override the pad_collate function for TextItemList, but the problem is I have no idea what kind of structure I should return…
A quick code of what I have is:
SiameseDataset just returns a datastructure ([sentence_1,sentence_2],label).
data_siamese = (TextList .from_df(df=df_1,path=path,cols='Body',vocab=data_lm.vocab) .split_by_rand_pct(0.3,seed=42) .label_from_df(cols='Label') ) train_dl = DataLoader( dataset = SiameseDataset(data_siamese.train), batch_size = 16, shuffle = True, num_workers = 0) valid_dl = DataLoader( dataset = SiameseDataset(data_siamese.valid), batch_size = 16, num_workers = 0) data_siamese = TextClasDataBunch(train_dl,valid_dl)
When I call x,y = next(iter(data_siamese.train_dl)), the code will crash give me:
invalid argument 0: Sizes of tensors must match except in dimension 0
Which I think the problem is pad_collate… now the sentence is in different size after numericalize…
Any inputs are highly appreciated…