NLP Sentences Siamese Network

Hi all,

I am trying to build a sentences siamese network. I have done similar work in computer vision, but very limited understanding of the TextItemList.

I have tried quick hack solution by change the data_loader, but I think the problem is the collate function… Since the data structure I have now is ([seq_1, seq_2),label), collate function doesn’t know what to do with the list.

I am wondering if anyone has any inputs, I can simply override the pad_collate function for TextItemList, but the problem is I have no idea what kind of structure I should return…

A quick code of what I have is:

SiameseDataset just returns a datastructure ([sentence_1,sentence_2],label).

data_siamese = (TextList

train_dl = DataLoader(
dataset = SiameseDataset(data_siamese.train),
batch_size = 16,
shuffle = True,
num_workers = 0)

valid_dl = DataLoader(
dataset = SiameseDataset(data_siamese.valid),
batch_size = 16,
num_workers = 0)

data_siamese = TextClasDataBunch(train_dl,valid_dl)

When I call x,y = next(iter(data_siamese.train_dl)), the code will crash give me:

invalid argument 0: Sizes of tensors must match except in dimension 0

Which I think the problem is pad_collate… now the sentence is in different size after numericalize…

Any inputs are highly appreciated…

I am not sure if this will help you, but @brian made a siamese network using V.7. I haven’t been successful in using in getting it to work with v1.xx I was going to try to recreate in 2.0.

If you you get it working I would lo love to see it!


Thanks for the input.

I will see if I can make it work… hopefully v2 will have a easier way to hack around…

I hope so too, If I get it working I will let you know.

I’m looking to get back into Fastai and see about updating that Siamese model. @Daniel.R.Armstrong did you make some progress?

@brian I ran into a few roadblocks, so I decided I was going to wait till V2 was ready, to give it try. I will let you know if get it working.

1 Like