Siamese Network DataBunch

I am trying to make a custom image list to create a databunch for a siamese network. i found this very helpful notebook:

however this uses the version 1.0.39 of fast ai. now, in the latest version, ImageItemList was replaced by ImageList. after changing that i get this error:
’SiamImageItemList’ object has no attribute 'xtra’
which is instigated by these lines in the SiamImageItemList:
imgs = self.xtra.Image.values
ids = self.xtra.Id.values

so what was the xtra attribute chnaged into?

this is what I found in docs

It inherits from ItemList and overwrite ItemList.get to call open_image in order to turn an image file in Path object into an Image object. label_cls can be specified for the labels, xtra contains any extra information (usually in the form of a dataframe) and processor is applied to the ItemList after splitting and labelling.

1 Like

I’ve been working on creating some simple abstractions to help with Siamese Networks:

Have a look and feel free to make PR’s althoughI am going to commit a better version this week with examples of how to work with images (currently it focuses on audio)

1 Like

update: xtra was an attribute in the old version of fast ai v1. now its called inner_df.
jeremy should at least document such API breaking changes, its really hard to keep up with these, especially since they take place ever so often, and i had to dig deep into the docs to find out.

1 Like

I’d also like to have a graphical representation of embeddings as shown in the notebook you linked. This is extremely useful in visualising the results.

thanks. i’ll also make sure to post my work on that thread when i’m done. siamese networks are extremely useful, and it would be great if fast ai would incorporate them.

Yes I’m very interested in them too! I’m almost finished with a repo that will help work with them with fastai. Would be great to get some help :slight_smile:

i’m afraid posting my work is all i can do now, since i’m in the middle of my graduation project.but i hope after that i can contribute to your repo and make a pull request :smiley:

This siamese architecture by fastai is useful for me, thanks a lot.
And I have another question about how to prediction:
for example, in Jeremy’s course Image classification, it did prediction just like

pred_class,pred_idx,outputs = learn.predict(“img.jpg”)

So, how can I implement siamese prediction just like below?

pred_target, dist = siam_learner.predict(“img1.jpg”, “img2,jpg”)


i believe that will not work for you. that shouldn’t be the a problem tho, you can easily implement prediction in pure pytorch. But prediction for siamese is going to be different in the sense that you will need a support set (images with known classes) and a test set containing test images each of which you will compare compare to the images in the support set and find the most similar one. thats how siamese networks do predictions. so you can see that fastai’s predict method will not work for this.

I have been troubled by this problem for several days, have you solved it?

as @mohamed1said, it can’t be implemented by fastai function directly. you need to use pytorch to achieve that. you should process your image to tensor format:

  1. crop image to fulfill network size.
  2. normalize image (cause ‘dist’ should in a range e.g. [0, 1] so that you can do some strategy)
  3. transfer images to tensor format and add 1 dimension (cause the network we train is using batch data, the network expect your input is [batch_size, channel_size, pixel, pixel], so you should add 1D be [1, channel_size, pixel, pixel]
  4. you should determine ‘pred_target’ based on the threshold, the threshold will be calculated by validation data.
1 Like

Thanks for your patience, I will try.

I have used pytorch to create a dataset, and fastai’s find_lr and fit_one_cycle to train a siamese network. It might be useful.

Here is the link to code.


@abhinavt Thats great well done :slight_smile: