Siamese Network DataBunch

(mohamed amin houidi) #1

I am trying to make a custom image list to create a databunch for a siamese network. i found this very helpful notebook:

however this uses the version 1.0.39 of fast ai. now, in the latest version, ImageItemList was replaced by ImageList. after changing that i get this error:
’SiamImageItemList’ object has no attribute 'xtra’
which is instigated by these lines in the SiamImageItemList:
imgs = self.xtra.Image.values
ids = self.xtra.Id.values

so what was the xtra attribute chnaged into?


(matej) #2

this is what I found in docs

It inherits from ItemList and overwrite ItemList.get to call open_image in order to turn an image file in Path object into an Image object. label_cls can be specified for the labels, xtra contains any extra information (usually in the form of a dataframe) and processor is applied to the ItemList after splitting and labelling.

1 Like

(Harry Coultas Blum) #3

I’ve been working on creating some simple abstractions to help with Siamese Networks:

Have a look and feel free to make PR’s althoughI am going to commit a better version this week with examples of how to work with images (currently it focuses on audio)

1 Like

(mohamed amin houidi) #4

update: xtra was an attribute in the old version of fast ai v1. now its called inner_df.
jeremy should at least document such API breaking changes, its really hard to keep up with these, especially since they take place ever so often, and i had to dig deep into the docs to find out.

1 Like

(Harry Coultas Blum) #5

I’d also like to have a graphical representation of embeddings as shown in the notebook you linked. This is extremely useful in visualising the results.


(mohamed amin houidi) #6

thanks. i’ll also make sure to post my work on that thread when i’m done. siamese networks are extremely useful, and it would be great if fast ai would incorporate them.


(Harry Coultas Blum) #7

Yes I’m very interested in them too! I’m almost finished with a repo that will help work with them with fastai. Would be great to get some help :slight_smile:


(mohamed amin houidi) #8

i’m afraid posting my work is all i can do now, since i’m in the middle of my graduation project.but i hope after that i can contribute to your repo and make a pull request :smiley:


(FangChung) #9

This siamese architecture by fastai is useful for me, thanks a lot.
And I have another question about how to prediction:
for example, in Jeremy’s course Image classification, it did prediction just like

pred_class,pred_idx,outputs = learn.predict(“img.jpg”)

So, how can I implement siamese prediction just like below?

pred_target, dist = siam_learner.predict(“img1.jpg”, “img2,jpg”)



(mohamed amin houidi) #10

i believe that will not work for you. that shouldn’t be the a problem tho, you can easily implement prediction in pure pytorch. But prediction for siamese is going to be different in the sense that you will need a support set (images with known classes) and a test set containing test images each of which you will compare compare to the images in the support set and find the most similar one. thats how siamese networks do predictions. so you can see that fastai’s predict method will not work for this.