What does the encoder actually learn? 🤔

(Arvind Nagaraj) #65

Hey, are you using the latest fastai version?
can you try with fastai version 0.7 please?

(Serbulent Unsal) #66

Thanks for answer. I am using 1.0 and I think I figure out this as;

resizeLength = len(sentenceTrimmed)
inputSentence = tensor(sentenceTrimmed).resize_(resizeLength,1)
# sentence encoding 400 dims. -1 is the last element that’s supposed to have the final encoded state
tmpEmbded = m[0](inputSentence.cuda())

I am also trying to convert your LM Evaluation notebook to fast.ai 1.0 and will share it by citing you if it is ok for you.

(Arvind Nagaraj) #67

Awesome. Sure, thanks.

(Serbulent Unsal) #68

And it doesn’t go well of course :slight_smile:

I can calculate distances on a pretrained model but they are wrong even for simplest words. Does anyone have an idea what could go wrong ?

(Serbulent Unsal) #69

One mistake that I made might be using LM encoder but even after I fixed it similarities are still not right even for words. For example word woman seems more similar to king than queen.

woman with king
woman with man
woman with woman
woman with queen