Sentence similarity

[brian]

@JensF Thank you for the reply.
I’ll use your example from the other post to double check my work.

I have used the lesson 10 model as the basis for creating vectors. I wrote up a bit of that in this post:

Understanding RNN hidden states: Building sentence vectors from ULMFiT

I’m trying to make sentence vectors to encode semantic similarity. The basic idea being that sentences with the same meaning should have the vectors that are close together. Here’s what I’ve done so far. Retrained the language model from lesson 10 IMDB using the text from the Stanford Natural Language Inference Corpus Verified that it trained properly by inspecting the sentences that it produces. New sentences sound like ones from the corpus. Training achieved a perplexity of 14.80. Ad…

I used the avgpool+maxpool+last technique to create the vectors, but they didn’t produce good results.

I’ve been re-reading some of the papers on this topic including Facebook’s InferSent.

I think that I’m going to try their approach of encoding 2 sentences then using a fully connected layer to classify them into entailed, contradictory or non-entailed. I’d be pretty happy with greater that 80% results like they claimed.