Is it possible to learn word or sentence embedding using lamguage model trained on wiki?
i was thinking about language model and how it was able to predict next word.Now idea that struck me was will it be possible to get a score for sentence out of model for use in sentence comparison.
ideally
sentence[w1…wn] ->language model-> wn+1
and
sentence[w1…wn] ->language model-> classifier+sigmoid ->0,1
could it be something like
sentence[w1…wn] ->language model-> +??? → sentence representation[1212,1521515,0212,451]
I know this is advanced topic and i found discussion going on in advanced forum on same here but just wanted to ask is it worth pursuing?