Text exploration and cleaning with UMAP

[wgpubs (WG)]

@morgan

that someone had found richer/more meaningful representations from BERT when they concatted the embeddings from the last 4 layers.

That would be here: The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.

@morgan

Playing around with tools like NMSLib/annoy was fun but unless you need an index for something like high volumes of search queries in production its probably not worth the time testing and waiting for it to generate an index for a lot of data.

Yah I’ve played with this before and was nice because I’m dealing with several hundred/thousands documents at a time (for example, all the comments for a survey).

Will check out UMAP and report back.