Unsupervised/semi-supervised deep learning models

I am looking for applying unsupervised (and preferably semi-supervised) deep learning models on a dataset with a very small set of labeled data (less than 1% labeled).
The data is not image (column-base business data).
What kind of models I can start playing with?

Check out One-shot Learning Paper
http://arxiv.org/abs/1605.06065

Go through its references. There are tons of papers that go through semi-supervised learning approaches, but one-shot learning is the most label efficient.