I have a big datasets of instagram photos and I would like to find ways to analyze them by grouping them into sub-category, like “scenery”, “people”, “party”, “building” and so on, as detailed as it can. And each image could appear in multiple sub-category. Anyone aware of any good algorithms/applications for this task?
I would try to use whatever semantic information came with the images, comments, tags, names, metadata, etc. They should provide some indication of labels. Perhaps you have enough there to run a topic modelling and cluster images that way.
Another approach is the one Jeremy mentions somewhere in DL part 2, following Frome’s work on using semantic embeddings in image classification. I guess what you could do there is after you’ve clustered your images and trained your classifier, you could ask image categories from unseen labels (i.e. words, or better the vector representing that word from a pretrained model like fasttext).
DeViSE paper: https://papers.nips.cc/paper/5204-devise-a-deep-visual-semantic-embedding-model
Follow up: https://arxiv.org/abs/1312.5650
1 Like