Thats nice work! Is this “altocumulus”? i opened that web site with my phone and snap the photo you posted. If it doesn’t take you much work, try to deploy it on a mobile device and have it run natively. for iPhone 7S +, the running speed is very respectable, and this allows making inference directly with the video feed. In my opinion, this is another level of UX than the website. Just imagine users pointing their phone and scan the sky, and have the cloud type reported in realtime!
A question I have, do you know of any dataset for identifying rock type? i.e. sedimentary, Igneous, and maybe rough estimate of mineral content (e.g. red looking one has high iron content). This will be highly educational, and for budding geologist, a must have going out in the field. I dreamt of building this ever since i heard that deep learning can make this much more reliable, but i am constantly distracted. I would be surprised if no one has made one yet.
If i have your app as well as this, I can tell whats above me, and below me…
Hi everyone! I created a Pokedex. It’s currently available here. You can input an image of any Pokemon and it tells you what Pokemon it is and you have the option to view more information about the Pokemon after that.
Unfortunately it is a single-label image classifier as I couldn’t find any multi-label Pokemon datasets out there and labeling them myself would be a long process.
Technical Details
It’s trained on resnet34, with just horizontal flipping for the images for data augmentation, and I managed to get an accuracy of 87% which I feel is quite alright for the quality of data that I have.
The dataset used to train was built through using Bing Image Search API to download around 90 images of each Pokemon, amounting to around 70,000 images before cleaning.
A lot of data cleaning had to be done as some images were duplicates of one another, especially for those Pokemon that usually appeared together, or those that evolved from one another. I modified the ImageCleaner code to automatically move around the relabeled images to the correct folders so that cleaning was easier. But even then, there’s still quite a bit of uncleaned data because of the sheer number of images.
Code for the webapp is available through the links inside the webapp. I can provide the code for my model if anyone wants it too.
Great fun
Regarding multi-label, its a bit of work but perhaps you can create it.
The algorithm could be something like:
Take your single-label dataset
Take a random 1-n-sized subsample from it, getting 1-n one-label items (where 1 is what you have now and n is max number of samples you support in multilabel)
Generate a new image from these 1-n images (for example blank canvas, random or roughly random positions where you insert images from 2…Imagine it like throwing cards on a table and then photographing the table). Label it with labels you already have from 2.
Train
…
PROFIT!!!
In theory, it could possibly work. In practice, well theory and practice can differ
I shall try a version of what you are suggesting using Pillow over the next few days.
But I’m not sure of how accurate the eventual recognition for actual multi-Pokemon images (such as drawings, posters, screen captures from the anime) would be, considering the fact that they tend to be in different positions or be hindered by parts of other Pokemon.
Hi everyone! I’m working on what I’ve learnt from lesson 1 and I wanted to create a classifier which could detect if two images are a visually similar. So I would have two folders: good matches which contains pairs of images which are similar and bad matches which contains image pairs which are dissimilar. I would then be able to submit any pair of images and the classifier would predict whether they are a good match or not.
Is there a way to set up the classifier so that it takes input as a pair of images instead of just a single image?
Thank you Glad you like it.
I’ve been thinking about a mobile app but will probably postpone it or let it pass, as I currently suck at mobile development and would probably waste a day or two at it learning the ropes, when someone half-competent could do it in a couple of hours. Also, id prefer it being cross platform (eg. Xamarin) and I am under no illusions that Im good at UI.
That being said if anyone is up for making a nice looking Xamarin shell of a mobile app,it should be relatively easy to integrate it and I’d love to do so.
Also, and perhaps more interestingly, such a project would be a usefull mobile starter for all the OTHER projects (so like Jeremys’ starter but for mobile) and looking at it that way it would be extremely useful.
Re sediments, like the idea. Dont know where you could get data, though. Sorry
Probably the easiest way would be to put them side by side (so, create a new image combining two alike, or two not alike images). Label according to type.
If i was doing it (it is slightly unusual) id do a small proof of concept not to waste too much time collecting data. Also id take care to exclude image order, so lets say A and B is similiar C and D are not. You’d want to create:
AB, similiar
BA, similiar
CD, not
DC not
(Note that I’m doubling each pair,as you are stiching images but dont really care that AB and BA are differently ordered)
I am familiar only with Apple Core ML, and only played around with tensorflow lite once.
So i like to state this from my own experience. It is easy only in a few simple case, (eg. single label classification), as soon as you venture off a bit, the difficulty and technical requirement will skyrocket very steeply. So very few ppl will do anything real and new in a couple of hours. Worse if direct GPU programming is required, then even fewer ppl have all these intersected skill set. The good news is that at this stage, it doesn’t take very sophisticated UI to have something effective, and impress users.
I aint familiar with Xamarin, but cross-platform AI mobile is a bit early given a single platform dedicated stuff is hard enough. Tools are still very buggy. ONNX may offer a bridge to that when it eventually comes. I would focus on single platform for now. I assume you only need to take care of CoreML and Tensorflow Mobile.
I think Andrej Karpathy had talked about a summer course last yr where they show how to do a complete end to end: Data -> Training -> Test -> Deploy Mobile. I don’t know what mobile platform he used though. Yes, it is an interesting exercise to provide a template project to do at least a single label classification. For apple, it is almost a drop in/dragdrop sort of thing using their sample app. The only hazard is to migrate the pytorch model to apple core ml model.
Hi Kelvin,
Yes, that is one of the challenges I was curious to see if deep learning could tackle. I wrote about it a bit in the notebook I used for research and plan to post it after it has a little more polish (draft linked in my original post). Dinosaurs are depicted in a variety of ways, yet we can still look at the different drawings of t-rexes and triceratops and know what they are. Can a deep learning algorithm pick on the same traits that we do?
It is not really related to the Deep Learning stuff but recently I realized that I am using argparse module a lot while writing various data preprocessing scripts, simple servers to host my models, etc. So I’ve decided to create a little guide on how one can use standard Python’s library to implement a CLI interface for their scripts and utilities (including a small overview about more high-level third-party solutions) as a single reference point for myself.
A very handy thing when you need to write a simple script to preprocess your data. Probably the post could help one to better understand what is going on in this built-in fastai’s snippet intended to help wrap Python’s utils with CLI. Especially, some interesting tricks about argparse that allows one to write a custom and a quite flexible solution.
Hi Shawn,
Well, the funny thing is no one had ever seen a hairy dinosaur. ;-). You can try your app on depiction of hairy ones. If it doesn’t perform well, i am guessing you need “add hair” data augmentation API. I don’t mean this as a joke. There are some researchers looking into super-sophisticated data aug coming from the video game industry.
This is my latest kernel on classifying Audio using Deep Learning. Found an Interesting Technique on Converting Audio to Spectrogram and to my surprise trained a model with 97% Accuracy. Do check this out if you’re working in the audio space.
Augmentation is one route and an interesting idea. But another approach is just to ensure that the test set includes dinosaurs drawn in different ways. My training set did have a few dinosaurs with hair and feathers, so it should perform well with them. Unfortunately all of the images I’ve found to test it are already in my training set.
