I spent a lot of time experimenting with the steps to make sure they all still applied to PyTorch and fast.ai v1 since the article was written a while ago.
Hangups I had were
coremltools does not work with Python 3.7, I had to re-install everything with 3.6
I found some stuff indicating that the AdaptiveMaxPool stuff had been integrated into onnx-coreml by now and by re-building some things from source I was able to get the model converted without it complaining about these layers. This turned out to not work in the end (CoreML spat out a cryptic error which appears to mean that it was unable to put the inference ops on the GPU) and I had to start over again. As far as I can tell you do still need to add the MyAdaptive***Pool2d classes as the writeup describes.
I spent a good deal of time trying to use the CoreML preprocessing_args to get my real-world data to match the format the model was trained on. Mostly my problems were due to inexperience (I didn’t realize that I had to call .normalize(imagenet_stats) on my data prior to training so I spent a lot of time trying to match the actual data to something that the model wasn’t matched to at all). In the end, I re-trained my data with normalization and used @jalola’s ImageScale layer at the beginning of the network, then I added the ImageNet RGB bias values in that layer as well.
CoreML initially didn’t recognize my model as accepting an Image so I spent a good deal of time trying to get my data formatted as an MLMultiArray --> this is very poorly documented and I never did end up getting it to work properly. The solution was to add image_names and mode="classifier"like the writeup told me to do (I had skipped that step because I didn’t think it’d be important to get things working… I thought I’d come back and clean things up after I got something working… turns out it was an important step and is actually called out in the writeup; I just glossed over that part the first time).
I had trouble with the dummy_input shape. The writeup has a sneaky .unsqueeze(0) slipped into the ImageScale function. So when I got rid of that to experiment with preprocessing_args I had to re-shape the dummy_input.
In v1, the LogSoftmax layer is not actually included in the network so you don’t need to remove the last layer of the Resnet CNN before adding your Softmax layer.
So the above can basically be summed up as “if you follow the writeup it will go pretty smoothly”. But on the bright side, wrestling with it for a couple of days gave me a lot of understanding about what’s actually going on.
Ok, I have a discussion with someone here and we think it may be good to have a more update to date tutorial on how put people project on mobile device. Not sure if you are interested. I am not sure when I can come around to do this. I have experience only with Keras model.
Also, take a look at:
and chime in your thinking, since you are mobile app dev.
I used fastai to create a classifier for white blood cell images from a Kaggle dataset. I was able to get about 87% accuracy, which is not bad! However, I am struggling to improve those results so if someone has suggestions please let me know!
Ah, unfortunately I’m not doing any data augmentation yet. I have plenty of blurry data from my actual captures so it probably won’t be something I try in the near term.
I created a skin lesion classifier from the ISIC Skin Lesion Challenge data set. I can’t say it’s state of the art but I did get 94% accuracy on a 2,000 image holdout set. I’m really proud of it and how easy it was to get such great results so quickly. I’ll probably come back to this in time to try and improve upon it but I wanted to release it and see if anyone wanted to check it out and give any kind of feedback!
I have classified the oxford based dataset this time for flower and got less accuracy, I believe my images were blur and data was not prepared correctly. Any thoughts and feedback is recommend.
After lesson 4, I was really keen to try building my own NLP / tabular / collaborative model.
As luck would have it, I got an email from Kaggle today announcing a new competition with tabular data. The competition requires you to use tabular data to predict whether Santander bank customers will buy products in the future. This sounded quite similar to the tabular example from the lecture, so it seemed like a good problem to try out.
I built a tabular model, which had ~91.6% accuracy on my validation set (training with 4 epochs at a max learning rate of 5e-3)
One challenge I ran into was that Kaggle scores this competition with an AUC-ROC metric (AUC = area Under Curve, ROC = Receiver Operating Characteristics). I tried to add this metric myself, and did a bit of Googling to try to find usable code, but wasn’t able to get it to work
I submitted my model to Kaggle with a score of 0.862, which got me to position 678/1067 on the leaderboard. I might come back to this model in the future after I learn some more optimization techniques (e.g. I don’t know exactly what the ‘layers’ input does when you create a TabularLearner, so I just put in the same [200,100] value from the lecture)
After watching the first two lectures, I was fascinated with the .fit_one_cycle() method and curious about its inner-workings. After reading Leslie Smith’s papers multiple times, I put my learning into writing:
when you come around to try data aug, you can test if the gaussian blur will help your accuracy, even if you have such sample in your training set. i suspect this will help if the digit is small.
I also have a few blurry ones in my own training set, but adding a little bit of gaussian blur during training can boost accuracy, sometimes up to 0.5%.
Follow up, do you be plan to write a more up to date “how to” to take your fastai trained pytorch model to apple .mlmodel, and maybe a skeletal app to try it out?
i think this will be really useful to see if what everyone has done actually generalizes in the real world, except may be the image is hard to obtain in the 1st place (like MRI or x-ray image).
I may come around to do this when I have a chance. Here’s what I am thinking
Provide a notebook tutorial on how to do this.
Structure this such that a single API call to the learner (or whatever model), will output the apple .mlmodel, if possible
If coremltools only work in 3.6, just make a note in notebook to ask people to switch to python 3.6 when ready to convert. I know this is ugly and turn off.
Provide a skeletal Xcode project for 1-label multi-class image recognition, such that you can just drop in the mlmodel and label.txt, and run.
If I get enough like here, I may put this slightly higher priority. Don’t want to spend too much volunteer time on stuff ppl don’t care about.
