Thanks Erick!
I believe that the onecycle policy was the main responsible for the improvement. I don´t know if the new fastai has a different augmentation feature, but if this is the case, then it sure helped too.
Yes - agreed that some kind of deeper exploration would be interesting; visualizing the layers or even something like this distill.pub article
Right now I still have a problem with my train/val split and major leakage.Will fix first then explore those solutions. Also if you have any other ideas I’m a taker!
Superconvergence? Explain pls
Been working on a Classifier of 13 different Brands of Swiss Watches… got a 0.280851 error_rate, after 7 epochs and tuning the learning rate…will experiment with more epochs.
5 Likes
I have been working on American sign language dataset. I have used resnet34 model and I have got an accuracy of 99.97%. I have used opencv for making live prediction of signs made by hand via. webcam. Here are some of the top losses
Here is a video of my ASL-live-predictor:
Link to ASL-live-predictor github repository:
42 Likes
Here is another example of application of Fast.AI library to the problems in cancer genomic domain. The problem is discriminating between true and false variants detected by automatic workflows in tumor-normal cancer sequencing. Comments, questions are very welcomed, Thanks!
9 Likes
Well, that’s just pretty awesome.
I am impressed by 1 cycle policy and superconvergence methods do automatic so I write a blog on it and it’s is my first blog can fastai community give me feedbacks about blogs or am I getting the concepts correctly to relate superconvergence and regularization.
I just published SuperConvergence with inbuilt regularization -
https://link.medium.com/gLmCDpt7BR
Thank you for that post, that was very useful - here is what happened:
-
following Rachel’s post, I split my training and test set more rigorously (the key here was to have all the images of the same city either in the train set or the test set, and not in both). For anyone running into the same issue, split your sets yourself, then use
ImageDataBunch.from_csvorImageDataBunch.from_df. -
Unfortunately the result of this was a much lower test accuracy - around 60%; and no fiddling with the training seemed to improve the result. So I went and got more data
-
I downloaded data from the largest 25,000 cities in the world; around 100,000 labeled images in the dataset. Interestingly, resnet34 (which was better on the small dataset) turned out to be much worse than resnet50 on the large dataset. Resnet34 got to about 75% accuracy while resnet50 easily got to 80% Finetuning also helped a lot with this larger dataset, almost 5 points improvement.
Conclusion: with this new model, accuracy is now back to 85% (using rigorously split train and test sets).
The result is extremely pleasing, and the webapp is now almost creepy to try out; it seems to get almost everything right: yourcityfrom.space
11 Likes
I wanted to check how fast.ai initialized weights in CNNs and was surprised to see that it changed from last year!
So I wrote a blog post on “He Initialization” 
Weight initialization for CNNs: A Deep Dive into He Initialization
https://medium.com/@tylernisonoff/weight-initialization-for-cnns-a-deep-dive-into-he-initialization-50b03f37f53d
2 Likes
Nice job! Do you have any interpretation on how it’s classifing?
1 Like
IMO, for local installations it is always better to use docker wherever possible, to avoid messing up your installed packages. You don’t want to end up with incompatible versions. Cheers.
For a project like this I don´t see the need for a docker, I use multiple conda environments and it works just fine 
2 Likes
I think data augmentation will help in this situation.
1 Like
Yes, we have much trouble and I would say pain, getting the model converted to ONNX binary protobuf file which contains both the network structure and parameters of the model we exported. We are able to go from:
fast.ai + PyTorch 
ONNX Caffe2 Android devices.
The SqueezeNet model in plain PyTorch from torchvision runs fine on Android devices. Exporting fast.ai ResNet-family of models (we have tested ResNet18, a smaller model with lesser parameters and reasonable GFLOPS on mobile ARM processor/GPU(DSP)) through the ONNX exporter with Caffe2 backend proceed successfully (no error). However, when we put the 2 pb files on the Android app, it runs and crashed on Android devices. Our initial investigation found the crash could be due to discrepancies all over the ONNX specifications and around the tooling that follows ONNX.
The bottleneck is in this step: “Export the model to run on mobile devices”, where ONNX-Caffe2 backend rewrite ONNX graph to Caffe2 NetDef.
Currently, in our project, we are mostly doing detective work on the ONNX and Caffe2 sides. In this project, we learned and discovered a big revelation on how “production-readiness of PyTorch” after the PyTorch 1.0 release announcement, about 2 months ago. We are trying to demystify that announcement.
Please bear with us. Our response might be delay as we are super busy now:
- to get the iOS part done
- make it easier and more pleasant to ship and test your neural network model in PyTorch on mobile devices
- ironing out all the kinks that slow down the shipping process
19 Likes
Supercovergence refers to a consequence of the one cycle policy: as the policy involves partly training with a high learning rate, a regularisation effect (overfitting prevention) and shorter training time with stable prediction performance are achieved.
More in-depth explanation by @sgugger: https://sgugger.github.io/the-1cycle-policy.html#the-1cycle-policy
and the original paper:
https://arxiv.org/abs/1708.07120
Actually Docker on Windows 10 can run Linux container images as well.
3 Likes
Just made a classifier which can recognize the language from the script.
I have chosen 4 Indian languages. ( Bengali, Gujarati, Hindi and Telugu )
Architecture: ResNet 34
Accuracy: 95 %
The model was trained on electronic scripts. The test images were having newspaper, posters, textbooks etc. i.e. the test images distribution was much larger than training.
Even then the classifier worked with a very good accuracy.
2 Likes