Share your work here ✅

ademyanchuk · November 1, 2018, 2:08am

Hello guys)

I want to share my work as an example of Jeremy’s words on how we can achieve great results without a ton of data and computations. I used a pneumonia dataset (http://www.cell.com/cell/fulltext/S0092-8674(18)30154-5) and compare fastai v1 results with original paper.
To not bother you with much citation, authors achieved accuracy of 92.8%. But I should mention, that they train Inception v3 network pretrained on the ImageNet for 100 epochs with a batch size = 1000 images, which is really huge.

With a fastai library and only 5000 x-ray images we can achieve 90% accuracy on pretrained resnet34 and no more than 6 learning cycles. Share the link of the notebook (https://github.com/ademyanchuk/course-v3/blob/master/nbs/dl1/lesson-1-pneumonia-dai.ipynb)

The key takeaway here is we can get really great results from deep learning with not so much data and computational power. Thanks to fast.ai and great community for sharing knowledge))

jeremy · November 1, 2018, 2:24am

If your new dataset is similar to imagenet, than fine tuning the earlier layers won’t necessarily help at all.

drscotthawley · November 1, 2018, 2:45am

I had previously built an audio spectrogram classifier using Keras, with a fairly simple CNN model I wrote in by hand, and Adam optimization.

So I wanted to compare my “old ways” to the “new ways” which Jeremy made use of in Lesson 1, e.g. the ResNets, 1cycle scheduling, and fine-tuning/freezing/unfreezing. Also I wanted to become familiar with using fastai on Google Colab.

The following is a comment-able link to my Colab notebook, : https://colab.research.google.com/drive/16BiLegPGx5G911B15wAUBSKjFZRcyhxR

This was for the audio spectrogram dataset I created from a friend’s sample library, which looks like:

The results are:

The “new ways” of Lesson 1 yield about 92% accuracy and my old ways produce about 91% accuracy.
Unfortunately the actual demonstration of my Keras model within the same notebook fails, apparently because of conflicting versions of CUDA between the new PyTorch and the CUDA which Keras/Tensorflow expects to see. So you just have to take my word for it right now, until I can get around that.

The lack of huge improvement in the score could be because

the particular audio dataset I’m using this time is fairly ‘noisy’, or
my old ‘simple’ model actually does tend to work pretty well (as I’ve used it on other datasets), and converges pretty fast.

It could be that the ResNet+1cycle method reaches the same accuracy in fewer epochs, although a comparison of wall time would also be worthwhile. Work in progress! I may just have to create a separate notebook to demonstrate the Keras model. (Alternatively, I could rewrite the Keras CNN itself in PyTorch…someday.)

ArchieIndian · November 1, 2018, 2:53am

On the Bangla Lekha-Isolated dataset for Bengali character recognition (84 class classification), I able to get the error rates ~5.3%. This is a 84 class classification. The error rates can still improve if I keep training a few more epochs. I have used the dataset available at https://data.mendeley.com/datasets/hf6sf8zrkc/2

I am not too sure as to what the state of the art accuracy is but this looks promising.
You can have a look and use the trained model from https://github.com/ArchieIndian/fast-ai-experiment

VarunT · November 1, 2018, 3:59am

My first experiment was to see how well model would classify some similar looking Indian dance forms:

First version of results:

After lesson 2 I realized that size of dataset needs to be increased. Also, to check how the classifier is performing I added a 4th class, which is expected to be very easy to differentiate for humans (this dance form specially uses face paint).
This time I found the results to be much better:

After seeing there is confusion between two forms specifically, googling lead me to a quora post which discusses the difference between two very similar dressing styles:
https://www.quora.com/What-are-the-differences-between-Bharatnatyam-Kuchipudi-dance-forms-How-can-I-tell-the-difference-between-the-various-South-Indian-classical-dance-forms-by-just-looking-at-the-costumes-the-performer-wears
So, it is possible to confuse between these dance forms just looking at the attire.

It is exciting to see how good classification turned out to be for just 200 images for each dance form!

Kaspar · November 1, 2018, 4:56am

Hi have not done the segmentation yet but will use the caravane notebook for that.

Kaspar · November 1, 2018, 4:59am

super what kind of medical data have you chosen

Kaspar · November 1, 2018, 5:00am

what a fun project

ArchieIndian · November 1, 2018, 7:22am

Will be interesting to see what part of the images got activated for the various dance forms. If it is eyes for one dance form and hand shapes for other dance forms, it will all be such wow

deepanshu2017 · November 1, 2018, 7:23am

I want to do the same for my classifier. Is there any link where you can refer me to check the same?

Thanks

ArchieIndian · November 1, 2018, 9:34am

I think Jeremy will come to that part in a later lecture. I don’t exactly remember but this was a part of last years lecture 7. Go to this notebook https://github.com/fastai/fastai/blob/master/courses/dl1/lesson7-CAM.ipynb and have a look. It will help you highlight the places which activated a certain class

ArchieIndian · November 1, 2018, 9:49am

Go to lecture 7 of last years MOOC. Go to this notebook https://github.com/fastai/fastai/blob/master/courses/dl1/lesson7-CAM.ipynb and have a look. It will help you highlight the places which activated a certain class

oguiza · November 1, 2018, 11:35am

Thanks for sharing @ademyanchuk! It’s certainly amazing how powerful DL is, even when applied to a dataset different to images in ImageNet.
Is this a publically available dataset? I’d like to experiment with it a bit if it is.

ademyanchuk · November 1, 2018, 12:00pm

Yes, it’s public.

This is link on kaggle (https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia/home)

And this is link from paper (https://data.mendeley.com/datasets/rscbjbr9sj/2)

Would be glad to hear about your results)

oguiza · November 1, 2018, 12:06pm

Great! Thanks a lot!
I’ll certainly share my findings.

bachir · November 1, 2018, 12:11pm

Was inspired by the lesson 2 image download, I re-adapted the code to use it to download high quality images from EyeEm (which is used by pro photographers).
I got a nice curve for the learning rate
eyeem_learning_rate
And some good results when looking at the confusion matrix:
eyeem_confusion_matrix
Here is a blog post summarizing the steps for creating a dataset of images using this service - link.
Here is the complete jupyter notebook - link.

bachir · November 1, 2018, 6:04pm

Tried to build a web app for serving an emotion classifier using same service used to win science hack day mentioned in lesson 2. The classifier didn’t had great results, but nevertheless was able to see that Samuel Jackson is angry:

> train_ds._get_x(10)

> learner.predict(train_ds._get_x(10))
('Angry',
 tensor(0),
 tensor([7.7620, 0.6239, 2.4098, 0.4851, 1.5716, 0.1141, 0.7207]))

Then hit this error on deployment of the app;

> Upload [====================] 100% 0.0s (83.34MB) [3 files]> Error! Upload failed
> Error! An unexpected error occurred!
  Error: File size limit exceeded (5 MB) (400)
    at responseError (/snapshot/repo/dist/now.js:2700:15)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7) Error: File size limit exceeded (5 MB) (400)
    at responseError (/snapshot/repo/dist/now.js:2700:15)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

It seems need to upgrade paid. I think Bardem next movie should be no country for poor men
Any way, I think the world can wait few more days until I find a free service

Jokes a side:

here is dataset from kaggle (it was mentioned in another thread here),
here is the jupyter notebook
and here is sample app code

jeremy · November 1, 2018, 6:29pm

FYI in python stuff starting with ‘_’ is generally meant for internal use. You should simply say train_ds[10].

ramon · November 1, 2018, 8:11pm

Hi fellow practitioners,

No web-app but sharing it anyway

I automatically downloaded 99 images for three kinds op medical packaging. Paracetamol 125MG, 500MG and 1000MG. Resulting pictures are of good quality, below some examples:

(For downloading I used google image download:)

PATH = "/home/jupyter/med_packages/train"
keywords = "Paracetamol 500MG,Paracetamol 125MG,Paracetamol 1000MG"
response = google_images_download.googleimagesdownload()
arguments = {"keywords":keywords,
         "limit":99,
         "print_urls":False,
         "output_directory":f'{PATH}',
         "chromedriver":"/usr/local/bin/chromedriver"}
paths = response.download(arguments)

I expected it to learn quite fast because imagenet is capable of segmenting/filtering text inside an image. So I would expect it learned to associate the numbers in the image with the corresponding class.

But even training with Resnet50 it is wrong on 1 out of 4 images, while the images are correctly classified, eg:

Probably the model sees other patterns. Any suggestions for debugging or improving? Of course more data would improve the model.
Can I somehow/force hint the model it should look for text? I’m thinking of generating some data myself with only text in different fonts/colours/sizes/angles.

Thanks for thinking along!

Blanche · November 1, 2018, 8:50pm

I’ve made story about classifying fruits dataset with fastai. I am aware that the code display method should be consistent (some is written in markdown, some is screenshotted), but I got lazy tbh.