Share your work here ✅

[dineshh]

Hi All, First I thank Jeremy for this awesome course, I did some fun project based on lesson 1, its a vehicle classifier, I took images from Google and feed it to the fastai library to classify different type of vehicle (SUV, Formula1, Hypercar, pickup truck, batmobile, container truck, heavy duty truck, convertible) without change any default setting achieved 90%~ accuracy with resnet50. my next step will be adding more images to each class and add more transfort medium like a bike, bicycle, auto,bus etc., then integrate with live traffic cam to do analysis about the transport medium movement. again thank you so much for this awesome course. with resnet 50

looking for suggestion to add more things in this fun project. Thanks

[derekmeer (Derek)]

Hi all,

Just finished Lesson 1. Here’s my writeup on a fun little bear classifier I built using ImageNet data: https://blog.derekmeer.com/what-kind-of-bear-is-best-building-a-bear-classifier-with-fast-ai

Big takeaway: image URLs from ImageNet aren’t always good. For the black/brown bear images, I found that I could use only about 400 images, or 15% of the data.

Despite this, I’m surprised by how good the results are. I probably need to test on something outside of the ImageNet dataset to be sure, but 2.5% error rate is pretty good!

I also found an ImageNet mis-classification. I have no clue how common that is, or how to report it. How do you normally deal with something like that? Thanks again for the great (free) course!

[dusten]

Color Swatch Dataset

I am excited to refresh my knowledge of deep-learning with this years release of Part 1 and Part 2 (soon to be released).

While we all wait for Part 2 to be released, I went back and rewatched Part 2 from 2018.

In Lesson 11 there is a new idea that was introduced with DeVISE that can find things in the dataset that it may not have learned natively. The quote that kicked off this line of thought was “…I don’t know much about birds but everything else here is BROWN with WHITE spots, but that’s not…”

The comment about the color brown now has me thinking about object detection and can we ask if the model knows something about color, for example, “Red Car.”

Anyways here’s a link to the notebook and one to the dataset.
https://github.com/dusten/Project_Ideas/blob/master/generate_color_squares.ipynb

dusten/Project_Ideas/blob/master/Color.csv

[dmbernaal (Diego Medina-Bernal)]

Hi everyone! Just wanted to share a quick and fun project I put together. Essentially I took everything I have learned from FastAI, found a very interesting White-paper on Arvix.org (link below) and gave it a shot to replicate their work!

Predicting price action movement for currency pairs with ~82% Accuracy

Research Paper:

Amazing work by: Yun-Cheng Tsai, Jun-Hao Chen, Jun-Jie Wang

1801.03018.pdf

Github repository:

dmbernaal/Nova

I have set up the repository with different Jupyter Notebooks for:

• Downloading data using Oanda API (Key has been destroyed)
• Data pre-processing & Applying indicators
• Converting Charts (sliding window approach)
• Using FastAI, DenseNet Architecture

I just finished Part 1 of the new FastAI course so thank you so much @jeremy, @rachel, and to the team for such an AMAZING course. I cannot wait for Part 2 coming in the summer.

Hope this helps others!

PS. I’m learning more about Finance/Quant trading & am very new to Machine Learning so please don’t mind any mistakes in the notebooks. Still learning

[disisbig (Gaurav)]

I’ve created a python package inltk: Natural Language Toolkit for Indian Languages, available for download on pip.

It contains Language Models, Language Classifier and Tokenizers for 10 Indic Languages, namely Sanskrit, Hindi, Punjabi, Gujarati, Nepali, Kannada, Malyalam, Marathi, Bengali, Odia which I had trained using fastai.

Here’s a Demo.

I believe this toolkit will be helpful in developing apps which will reach and impact millions in their local language as we bring next billion users online.

Big Thanks to @jeremy and fastai team, for everything you do!

[javiber (javier berneche)]

Hi everyone!

I’m working on lesson 2 and decided to make a movie poster classifier, was not expecting much since the data from google was really noisy and movies usually have more than one category but I decided to give it a try.

output of learn.fit_one_cycle(8):
|epoch|train_loss|valid_loss|error_rate|time|
|—|---|—|---|—|
|0|2.648347|2.040480|0.718905|00:11|
|1|2.306737|1.947001|0.685323|00:11|
|2|2.088367|1.916026|0.684080|00:11|
|3|1.922450|1.866856|0.657960|00:11|
|4|1.790923|1.856265|0.669154|00:11|
|5|1.686638|1.840987|0.670398|00:11|
|6|1.597364|1.831289|0.665423|00:11|
|7|1.541694|1.831808|0.670398|00:11|

That seems like a disaster but checking the confusion matrix is a little bit more encouraging

It learned something and the most confused categories make a lot of sense.

What I found weird is that my learning rate plot after unfreezing just goes up

Anyone know what that means?

[gai (Sven)]

this is a pet project I am involved in, not sure if this is the right place to post but it’s loosely inspired by me learning fastai, so I thought it might be interesting:

We felt it’s important to keep up to date with recent discussions in machine learning across the net, so I helped writing a site that collects this kind of content: hype.machlearning.net

It can do some interesting queries, e.g. changes to SotA in the last month, sorted by “top”, meaning: first places first:

https://hype.machlearning.net/?so=top&t=1m&SotA=

It also knows which arXiv papers have been written by which group, so you can e.g. see all papers discussed which were written by Google in the last 3months, ordered by date:

https://hype.machlearning.net/?so=new&t=3m&a=Google

It uses a sentiment model to decide which twitter messages are related to machine learning and also tries to find the most significant phrase in a conclusion of an arXiv paper (Could it change the SotA? What are problems with this approach?) and displays it next to the paper’s title

[anubhavmaity]

I have worked on news categorization of AG News dataset using the Fast.ai library. Got an accuracy of 93%. You can check out the GitHub repo here

anubhavmaity/Ag-News-Category-Classifier/blob/master/README.md

[champs.jaideep (jaideep v)]

Try with Arcface loss.
I have implemented it in following kernel which I have created for identification of whale species using tale. This has hoping 5004 classes. i stand with Score of ~~93 on test set
https://www.kaggle.com/jaideepvalani/arcface-humpback-customhead-fastai-score919/

[prosti]

From your diagram it looks like you cannot gain super-convergence. Just run learn.lr_find() again, and let us know if the results are different. It seems to me that this plot always provides different results if I repeat:

learn.lr_find()
learn.recorder.plot()

You can always use the learn.fit_one_cycle(1, max_lr=slice(3e-5,3e-4)) which should be the safe default.
I am not sure why the plot is always different. Maybe I am missing something.

[sidthekid (Sid)]

Hey, interesting project and my mother had the same problem!

[hwasiti (Haider Alwasiti)]

A tip for everybody who has a small dataset…

@kodzaks
and
@mididou

If you have small dataset maybe you can try the powerful mixup technique that is already integrated in fastai. I had EEG data and experimented with very limited data, and got from 75% to 83% accuracy boost by adding .mixup() after creating the the learner like in this example:

from the fastai docs:

Mixup data augmentation

What is Mixup?

This module contains the implementation of a data augmentation technique called Mixup. It is extremely efficient at regularizing models in computer vision (we used it to get our time to train CIFAR10 to 94% on one GPU to 6 minutes).

As the name kind of suggests, the authors of the mixup article propose to train the model on a mix of the pictures of the training set. Let’s say we’re on CIFAR10 for instance, then instead of feeding the model the raw images, we take two (which could be in the same class or not) and do a linear combination of them: in terms of tensor it’s

new_image = t * image1 + (1-t) * image2

where t is a float between 0 and 1. Then the target we assign to that image is the same combination of the original targets:

new_target = t * target1 + (1-t) * target2

assuming your targets are one-hot encoded (which isn’t the case in pytorch usually). And that’s as simple as this.

Dog or cat? The right answer here is 70% dog and 30% cat!

As the picture above shows, it’s a bit hard for a human eye to comprehend the pictures obtained (although we do see the shapes of a dog and a cat) but somehow, it makes a lot of sense to the model which trains more efficiently. The final loss (training or validation) will be higher than when training without mixup even if the accuracy is far better, which means that a model trained like this will make predictions that are a bit less confident.

Example Training

model = simple_cnn((3,16,16,2))
learner = Learner(data, model, metrics=[accuracy]).mixup()
learner.fit(8)

================================
This powerful technique needs more visibility… I hope Jeremy will be kind to mention this mixup augmentation in one of the awesome lectures that we are enjoying in the part2 v3 course…

================================

@kodzaks
How many images you have in your dataset? and how many classes? How is your best accuracy so far? If mixup worked for you, I would love to know…

I love any project related to sounds and waves… FFT and understanding how is the speech and different musical sounds created by mixing only pure sine waves with different freq, phase and amplitude was intriguing me since I was a kid… This was something that I was dying to know and nobody could help (pre-internet era)… After several years when I got into, college I could understand it, and implemented FFT on my old MSX2 computer in BASIC and did some FIR filtering on the waves… That was truly a joy for me that I still remember vividly… Now kids are lucky that anything they want to know, it is only few clicks away…

[hwasiti (Haider Alwasiti)]

@butchland

Note: all the training was done using Mixed Precision ( learn.to_fp16() )
except when running an interpretation, in which the model was converted to use 32bit fp ( learn.to_fp32() ) in order to run classification interpretation steps – for some reason, fastai barfs when classification interpretation is run on a mixed precision model.

That’s a clever workaround… I had the same issue when interpretation could not run on fp16 models.

Thanks for sharing!

[hwasiti (Haider Alwasiti)]

@butchland

Another notable finding was the GPU memory usage during training:

• Using a batch size of 256 and image size of 112x112, Mixed Precision used up approx. 3-4 GB GPU Memory.
• Using a batch size of 128 and image size of 224x224, Mixed Precision used up approx 3-4 GB as well.
• Lastly, using a batch size of 128 and image size of 299x299, Mixed Precision used around 11GB GPU Memory (triggering warnings in Colab, but otherwise ran to completion)

That is weird… All the memory sizes are as expected, except the 224x224…

Perhaps you have looked at it when your model was freezed… After unfreezing usually it will take more gpu ram… you can test it by doubling BS with 224 size and you will get cuda OOM error.

[offirinbar (offir)]

NBA MVP predictions

Hi everybody!

I am trying to do my first project: precidt the NBA MVP winners. I made a training and testing csv files (based on the last 30 years).

have few problems:

1. Each year only 1 players wins the mvp. how do I add this condition to my program? (ofcourse that every year 1 player must win the title). (1 for winning the title, 0 else)

2. I splited my train and test data in this pattern:
   2018 data - test
   2017 data - train
   2016 data - train
   2015 data - train
   2014 data - test
   so on…
   is it the right way?

3)What is the right way to split between the train and validation sets?

4)What is the right way to define the cont and cat variables?

Your Help we be great!

Here is 2016 training data (Westbrook won the MVP)

Part of my code:

[hwasiti (Haider Alwasiti)]

This is an extremely exciting project… I have read your blog post when @radek tweeted about it with a lot of enthusiasm…

I read the amazing blog post that has inspired you to do your project. Thanks for sharing…

This is something that made my heart palpitating…
I think the most exciting part is that the dataset is human comprehensible, so we can understand the math operations on the embeddings by just looking on the resulted image. People are lucky who work on datasets that both us and machines share the same insight and both understand the same way. Such visualization are not only useful for debugging, but excites you that the machine not only learned to recognize images, but can understand what are the differences between them…

And I could not resist but trying vec2whale operations (like whale - whale = ? ; whale + whale = ?). That was after finishing the kaggle whale competition with a silver medal (which I haven’t think that is even a possibility for my 1st serious kaggle comp)…

Here are few images of those whale2vec… Note that the whale identification features are way more subtle than roads, buildings…etc… The model has been trained to identify whales from small scratches and colors. So its embedding math operations should be understood in this context.

WHALE2VEC operations:

Minus:

(X) whale - (Y) whale = (Z )whale

Colored - colored = Black and white

The following minus operation is interesting:

Mostly whitish - middle whitish and blackish on edges = middle blackish and whitish on edges (just what you are expecting from minus operation)

Plus:

(X) whale + (Y) whale = (Z )whale

Middle white (with streaks) + edge white = ALL white (with streaks)

[lesscomfortable (Francisco Ingham)]

Hey, I published my own review on the Kaiming paper, without the mathematical derivation but with all the important intuitive concepts. I hope it serves as a complement of @PierreO’s post. You can find it here. Feedback is welcome appreciated!

[SebastianP]

Hi, I recently tried to build a joke generator using the NLP stuff taught in course 4.
I used the jokes in wocka.json and stupidstuff.json from this repo.

It didn’t really work that great but some of the generated “jokes”:

Yo mamma so old, she is still in the shower!
There was a blonde who was working on a computer. She was a BLONDE PROGRAMMER.
What do you call a man with a dog in his mouth? - An Irishman.

I know reaaaally good ones!

[ajthinking (Anders Jürisoo)]

Me to, seems it is always different for no apparent reason. I cant understand why. Could it be because of me forgetting to seed?

numpy.random.seed(42)

[javiber (javier berneche)]

Hey @prosti thanks for your reply, it turns out I was doing all kinds of mistakes in my code.

I ended up giving up with this dataset from google images and started to work on another one, will post my results shortly