Share your work here ✅

[hwasiti (Haider Alwasiti)]

Note: all the training was done using Mixed Precision ( learn.to_fp16() )
except when running an interpretation, in which the model was converted to use 32bit fp ( learn.to_fp32() ) in order to run classification interpretation steps – for some reason, fastai barfs when classification interpretation is run on a mixed precision model.

That’s a clever workaround… I had the same issue when interpretation could not run on fp16 models.

Thanks for sharing!

[hwasiti (Haider Alwasiti)]

Another notable finding was the GPU memory usage during training:

• Using a batch size of 256 and image size of 112x112, Mixed Precision used up approx. 3-4 GB GPU Memory.
• Using a batch size of 128 and image size of 224x224, Mixed Precision used up approx 3-4 GB as well.
• Lastly, using a batch size of 128 and image size of 299x299, Mixed Precision used around 11GB GPU Memory (triggering warnings in Colab, but otherwise ran to completion)

That is weird… All the memory sizes are as expected, except the 224x224…

Perhaps you have looked at it when your model was freezed… After unfreezing usually it will take more gpu ram… you can test it by doubling BS with 224 size and you will get cuda OOM error.

[offirinbar (offir)]

NBA MVP predictions

Hi everybody!

I am trying to do my first project: precidt the NBA MVP winners. I made a training and testing csv files (based on the last 30 years).

have few problems:

1. Each year only 1 players wins the mvp. how do I add this condition to my program? (ofcourse that every year 1 player must win the title). (1 for winning the title, 0 else)

2. I splited my train and test data in this pattern:
   2018 data - test
   2017 data - train
   2016 data - train
   2015 data - train
   2014 data - test
   so on…
   is it the right way?

3)What is the right way to split between the train and validation sets?

4)What is the right way to define the cont and cat variables?

Your Help we be great!

Here is 2016 training data (Westbrook won the MVP)

Part of my code:

[hwasiti (Haider Alwasiti)]

This is an extremely exciting project… I have read your blog post when @radek tweeted about it with a lot of enthusiasm…

I read the amazing blog post that has inspired you to do your project. Thanks for sharing…

This is something that made my heart palpitating…
I think the most exciting part is that the dataset is human comprehensible, so we can understand the math operations on the embeddings by just looking on the resulted image. People are lucky who work on datasets that both us and machines share the same insight and both understand the same way. Such visualization are not only useful for debugging, but excites you that the machine not only learned to recognize images, but can understand what are the differences between them…

And I could not resist but trying vec2whale operations (like whale - whale = ? ; whale + whale = ?). That was after finishing the kaggle whale competition with a silver medal (which I haven’t think that is even a possibility for my 1st serious kaggle comp)…

Here are few images of those whale2vec… Note that the whale identification features are way more subtle than roads, buildings…etc… The model has been trained to identify whales from small scratches and colors. So its embedding math operations should be understood in this context.

WHALE2VEC operations:

Minus:

(X) whale - (Y) whale = (Z )whale

Colored - colored = Black and white

The following minus operation is interesting:

Mostly whitish - middle whitish and blackish on edges = middle blackish and whitish on edges (just what you are expecting from minus operation)

Plus:

(X) whale + (Y) whale = (Z )whale

Middle white (with streaks) + edge white = ALL white (with streaks)

[lesscomfortable (Francisco Ingham)]

Hey, I published my own review on the Kaiming paper, without the mathematical derivation but with all the important intuitive concepts. I hope it serves as a complement of @PierreO’s post. You can find it here. Feedback is welcome appreciated!

[SebastianP]

Hi, I recently tried to build a joke generator using the NLP stuff taught in course 4.
I used the jokes in wocka.json and stupidstuff.json from this repo.

It didn’t really work that great but some of the generated “jokes”:

Yo mamma so old, she is still in the shower!
There was a blonde who was working on a computer. She was a BLONDE PROGRAMMER.
What do you call a man with a dog in his mouth? - An Irishman.

I know reaaaally good ones!

[ajthinking (Anders Jürisoo)]

Me to, seems it is always different for no apparent reason. I cant understand why. Could it be because of me forgetting to seed?

numpy.random.seed(42)

[javiber (javier berneche)]

Hey @prosti thanks for your reply, it turns out I was doing all kinds of mistakes in my code.

I ended up giving up with this dataset from google images and started to work on another one, will post my results shortly

[lesscomfortable (Francisco Ingham)]

Hi, I just reviewed the Transformer-XL paper and architecture which is implemented in fastai. The improvement over Transformer is quite interesting and makes a lot of sense. You can find it here. Let me know what you think!

[javiber (javier berneche)]

Hi @lesscomfortable I’m not familiar with Transformer architecture, will definitely take a look but I’m not doing text classification.
I’m trying to train a classifier that takes the poster of a movie and tries to predict the genre

[lesscomfortable (Francisco Ingham)]

Hey! That’s a computer vision problem, my post is on NLP.

[mididou (Zobiri Mehdi)]

Thank U @hwasiti , great tip , definitely need to delve more into the Documentation.

[baz (Harry Coultas Blum)]

I managed to get 98% accuracy classifying speakers from a small dataset! More details:

Deep Learning with Audio Thread

I noticed that the transform to spectogram wasn’t expanding the channel dimension to 3 as is done by the library for the mnist dataset. I’ve added that into my fork of the project. You were replacing the first Conv2d layer which would loose all pretrainined learning? I may be wrong about this. After making these tweaks I checked to see how well this approach performed on Free ST American English Corpus datset (10 classes of male and female speakers) I was able to get these results: [Screenshot…

[LessW2020 (Less)]

Hi all - happy to share my results in using FastAI and Resnet152 and a lot of differential learning rate cycles on a cellular histo-pathology dataset - up to 100% accuracy!

Up to 100% accuracy for cellular histo-pathology image classification… with...

I saw two papers on this dataset and noticed that in both cases, the CNN’s they were using were pretty bland and in addition, very standard practice of a simple fixed learning rate, etc. One was from summer 2018, so while relatively recent, I could see the techniques we are learning here are way ahead of the curve.
I thus took it up as a challenge to apply FastAI to it. Interestingly, I started with Resnet50 and while it got to 98% and was very stable there, it became stuck on two very similar classes and could not move beyond it. I ultimately had to restart with ResNet152.
That still took a lot of cycles with the learning rate finder and differential learning rates, and a very steady train/check learning rate/ retrain process, but I did manage to tune it to repeated 100% results and thus outdo both of the papers in accuracy by a reasonable margin.
(91-95% was their best, and in one case they oddly only tested subsets of 4 classes to get that averaged 91%, not all 20 at once).

[nialloh23 (Niall O'Hara)]

Hi everyone,

I’m glad to share my first computer vision project (Irish Gaelic Footballer Classifier!)

After completing lecture 1 of FastAI I decided to build a data set of Irish Gaelic football team players.
The goal of my computer vision classification model was to distinguish which players played for which team. Out of interest I built and trained my model using both Keras and FastAI.

*** FastAI Classification Model:** (Best Result: 92% Validation Accuracy) Link to Collab File

*** Keras Classification Model:** (Best Result: 91% Validation Accuracy) [Link to Colab File]

Some practical things which improved my model:

1. Data augmentation seems to work very well for small data sets (+8% Acc Improvement)
2. Fine tunining on learning rate seems to work well (this feels like a very important hyperparameter)
3. Tracking the Train & Val error allowed me to diagnose under fitting issues. Adding more capacity to my custom CNN network in Keras gave me some big improvements (+12% Acc Improvement).
4. While it was easier for debugging & experiments with lower epochs I found for my final models increasing the number of epochs gave me better results. I could see this trend from the training curves (+4% Acc Improvement)
5. I completed a visual error analysis of the misslassified images with the biggest loss. I removed images from the validation set which were clearly misllabelled. (+12% Acc Improvement)

Some other takeaways from the project:

1. I was suprised by how similar my results were between Keras and FastAI. I used a custom CNN network architecture using Keras while I used Resnet 50 architecture using FastAI (I didn’t do a like for like comparison).
2. Building custom image datasets isn’t as hard as I thought it would be and it’s far more rewarding than using out of the box datasets
3. Google Collab seems super useful for these quick hacky projects

[votamvan]

Hi @tank13,

Did you finish running the file dl2/imdb.ipynb. can you share me the model output such as lm_last_ft, lm1, lm1_enc, clas_0, clas_1, clas_2 …
It took me a lot of days to run all of the fit() commands. Please help me if you have these model files, thanks.

[javiber (javier berneche)]

Hi everyone,

I tried to create a milti-label classifier that takes a movie poster as its input and predicts the different genres of that movie.

I thought it was an interesting experiment regardless of the results and definitely learned a lot while doing it.

You can find my final version on this repo. Any tips or recommendation is welcome.

My final result is ~0.59 f2 score but I’m not sure how to evaluate that, I didn’t find other classifiers to compare my results so if anyone knows about other solutions I’d love to see them.

I found that ClassificationInterpretation was lacking some functionality for multi-label classifiers, I created some functions manually but if anyone knows about how to interpret results in multi-label problems better I’d appreciate it.

[marcello_m (Marcello Morchio)]

Great work @oguiza I am reading all the thread on the TS study group and I am going to try our your examples on my dataset. One concern: the paper looks to be a broken link:

https://aaai.org/ocs/index.php/WS/AAAIW15/paper/viewFile/10179/10251

gives

DB Error: Table ‘./aaaior5_ocs/sessions’ is marked as crashed and should be repaired
It’s only me?
Any other link to the paper?

[sheepish]

I have trained a classifier with fine tuned embedding language model which assigns content labels to basic restaurant descriptions. So you have a text like:

The three star coffee shop, The Eagle, gives families a mid-priced dining experience featuring a variety of wines and cheeses. Find The Eagle near Burger King.

And the output are the following labels:

eatType[coffee shop],
food[French],
priceRange[moderate],
customerRating[3/5],
kidsFriendly[yes]

This is the dataset used for the e2e Natural Language Challenge which consists of 50k <text,content labels> pairs. It achieves an F-score of 92% thanks to the gradual unfreezing of the layers.

You can find out more in my kaggle kernel. Comments are welcome.

[oguiza (Ignacio Oguiza)]

Thanks @marcello_m! Could you please point me to the right reply where this link appear. I don’t have any context and don’t know which is the linked paper.