Share your work here ✅

maxim.pechyonkin · July 18, 2019, 8:18pm

Yes, I am passing data.train_ds to .get_preds(). The problem is that the callbacks are not working as I would expect with .get_preds() as opposed to working well with .fit_one_cycle(). For now what I am doing is setting learning rate to zero and using .fit_one_cycle() to record activations without training the network.

abhikjha · July 19, 2019, 4:01am

Hello everyone

I recently wrote a medium article on the integration of Fastai with BERT (huggingface’s pretrained pytorch models for NLP) on a multi-label text classification task. After that I compared the performances of BERT and ULMFiT.

Here are few things which I did to integrate Fastai with BERT:

Using BERT’s tokens and vocab
Some modifications in BERT’s tokens for eos and bos parts
Splitting the model for discriminative learning techniques

Here is the link:

I was amazed by the level of accuracy using just 2 epochs:

BERT - 98% accuracy
ULMFiT - 97% accuracy

I would be glad if you have any feedbacks or comments on this.

BR
Abhik

Shubhajit · July 19, 2019, 5:34am

Copy of the post from Share your work here (Part 2) :

Hi there!
Check out my recent blog post explaining the details of One-Cycle-Policy (fastai’s famous method .fit_one_cycle() ): The 1-Cycle-Policy: an experiment that vanished the struggle in training of Neural Nets.
Efforts have been made to make the entire things as simple as possible, especially explaining the codes. Hope you will enjoy it

morgan · July 19, 2019, 10:15am

very cool! well done! A great test would be to see how it works with hurling

AjayStark · July 21, 2019, 1:27pm

Hi everyone, from lesson-2 concepts, i created an emotion classifier, which jeremy talks about in the video.
Just by changing the wight decay, i was able to get around 66% accuracy, and the top values as jeremy says in the video are around 65.7%.
As i am using google colab i am not able to clean the dataset using the data cleaner,3rd party app, but still was able to get a pretty good accuracy.

Is there any other ways to clean the dataset which is supported by google colab?

Thanks,
Ajay

jeffslofish · July 21, 2019, 10:35pm

What Programming Language Is It?

Programming Language Classifier

After watching lessons 1-4, I decided to make a web app that classifies text according to the programming language. I searched the web, but couldn’t find much research on this topic. There was one example (https://arxiv.org/pdf/1809.07945.pdf) that uses a Multinomial Naive Bayes (MNB) classifier to achieve 75% accuracy, which is higher than that with Programming Languages Identification (PLI–a proprietary online classifier of snippets) whose accuracy is only 55.5%.

I was able to reach about 81% accuracy(according to fastai) (although I’m not sure I’m measuring it the same way as the paper) after following the same basic steps as the IMDB example. This was done with the dataset I found from the author of the paper, here: https://github.com/Kamel773/SourceCodeClassification. I noticed that the dataset is pretty messy and a lot of the css/html/javascript is misclassified as another one in that group. That is apparent in the confusion matrix:

But, regardless, I made a web app, which is currently here: https://programming-language-classifier.onrender.com. It lets you paste in a snippet of code and it will tell you what language it thinks it is. Give it a try and see if you can confuse it! I created the web app based on the template here: https://github.com/render-examples/fastai-v3.

I did find another data set of classified code on Kaggle, which is way bigger (called “lots of code”) (and hopefully has less misclassifications) that I am going to try to use to improve the accuracy even further.

I am having way too much fun with this course. Thanks everyone!

Shavages · July 22, 2019, 11:09am

Please highlight your specific reply so i can go back and follow that thread. Thank you

michaeltlu · July 22, 2019, 8:28pm

Excited to share work predicting 12-year mortality from chest x-rays. Deep learning can extract prognostic information about health and longevity embedded in routine medical imaging. Made using fastai.

JAMA Open manuscript
Editorial podcast

abhikjha · July 23, 2019, 7:32am

Nice project! Can you please share your notebook / kaggle kernels / github repository for this work?

JoshVarty · July 23, 2019, 4:20pm

@NathanHub and I recently participated in the Freesound Audio Tagging 2019 Kaggle Competition and received our first bronze medal (95/880).

We used FastAI and XResNet in our model. I’ve written up a more complete description here: https://joshvarty.com/2019/07/23/freesound-audio-tagging-2019/

Our repository with notebooks: https://github.com/JoshVarty/AudioTagging

zlapp · July 25, 2019, 6:46pm

Fastai Active Learning

tl;dr First attempt at active learning for fastai library. Includes random, softmax, and Monte Carlo Dropout std calculations to measure the uncertainty of an example for a machine learning model.

I recently came across a paper called BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning and the following blog which was a segway to explore active learning.

I thought it would be cool to have active learning as a part of the fastai library. I am not an expert on active learning but thought it would be a great way to learn about the field and different algorithms. P2 of the fastai lessons has been very helpfuf in implementing the gist.

The implementation separates the uncertainty measurement from the active learning selection process. There are two options for selection:

Select x most uncertain examples from the entire dataset
Select x most uncertain across all batches in dataset.

I hope to continue working on the gist and fully implement different papers. If anyone wants to contribute or finds bugs feel free to pm me.

Shout out to @mrdbourke for implementing Monte Carlo Dropout in fastai.

gist.github.com

https://gist.github.com/zlapp/e82902577aade2d296e52e0c26bd5ac0

fastai-active-learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Fastai Active Learning.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,

This file has been truncated. show original

daveluo · July 25, 2019, 6:56pm

Hi everyone,

I’m happy to present v1 of a comprehensive intro tutorial to geospatial deep learning (focused on building segmentation from drone imagery in Zanzibar) using fastai v1, the latest cloud-native geodata processing tools, and running fully self-contained on Google Colab for ease of learning (and free GPUs!):

Announcement:

Conceptual overview:

Colab notebook (previewed in nbviewer):
https://nbviewer.jupyter.org/github/daveluo/zanzibar-aerial-mapping/blob/master/geo_fastai_tutorial01_public_v1.ipynb

More info & highlights over on the geospatial deep learning thread: Geospatial Deep Learning resources & study group

Given there’s a lot covered here, I’m sure that I missed many things (bugs, mis-assumed knowledge, janky code, bad links). I appreciate any and all of your feedback to make the next versions of this tutorial (and next ones) even better so thank you in advance!

Dave

poseidon · July 26, 2019, 1:31pm

I watched the Part 1 video last year, with fastai v0.7, and just i was amazed to see how much better fastai performs in comparison to other deep learning libraries. I then wondered how would it perform against itself, and needless to say, the library did not let me down. I found a paper written by one of my college senior in early 2019, using a thermal image dataset. At the time, they got a best case accuracy of 97.08% and a validation loss of 11% using resnet101 and fastai v0.7.1, which was achieved after multiple parameter modifications and model tuning.

In July, 2019, I present, fastai v1.0, resnet50 and 10 minutes of coding:
Model Accuracy: 99.38%
Training Loss: 1.4%
Validation Loss: 1.7%

2019-07-26%2018_56_21-thermal_potholes_classification%20-%20Colaboratory

Original Paper: https://www.sciencedirect.com/science/article/pii/S1319157818312837

My respect to fastai - destroyer of scientific papers (RIP) since 2017

Cheers,

blueharen · July 26, 2019, 2:26pm

Wonderful project!

What did you use to develop/deploy your webapp? I made something very similar with Flask and was just curious as to what you used.

Cheers!

abhikjha · July 27, 2019, 5:21pm

Hello everyone

I recently wrote a medium article on building an Image Similarity search model using Fastai, Pytorch Hooks and Spotify’s Annoy.

Results of this project was simply outstanding and I was blown away by how easily we can implement this.

Here is one base image for which we need to find similar images:

1_0D1OufSUAnXXhiget-VjmA

and the model returned following images which it thought are similar to the base image:

Please read through the article wherein I posted the link for Kaggle Kernel as well.

BR
Abhik

steveoni · July 27, 2019, 10:43pm

After taking the first two class of part 2 of 2019, I was able to get how autograd works and how it was used in pytorch. And to be sure I grasp the concept, I created a simple autograd in javascript and also create a pytorch like implementation in javascript.
It was implemented in obsevablehq js interactive notebookhere
and also you can help me check this medium post draft explaining the basic concept here

Thanks, fastai and the whole family

jeffslofish · July 28, 2019, 3:16am

Thanks! It’s a Starlette app that I deployed to Render.

kampamocha · July 29, 2019, 5:08pm

I trained a resnet50 to classify emotions on faces with the CK+ dataset (6% error).

cican · July 30, 2019, 7:46pm

Hi everyone, after complete the lesson 1 I have create my own dataset to predict the Felidae. You could find the blog from my personal website.

paulsolomon · August 2, 2019, 2:41pm

For my last Part 1 mini-project I decided to have a go using the tabular learner to predict Fantasy Premier League player scores. I’ve written a blog about it! https://medium.com/@sol.paul/how-to-win-at-fantasy-premier-league-using-data-part-1-forecasting-with-deep-learning.

Spending some time applying the approach really helped me understand the concepts more deeply, although I’m still not sure I fully understand how the time series aspect works with regards to trend. I noticed that in the Rossmann notebook there are no ‘recent trend’ type variables i.e. something that describes the days and weeks leading up to each observation. Of course, this information is still there in the other observations, so I’m guessing that the ‘date’ (day, month, year) embeddings encompass this in the network e.g. October’s embedding encodes in some way that September is ‘nearby’, meaning that the model can account for recent trend (assuming it is predictive).

Loved the first part of the course so going onto part 2 now!