Share your work here ✅

Thanks @jeremy ! Already wrapping up part two! Thanks to your course, I finally stopped making excuses and writing stuff down. Now it seems to get addicting.

if you find a way to save your model on Kaggle then please tag me in the response as well. I’ve been suffering from the same issue as well and it’s pretty time consuming

Hi Suresh, did you try training with different sizes? (training with 128 and then using those weights for 256). I haven’t tried much hyperparameter tuning since I did not get the time (I am working as well.) Also, I haven’t made any notes as such, I refer to the official course notes only. We can stay in touch and keep each other informed about interesting projects we work on or hackathons we participate in. Cheers.

Hello Alex, a 99% accuracy rate on a data set that small is a classic sign of overfitting. You may want to explore dropout layers within the network along with increasing volume of content to increase diversification of course. I’m super interested in your work!

1 Like

Impressive work! And congrats on being featured by Jeremy himself!

Please see my response here (regarding your previous comment on saving models in Kaggle Kernels):

3 Likes

Please see my response here:


Hope this helps!

My apologies for forgetting this thread, I should have posted about my work here in the first place!

In order to avoid spamming the thread, I will post the link again to my original description.

In a sentence, I’m currently using Deep Learning models (mainly image-to-image translation) to predict, in real time, urban environmnetal performance. I’m generating my own synthetic and real life data to train the models and I have already pretrained models that can predict certain aspects of performance for 3 locations in the US. Results are quite encouraging (you can see some example predictions below, left image is input, middle is simulated, right is predicted) and the next step is to move to actually complex studies like thermal comfort.

I’m only partially using fast.ai for this, the cycleGAN implementation, and I’m eagerly waiting for more generative models to be introduced by the community (I will do my best as well once I get some time).

If anyone is interested in this let me know. I’m more than happy to help with my domain knowledge (environmental design) to get introduced into this important topic.

Kind regards,
Theodore.

1 Like

Thanks for the code. Congratulations on the Kaggle entry

Hi to everyone, I am on the third year of college and I have for my final work pedestrian counter on raspberry pi. Is it possible to do it with PyTorch and FastAI and perhaps some help about that how to start what to use.
Thank you and I hope soon i will have my project done and be able to publish it here.

Thank you.

First step, identify a training dataset of pedestrian images. After that, this is a fairly straight forward problem. There are most likely similar Kaggle competitions and commercial tools are available for this task. You should do some market research to get ideas for implementing your project. After that. Work your way through the fastai image classification and segmentation lessons. Best of luck!

Thank you, didn’t mentioned that project is mentioned to be done by me, and by that I mean coded by me, and i want to learnDeep Learning, PyTorch and FastAI.

Thank you for your response Jeff.

Hi,

I prepared a model for Twitter posts to check the mood of the text :grinning:

https://klemenko.pl/twitter

mood

And also a webpage to upload images and check if it’s a bear/grizzly or teddy from first lessons, from a camera or a picture:

https://klemenko.pl/images

The instruction how to do it I put a model and use your camera you can find:

How to prepare text recognition for twitter (I’ve got for now 0.74 accuracy)

7 Likes

I’m thinking of using the spectrogram or wavelet for a dataset of mine. Would you mind sharing the code that you’ve used to turn the sounds into well formatted spectrogram images (with no axes, etc.) ?

1 Like

Hi Everyone,

Special thanks to Jeremy for the great class.

I am new to the data science world (started on December 2018), so I decided to dive inside the fastai library, to really understand what’s going on under the hood.

It turns out that if eventually you want to create your own model, or prepare your own data in a non-default way (imagenet style for example), you will need to get to know with datablock API, custom ItemList, create_head(), create_body(), and adaptiveconcatpooling()… And once you start to play with the low level API, you will be shocked how powerful fastai is. :joy:

I want to eventually create my own Siamese network using fastai, (or TripleItemList with anchor / positive / negative). Based on my understanding, this is essential because if you have thousands of classes, Resent accuracy wont be good enough. 1/1000 is a very low probability for guessing, and if you have imbalanced and small dataset,you will have more problem.
(Currently still working on Siamese, and it is far from done and polished to share :joy:)

I used MNIST as my first step to understand custom Item List, so I would like to share to the folks like me new to the fastai and thinking custom Item List is really some advanced stuff (labeled ‘advanced tutorial’ in the doc :grinning:, it is really not, and it is very easy to use)

Here is the kaggle kernel for the traditional MNIST dataset

MNIST with transfer learning

The reason I am using MNIST is following:

  1. It’s a very simple dataset, and very easy to understand
  2. We talked a lot in the class (lesson 5 and 7)
  3. It doesn’t work with default fastai transfer learning style (the data is not img file, it is tabular formatted)

If you ever wondering,

Lesson 5, we did use the pixel value as input, but we create dense net from scratch, no transfer learning involved. We pretty much prepared data in Pytorch style: ‘dataloader, dataset’

Lesson 7, we created our own Resnet CNN model, but the data is in img format, not pixel anymore.

What if I want to use ItemList with pixel input so I can take advantage of transfer learning / argumentation and other cool method like predict with TTA?

We will have to create own ItemList.

For the vision part, the call flow like the following:

from_folder(), from_csv(), from_df() all looking for a filename, then eventually calling get(), the get() will call open() to get single Image (Itembase), then you can start split(), label(), transform(), databunch(), normalize()…

In the MNIST example, if you follow the above path, the custom ItemList is simple, you only need to override open(), and leave the rest part to the ImageItemList(), your super() class. (You don’t even need a init() for your custom class).

Another point I would like to share is how easy add_test() can be used instead of add_test_folder()

add_test() takes an ItemList iterator, when you want to add the test-set after training, you can simply call add_test(CustomItemList.from_df()), you don’t need to override anything, fastai takes care for you.

The computer vision path seems fascinating to me at the moment,

Identify cat / dog -> Identify Species --> Identify Large class (1000+) —> Identify Large class with multiple items in one image

Really looking for part 2 for the bounding box explanation, so I can further my identification project to more than 1 item in single image :slight_smile:

3 Likes

I wrote a tutorial on connecting a text generator (like the one from the IMDB notebook) to twitter – Medium article here. Comments/feedback very welcome!

1 Like

hey @ilovescience. I will surely try that and see if it works. Seems like it does in your kernel. Thanks.

After lesson 5, I followed Jeremy’s suggestions and tried a few different pieces of homework.

Writing my own linear class for the Lesson 5 SGD notebook:

  • Per Jeremy’s suggestion, I decided to try out writing my own linear class (to use instead of the nn.Linear class)
  • I’m still quite new to coding, so this was pretty difficult for me (it took a few hours, with lots of Googling). After completing it, I feel like I know a lot more about PyTorch, so it was definitely a valuable exercise
  • The full code is available at GitHub

Adding momentum into the Lesson 2 SGD notebook:

  • I edited the SGD update function to incorporate momentum. For each step, I calculated the size of the step as step = moms * last_step + (1-moms) * gradient. This was then multiplied by the learning rate to calculate the adjustment to the weight. For the first epoch, I set momentum to zero (a previous step does not exist, so the formula does not really work)

  • I found that with the momentum value set to 0.8, the model trained significantly faster than the previous version

  • The code is available at GitHub

Adding momentum into the update function in the Lesson 5 SGD notebook:

  • Similarly to what I did for the Lesson 2 notebook, I edited the Lesson 5 update function to incorporate momentum. This was a bit harder than the Lesson 2 version, since this model contains multiple sets of parameters

  • The code is available at GitHub

Going back to previous models and trying to improve them:

  • During Lesson 5, we learnt about some different options you can use when building models (e.g. weight decay, momentum)
  • I went back to one of my previous models and played around with the weight decay value (I increased it a lot from its default of 1e-2), and was able to improve the accuracy significantly
5 Likes

Cloud Classification

While revisiting the earlier lessons of Part 1 using the new version of fastai, I tried to train a model to classify the type of cloud in the sky based on appearance (cumulus, stratus, cirrus). They look something like this:
image

I was able to get a good enough accuracy but the model seems to be confused between Stratus & Cirrus clouds. On visual examination, I could see that these two clouds are indeed difficult to distinguish between, even by a human at the first glance. Can the model do better?
image
Here’s a link to my research notebook:
Cloud Classification Research Notebook
Cloud Classification Repo


Bonus Inspiration: ‘Quick, Draw!’ for Clouds
As I was going through pictures of clouds, it occurred to me that as a kid I used to look at the sky when the storm was gathering and tried to imagine the clouds as an approaching army of mythical beasts. Turns out it is indeed a real hobby for some people: The joy of cloudspotting: 10 incredible visions in clouds

“Is that a UFO?”

Can we make something like Quick, Draw! but for clouds, where a model learns to recognize objects in clouds instead of doodles and drawings?

3 Likes

This has been a project I procrastinated on posting, but it is a fun one! Identifying different pieces of cardiac anatomy inside real human hearts. I made a longer post here on medium. (https://medium.com/@erikgaas/teaching-a-neural-network-cardiac-anatomy-f14f91a4c4bf)


Above are different shots of camera inside a heart perfused with clear liquid. If you want to see videos of these regions in action, visit my lab’s webpage on cardiac anatomy. (http://www.vhlab.umn.edu/atlas/) I scraped the site’s videos and converted them to still frames.

Resnet34 does really well at the problem, and reveals some fun things about our dataset whether it be regions that happen to look similar, or simply are present multiple times in the same image.


Fun stuff! I did this before in keras back in the day. Fastai makes this so much easier and more satisfying.

7 Likes

Hey Guys,
I have created a github repository for the CV and NLP datasets that I have implemented using fastai on Google Colab.
Main Repo-https://github.com/navneetkrc/Colab_fastai

Most recently I did a project on my own dataset automating the KYC document classification task.
KYC Classification
You are welcome for your reviews and feedbacks.

3 Likes