I just finished Part 1, Lesson 2 and deployed a simple web app. It takes a link to an image of art and attempts to identify the artist. I’ve only trained it on a few artists right now so it’s a bit limited in its scope but was informative to work on nonetheless! Link is below.
It has been a while since I wrote a blog post, but here it is
Besides some thoughts on testing, it gives an overview of how a machine learning project could be structured to improve the chances of arriving at a working solution. In some sense, this is a follow up on a post I wrote during taking the v2 version of the course:
In some sense, this is probably my favorite part of the post, one of my very first attempts to speak to a subject I care very deeply about
If youtube is pushing a change to its recommendation algorithm that will drive engagement (and ad revenue) but will do so by promoting conspiracy videos, should it test the impact on society? Yes. There is no absolution through the fact of not having considered something. The more powerful algorithms become in shaping our reality, the more responsibility we as the authors need to take. You cannot be driven by profit and say you are agnostic to the impact that you are having. That is the definition of an externality. Whether a corporation destroys an ecosystem by dumping waste into a river, or poisons people’s minds through promoting malicious content, the end result is the same.
so after lesson 2 I created a neural network that can differentiate buildings of following architectures:
Ancient Greek, Gaudi, Gothic, Islamic, Modern, Renaissance and Traditional Chinese
with an accuracy of about 95%. There used to be 17 classes but a lot of them were very similar like Traditional Chinese and Japanese or Postmodern and Modern so it only had an accuracy of about 45%, so I decided I would trim them down a little bit.
Hey fast folks!
So all you know already about the datasets @jeremy released a few days back(imagenette and imagewoof). I trained both of them on the same network. Below I am summarizing the results I got so far. Will be experimenting on this for sometime, after the revelation I had after seeing the results in Imagewoof.
Both the experiments have been done under 160px resolution and 40 epochs as suggested by Jeremy.
Imagenette results :
Training Accuracy: 71.12 Training Loss: 0.8912 Validation Accuracy: 70.39% Validation Loss: 0.8913
Now, for Imagewoof. drum rolls…
Imagewoof results :
Training Accuracy: 54.02% Training Loss: 1.2670 Validation Accuracy: **30.99%** Validation Loss: 2.0365
Thank you @jeremy for providing this awesome task to work on. So much to improve and learn from this! Time to implement the tricks you taught us in the courses. ** evil laugh **
All of my further work will be kept here.
Github Repo: https://github.com/hasibzunair/bonjour
Walk RNN - LSTM over graph
We recently conducted a small R&D project using a fast.ai language model for graph classification and compared accuracy to a recent paper which uses a CNN. I wrote a short blog yesterday about our results which has a link to our repo in github, if interested. In short, we passed in random walks over graph, enriched with structural and graph property information, to an RNN and trained a classifier with the purpose of labeling small graphs. (Apologies - I posted this yesterday, but this seems a more fitting location) Thank you Jeremy and the community for fast.ai!
Hi everyone , so I went for a Spectogram Classification ‘Problem’ , i made spectograms of 8 different types of whales sounds and fit trained a ResNet50 on them but this is what I get , i think it is overfitting , can somebody give me a cofirmation about it , thanks.
I’m trying to teach a CNN how to add and subtract… Is it possible? To keep this thread clean, I’ll post more details here:
After Lesson 6, I attempted two of the homework exercises that Jeremy suggested:
Creating my own dropout layer:
- For the Lesson 5 homework, I had created my own Linear class. So I took that class as a starting point, and updated it to add dropout functionality
- To do this, I edited the “forward” function to have an option for dropout (see screenshot below). This made my model run really slowly, so my code was obviously inefficient and not how you would do it in practice! But building it was still helpful for making sure I understood dropout properly
- I also tried the Bernoulli trial function that Jeremy showed, where you multiply activation values by 1 / (1/p) whenever you do not change an activation value to zero. But I got stuck with an error and did not end up implementing it it (the error was “one of the variables needed for gradient computation has been modified by an inplace operation”). Since I did not implement this part, if I was to use this model at test time (instead of just for training), I would need to multiply the model by (1-p) to adjust appropriately
- Full code available at GitHub
Creating a mini-batch without using data.one_item:
- Per Jeremy’s suggestion, I decided to try creating a manual mini-batch as input for our pets heatmap
- This required a lot of different steps (e.g. cropping my image, resizing it, normalizing it, converting it from a numpy array to a PyTorch tensor), so it took me a little while to figure it all out. It was definitely helpful for understanding exactly what happens when we normalize an image
- The heatmap seems to show correctly, but when I show it I also get an error message (“Clipping input data to the valid range for imshow with RGB data ([0…1] for floats or [0…255] for integers”). Does anyone know what this error means, and what I should have done differently?
- Full code available at GitHub
Have a look at the data type and range (min & max) you are handing over to the plot function.
From the code I would guess
avg_acts is a float value so normalizing it to lay between 0…1 should do it (if it is an integer the values should be between 0…255).
Certainly not - it’s classifying your validation set perfectly!
Thank U Jeremy , it’s the reason why i asked the question , in the sense that it was too perfect to be true .
thank’s for the confirmation.
Thanks for the help! I was able to fix it, it turns out I had an issue with the way I was normalizing / denormalizing, and the denormalized image values were not all between 0 and 1
hope I can reach your level as well. Trying to catch up with Pytorch fundamental recently. Although it is not useful, I still think implementing the simplest model (LeNet) with PyTorch DataLoader, Dataset, training loop is a great practice. A lot of “blackbox” vanish once you print every variable shape and see how it changes step by step.
The way @heye0507 describes working through my notebooks… is very reminiscent of how I would use the lecture notebooks.
That is a super impressive progress congrats! I am guessing you might have a bit of scientific or developer background going on Above the speed of progress though, what counts more I think, is that the approach seems very sound. I am guessing most of the folks out there (including myself) would not be able to make progress that fast. But it does not matter - as long as one is able to tinker a bit here and there, follow the advice Jeremy gives in lectures (work through notebooks cell by cell, change a few things here and there, recreate notebooks on new datasets) I have reasons to believe all should be well
I am in the same boat as you - have been driven to understand things in the lowest of details I think this works quite okay, though I wished when doing the v2 course I would have spent more time playing with the high level API instead of jumping so quickly to trying to peek behind the curtain.
On the other hand, understanding how Datasets / Dataloaders work is super useful in the flexibility they afford… and one gets a general sense of how things work. I don’t think there is a right or wrong answer as long as one still uses the high level APIs from time to time
All I really wanted to say when I started to type this message is that the way both of you describe learning is how I continue to learn Super happy to be sharing the journey with you and all the other fine people in this awesome community!
I did a picture of Barack Obama before his presidency and got 33 years, and after he got 52 years. Stressful 8 years aged him 19 years!
An “AI philosophy” paper I wrote that passed (pretty rigorous) peer review last fall is finally out of “on hold” status on arXiv (Computers & Society). Not sure when the journal issue will come out, but it should be soon. In the meantime, happy to share this!
I have written another blog about altair, ipywidgets and qgrid which is useful for interactively explore and plotting data. I have quite enjoyed how you can make interactive plot with altair so far. though it may not support all kind of plots yet.
i trained the ULMFIT in hebrew, and create a web that trying to predict the next word of some Political commentator.
you can try translate to hebrew, predict some, and them translate back to see the result