Hey,
so after lesson 2 I created a neural network that can differentiate buildings of following architectures:
Ancient Greek, Gaudi, Gothic, Islamic, Modern, Renaissance and Traditional Chinese
with an accuracy of about 95%. There used to be 17 classes but a lot of them were very similar like Traditional Chinese and Japanese or Postmodern and Modern so it only had an accuracy of about 45%, so I decided I would trim them down a little bit.
Hey fast folks!
So all you know already about the datasets @jeremy released a few days back(imagenette and imagewoof). I trained both of them on the same network. Below I am summarizing the results I got so far. Will be experimenting on this for sometime, after the revelation I had after seeing the results in Imagewoof.
Both the experiments have been done under 160px resolution and 40 epochs as suggested by Jeremy.
Imagenette results :
Training Accuracy: 71.12
Training Loss: 0.8912
Validation Accuracy: 70.39%
Validation Loss: 0.8913
Now, for Imagewoof. drum rolls…
Imagewoof results :
Training Accuracy: 54.02%
Training Loss: 1.2670
Validation Accuracy: **30.99%**
Validation Loss: 2.0365
Thank you @jeremy for providing this awesome task to work on. So much to improve and learn from this! Time to implement the tricks you taught us in the courses. ** evil laugh **
All of my further work will be kept here.
Github Repo: https://github.com/hasibzunair/bonjour
Walk RNN - LSTM over graph
We recently conducted a small R&D project using a fast.ai language model for graph classification and compared accuracy to a recent paper which uses a CNN. I wrote a short blog yesterday about our results which has a link to our repo in github, if interested. In short, we passed in random walks over graph, enriched with structural and graph property information, to an RNN and trained a classifier with the purpose of labeling small graphs. (Apologies - I posted this yesterday, but this seems a more fitting location) Thank you Jeremy and the community for fast.ai!
4 Likes
Hi everyone , so I went for a Spectogram Classification ‘Problem’ , i made spectograms of 8 different types of whales sounds and fit trained a ResNet50 on them but this is what I get , i think it is overfitting , can somebody give me a cofirmation about it , thanks.
I’m trying to teach a CNN how to add and subtract… Is it possible? To keep this thread clean, I’ll post more details here:
After Lesson 6, I attempted two of the homework exercises that Jeremy suggested:
Creating my own dropout layer:
- For the Lesson 5 homework, I had created my own Linear class. So I took that class as a starting point, and updated it to add dropout functionality
- To do this, I edited the “forward” function to have an option for dropout (see screenshot below). This made my model run really slowly, so my code was obviously inefficient and not how you would do it in practice! But building it was still helpful for making sure I understood dropout properly
- I also tried the Bernoulli trial function that Jeremy showed, where you multiply activation values by 1 / (1/p) whenever you do not change an activation value to zero. But I got stuck with an error and did not end up implementing it it (the error was “one of the variables needed for gradient computation has been modified by an inplace operation”). Since I did not implement this part, if I was to use this model at test time (instead of just for training), I would need to multiply the model by (1-p) to adjust appropriately
- Full code available at GitHub
Creating a mini-batch without using data.one_item:
- Per Jeremy’s suggestion, I decided to try creating a manual mini-batch as input for our pets heatmap
- This required a lot of different steps (e.g. cropping my image, resizing it, normalizing it, converting it from a numpy array to a PyTorch tensor), so it took me a little while to figure it all out. It was definitely helpful for understanding exactly what happens when we normalize an image
- The heatmap seems to show correctly, but when I show it I also get an error message (“Clipping input data to the valid range for imshow with RGB data ([0…1] for floats or [0…255] for integers”). Does anyone know what this error means, and what I should have done differently?
- Full code available at GitHub
1 Like
Have a look at the data type and range (min & max) you are handing over to the plot function.
From the code I would guess avg_acts is a float value so normalizing it to lay between 0…1 should do it (if it is an integer the values should be between 0…255).
1 Like
Certainly not - it’s classifying your validation set perfectly! 
2 Likes
Thank U Jeremy , it’s the reason why i asked the question 
, in the sense that it was too perfect to be true .
thank’s for the confirmation.
Thanks for the help! I was able to fix it, it turns out I had an issue with the way I was normalizing / denormalizing, and the denormalized image values were not all between 0 and 1
1 Like
hope I can reach your level as well. Trying to catch up with Pytorch fundamental recently. Although it is not useful, I still think implementing the simplest model (LeNet) with PyTorch DataLoader, Dataset, training loop is a great practice. A lot of “blackbox” vanish once you print every variable shape and see how it changes step by step.
1 Like
Thank you very much for your kind messages @heye0507 and @nok! Really nice to hear that you are finding the code I share useful.
The way @heye0507 describes working through my notebooks… is very reminiscent of how I would use the lecture notebooks.
That is a super impressive progress congrats! I am guessing you might have a bit of scientific or developer background going on 
Above the speed of progress though, what counts more I think, is that the approach seems very sound. I am guessing most of the folks out there (including myself) would not be able to make progress that fast. But it does not matter - as long as one is able to tinker a bit here and there, follow the advice Jeremy gives in lectures (work through notebooks cell by cell, change a few things here and there, recreate notebooks on new datasets) I have reasons to believe all should be well
I am in the same boat as you - have been driven to understand things in the lowest of details I think this works quite okay, though I wished when doing the v2 course I would have spent more time playing with the high level API instead of jumping so quickly to trying to peek behind the curtain.
On the other hand, understanding how Datasets / Dataloaders work is super useful in the flexibility they afford… and one gets a general sense of how things work. I don’t think there is a right or wrong answer as long as one still uses the high level APIs from time to time
All I really wanted to say when I started to type this message is that the way both of you describe learning is how I continue to learn Super happy to be sharing the journey with you and all the other fine people in this awesome community!
4 Likes
I did a picture of Barack Obama before his presidency and got 33 years, and after he got 52 years. Stressful 8 years aged him 19 years!
3 Likes
An “AI philosophy” paper I wrote that passed (pretty rigorous) peer review last fall is finally out of “on hold” status on arXiv (Computers & Society). Not sure when the journal issue will come out, but it should be soon. In the meantime, happy to share this!
1 Like
I have written another blog about altair, ipywidgets and qgrid which is useful for interactively explore and plotting data. I have quite enjoyed how you can make interactive plot with altair so far. 
though it may not support all kind of plots yet.
5 Likes
i trained the ULMFIT in hebrew, and create a web that trying to predict the next word of some Political commentator.
https://segal-generator.onrender.com/
you can try translate to hebrew, predict some, and them translate back to see the result
1 Like
Hey! I have started a little challenge of reading and summarizing a paper a day. I’ve started two days ago, and today I dived into Bag of Tricks for Image Classification with Convolutional Neural Networks which tests a number of techinques many of which are now implemented into fastai. If you are interested you can find it here. I hope you find it useful.
5 Likes
Thank you very much for the great articles!
I really enjoy them! 
1 Like