Hey! Can you include a notebook generating those loss plots/animations? I’d be very interested to incorporate such things into some of my presentations 
working on a blog wrt the 2 swift lessons here: https://docs.google.com/document/d/1wvGUaYu7kDcmOkr-Aw2sSwl4WsxGiBdtP0OgzD149Wc/edit?usp=sharing
Its a draft, would only be shared when the MOOC goes live (so pls do not use the share function, download or alike). You can use it as notes and links into the 2 classes and comments are of course welcome
2 Likes
it needs some polishing/cleaning (it cannot be rerun as is), I am planning to write something about the experiment above.
Hi Everyone,
I have created a thread in the Part 1 of the forums for more people to be able to access it, However, wanted to share it in the active part of the forums as well.
AMA Link: Interviewing for Google AI Residency: AMA (Sanyam Bhutani)
Earlier this year, I had been invited to the Google AI Residency’s final round of interviews, I did not make it however I’ve created the AMA to help anyone else prepare and hopefully even get in, next year.
Please check it out if you’re interested.
PS: Also, this was the reason for me not being as active during the earlier months. Apologies for not contributing as much as I wanted to.
2 Likes
Part 2 of The Hitchhiker’s Guide to AI Ethics is out as well. Read it here.
Look forward to feedback from the Fast.ai community!
3 Likes
Ended up missing a lot of Part 2 (2019) and/or watching Jeremy at 1.5x speed, because I was trying to finish this project/paper, which is finally out!
"SignalTrain: Profiling Audio Compressors with Deep Neural Networks" https://arxiv.org/abs/1905.11928
Demo & audio samples: http://www.signaltrain.ml
A shorter form of this paper has been submitted for the fall AES conference.
Used Fasti.ai know-how if not the actual library. Thanked @jeremy & @sgugger in the Acknowledgements for so freely sharing with us & answering my questions!
I may wait until June 11 to un-hide the code repo & datasets; that’s when I present to the local “Data Nerds” meetup, where I hope to get some feedback on my documentation & organization, etc.
4 Likes
TL;DR Long due thank you note to Jeremy and this community, fastai changed a lot in my life, AGAIN!
I have been waiting to write this for 3 months but due to a hectic work schedule and other commitments, I couldn’t find the time and I felt bad every passing day, so here it goes.
I started my journey with fastai almost 2 years ago and easily got a job at Amazon (linked above), that was the first turning point in my life and first acceptance of my skills as a deep learning researcher without any formal computer science or ML background.
I found parts of my work toxic to my research aptitude and health in general and decided to leave within 10 months and again sit back and re-visit fastai courses and notebooks in great detail. This was indeed the most productive time, I found new things every time I re-watched a lesson (I suggest that, if you like watching something while eating). This boosted my confidence and I applied for Google AI Residency again, my 3rd attempt, and I cleared the pre-screening and went on for 2 rounds. I’m happy to talk about it in details but overall I can say whatever we learn with fastai is sufficient for anyone to apply and make it, although I didn’t make it but I can say with good confidence that the reason was not my deep learning knowledge.
Takeaway: Don’t be afraid of AI labs, always try!
While applying for residency, I also applied to more than 500 jobs outside India and fortunately got 3 replies back, 2 from London and 1 from Canada and I received offers and visa sponsorship from all three. Currently I am working in London as a computer vision researcher and between changing jobs and countries since October 2018, I somehow managed to complete both parts of 2019 course. I must say that the second part of 2019 is as eye opener for me, ‘Foundations’ idea really worked for me and it sparked more confidence and enthusiasm.
Takeaway: Apply for ambitious roles and countries, knowing odds are roughly 0.5%. Also, be regular with something which makes you feel positive and helps in learning, basically fastai
My current job allows me to implement research papers and produces results, I recently implemented an improved version of hierarchical CNN (HD-CNN), using all my learning from part 1 and part 2, this was only possible because we are all taught how to read the greek and write code in this course. It’s my 2nd productive(making a few million for the company) implementation and I plan to implement more whenever I find time.
Takeaway: Research papers are weekend friends, they don’t sound cool but are very satisfying once you make a habit
A month ago (after 4 years of constant struggle and learning), I received my first invite as a DL speaker for a conference in London, I’ll be talking about ways of using computer vision to boost business productivity. At the same time, I also received an offer to write a book on deep learning and pytorch from Packt (can’t talk a lot about it for now but I promise to find free e-copies for this forum when it’s finished, and of course if you are interested). Foundations also encouraged me to start learning swift so I spend 2 hours every week learning swift and trying to create my own version of fastai library for a specific use case, maybe it will become something someday 
Takeaway: Don’t give up easily, it takes time to reap benefits of what you sow
My only regret is that I failed to contribute to fastai library even after so many tries, until last year I always felt inferior about my quality of code and understanding of concepts, and lately due to involvement in so many things. I hope I’ll find some time and confidence to support harebrain, once I complete with other commitments.
I would also like to mention that Sylvian, @radek and Christine were my source of inspiration throughout, I always looked up to their achievements and efforts to motivate myself.
Thanks if you read so far, I really appreciate it and I hope it helped
28 Likes
inspiring words! Thanks very much for sharing your experience
A new Unsupervised Parking Lot Detection method. I build it for a competition and thanks to it I got selected for the finals. Most of the details are in the github repo README with presentations, so here I just summarize the approach.
There was no dataset used to detect parking spaces. Just pre-trained models and I developed some ways using which we don’t even need to fine-tune the models. My implementation can be divided into these three modules:
- Object Detection Module :- Use COCO pretrained model, no need to do finetuning.
- Label Processing Module :- As our model is not finetuned, there are some tricks that I add to overcome these limitations
- Classification Module :- Use the processed labels/bounding_boxes to tell if that parking space is occupied or not.
Some details
- Object Detection Module:- Why not use fine-tuned model? You can finetune your model. But in order to show the generalization power of this approach, I refrained from finetuning my model. So t oovercome the limitations of pretrained models, I use the following approaches:
-
Use multiple images for getting the parking spaces
-
Split the image into a 3x3 grid of overlapping regions so as to get more occurate bounding boxes.
By using the above two methods the need for fine-tuning is essentially removed and as a result of this you don’t need any data for Parking Lot Detection thus making it an example of unsupervised learning.
- Label Processing Module :- I use label to refer to the bounding boxes produced by the object detection model. Now to process the labels/bounding boxes we require this module. This module mainly solves the following:
- Combine bounding boxes from multiple images
- Combine bounding boxes from the multiple splits of an image with the original image
- Techniques to remove all the redundant and overlapping bounding boxes and cleaning the labels.
- Classification Module :- This module takes as input the processed labels as produced by the Label Processing Module and classifies the patches of bounding boxes as occupied or not.
I am thinking about writing my first paper on this approach any suggestions on how to go about it.
3 Likes
I wrote a blog post after competing in the Haha 2019 Competition detailing my lessons learned:
Hopefully it can help some of you!
4 Likes
I used the DALI library to build an Imagenette image augmentation pipeline that runs on the GPU. Added three modifications that gets it working with the Learner class that we use here in part II. Notebook here.
Also wrote down my thoughts on DALI lib’s speed (yes, it’s fast) and flexibility (there’s room for improvement):
9 Likes
I have written a post about Neural net init: How to initialize a Neural Network
I made some experiments to compare different init schemes (Kaiming vs Kaiming+LSUV vs pytorch defaults) on a simple convnet and on xresnet50. The experiments results are in the post.
It seems that there is a big impact without batchnorm. But with batchnorm, the performances get quite similar.
1 Like
Here is some work I did to make pre-trained CartoonGAN significantly smaller and automatically correct problems with the way it produces colour. If you are interested in shrinking networks and fixing imperfections have a read. I’ve called it Fast Cartoon GAN 
All feedback welcome.
1 Like
I did some work to try to understand the effect of Batch Norm when transfer learning. I tried a ton of different visualisations (see here).
I’ve written a short blog post as well, I have a much better understanding of hooks and pytorch internals after lesson 12 (and part 2 in general). By freezing the non BN layers rather than freezing all the layers before the head the weight changes look smoother but I havent really been able to quantitatively understand whats happening.
I will add more detail about the Fasta part 2 course to the blog linked above once this part goes public.
NB in the code above I am training with dataloader shuffle off for weight comparisson. I will run again with shuffle on and update the notes shortly.
3 Likes
I’ve written a post about the MixMatch paper and tangentially pseudo-labeling. Happy to get any feedback!
https://laurenth.me/2019/07/03/tldr-mixmatch
2 Likes
I am working with a team in Omdena that is collaborating with the United Nations World Food Program. We are interested in using satellite imagery to identify and track staple food crop growth in Nepal to fight hunger. Check out how I used super-resolution to improve the quality of our data below!
6 Likes
Now that Tensorboard is supported in Pytorch (since v1.1), I have used the callback system to integrate it to fastai. Very easy, only a few lines of code!
9 Likes
1 Like
Hi everyone,
While playing with torch.sum, I was quite confused about dimensions and their order (mostly about the dim parameter).
That’s why I wrote an article about the process of summation of a 3D tensor illustrated with some visualisations. Hope it will help some of you to better understand tensors’ operations.
2 Likes