Finally I finished building ImageSegmentation pipeline for a Kaggle challenge TGS Salt Identification. The solution should be able to get you to top 1 - 5%.
The solution is an update to my old repo, which is based on fastai 0.7.
Key Features of the notebook.
Creating DataBlock (Dataset, Dataloader)
Model
Create FastAI unet learner
Create a custom unet model demonstrating features like
Deep Supervision
Classifier branch
Hyper columns
Train on K-Fold
Ensemble by averaging
Loss function
Classifier loss
Loss for handling Deep supervision
Segmentation loss
TTA - Horizontal Flip
Create a Submission file.
I wanted to record a code walkthrough and post it soon here. Posting it here so that I do not escape from doing it. Planning to pick up another competition probably quickdraw and build a complete pipeline. If anyone wants to join me on the journey please let me know.
I trained most of these models around Feb 10th with the work-in-progress v2 library. I went back and duplicated the work for one the models today: the API is still in place and everything works but the results got a lot better. Perhaps just lucky seed, but exciting to see improvements emerge when you haven’t done anything
After learning about fine_tune and trying to explore it further, I found a paper on coral identification inwhich they used Keras and a ResNet 50 for ~300 epochs and got an accuracy of 83%. Using some of the techniques from the first lesson, chapter 6, and chapter 7 (Progressive resizing, Test-Time Augmentation, and Pre-Sizing) I was able to get 88% accuracy in just 9 epochs! Read about it here
Edit: sorry it was a 404 for a moment, briefly rearranged things on the site and it broke the link
I worked to get U-GAT-IT working with fp16. It takes in a picture of a person, and then maps it to an anime image. (Cyclegan training in fp16)
Everything is currently a work in progress, but here is the results and a WIP blog: (btw looking for job)
Yes, all of this was done in fastai2. I have been working on it since October.
Hey! I happened to be learning about Auto Encoders when the invitation for this V2 course came in so I implemented three experiments in v2: https://github.com/jerbly/fastai2_projects. This was a good way to learn implementing simple pytorch models into v2 (small enough to run on CPU) and includes a custom batch transform class to add random noise for the Denoising Auto Encoder.
I’ve created a notebook to show how you can use np.arrays larger than memory. It’s based on np.memmap. I’ve used it to train a 20GB dataset on an 8GB RAM.
Hi everyone, I don’t know whether this belongs here, but sharing it anyway.
I made a NotifierCallback that notifies you through almost any Instant Messaging(Telegram/Slack/ and more…), SMS, Email, Push Notification service at each Epoch’s end.
Callback Code:
class NotifierCallback(Callback):
"Notifies you the losses and the metrics"
def __init__(self, service_addrs):
self.service_addrs = L(service_addrs)
try:
import apprise
except:
raise ModuleNotFoundError("Apprise module not found, Install it to use this Callback.")
self.apobj = apprise.Apprise()
for addrs in self.service_addrs:
self.apobj.add(addrs)
def begin_fit(self):
"Replace default logger with _notify"
self.old_logger,self.learn.logger = self.logger,self._notify
def _notify(self, log):
"Notifies all services and call the old logger."
msg_body = ""
for data in zip(self.recorder.metric_names,log):
msg_body += f"{data[0]}: {str(data[1])}\n"
self.apobj.notify(title="",body=msg_body)
self.old_logger(log)
def after_fit(self):
"Restore old logger"
self.learn.logger = self.old_logger
This is made possible by the awesome Apprise library.
There is a huge list of services supported by Apprise which you can view at Apprise Github Page
This is useful if you’re training a model that takes a long time to train and you need to do other work instead of checking often how your model is performing.
Yes Sanyam, I’m familiar with Knock knock, but Apprise supports a huge list of services (50+), Knock Knock only supports a few(12). You can view all the notification services they support here.
This is very cool, I wanted to try something like that myself. However from your blog I understand you were using batches of 3 256pixels images and still running out of memory on 2070 GPU?
In that case, as someone just renting a paperspace machine, I guess I’d better wait for more efficient algorithms or better GPUs to become avaialable.
Depends on the paperspace machine you get, from what I can see, the smallest Paperspace GPU still have 8GB of GPU ram, which is the same as the 2070, other machines have more again
Good point @morgan, I wonder about the training time as well. I might spend few bucks to see If I can run the code from the original paper on p5000 and get it to finish.
I used 2 2070S so I had 16GB in total. It took me about 7 days to train it. Though I used a lot of memory saving techniques. I personally do not know of a way to get it to train in a “reasonable” amount of time on a 8GB GPU. I think model architecture changes would have to be made to get it to work on smaller GPUs.
I happened to be learning about Auto Encoders when the invitation for this V2 course came in so I implemented three experiments in v2: 