DataBlocks API examples

[arora_aman (Aman Arora)]

Creating a separate thread which could serve as a megathread and a central repository to share various different examples using the DataBlocks API for various different applications such as images/text/tabular/colab.

We already have this wonderful notebook https://github.com/fastai/fastai2/blob/master/nbs/50_tutorial.datablock.ipynb with some great examples.

Various different DataBlock examples available so far:

Image Classification

1. (ImageBlock, CategoryBlock) Input: 1-channel image, Output: 1 label
   MNIST
2. (ImageBlock, CategoryBlock) Input: 3-channel image, Output: 1 label
   PETS

MultiLabel Image Classification

1. (ImageBlock, MultiCategoryBlock) Input: 3-channel image, Output: Multiple Labels
   Amazon Planets MultiLabel

ImageSegmentation

1. (ImageBlock, MaskBlock) Input: 3-channel image, Output: Mask
   Camvid

Image Regression

1. (ImageBlock, PointBlock) Input: 3-channel image, Output: Multiple Points
   Facial Landmarks
2. (ImageBlock, RegressionBlock) Input: 3-channel image, Output: 1 Real Number (Age)
   IMDB-WIKI – 500k+ face images with age and gender labels

Object Detection

(ImageBlock, PointBlock) Input: 3-channel image, Output: Multiple Points

1. COCO

Text

1. IMDB

[jeremy (Jeremy Howard)]

Might want to link to nb 50?

[VishnuSubramanian (Jarvislabs.ai)]

@muellerzr already has a datablock for Bengali competition, which we may use as a starting point, and see if we can include some other tips from the winning solutions.

[arora_aman (Aman Arora)]

Wow, I didn’t know!

[barnacl (Rekil Prashanth)]

Not sure how we go about arranging all our examples.
I was thinking all in a notebook will make it very long.
At the end of the day in either the documentation section or tutorials we should have Datablock examples for different categories.

@arora_aman https://github.com/muellerzr/Practical-Deep-Learning-for-Coders-2.0/blob/master/Computer%20Vision/06_Multimodal_Head_and_Kaggle.ipynb

[muellerzr (Zachary Mueller)]

@arora_aman

I didn’t know!

Was about to say, my repo should have a few creative examples we can put on here, let me check real quick

Off the top of my head:

• Style Transfer (ImageBlock, ImageBlock): url It’s different because it’s image to image where you do not have a specification for a y as your input is your labelled output

• Multi-Keypoint regression (ImageBlock, PointBlock): url bringing in a pure dataframe with 17 different keypoints and formatting it in a way the DataBlock API will accept

• Multi-Label from single label classes (ImageBlock, MultiCategoryBlock) url

• Binary segmentation + common pitfalls (very specific about segmentation in general, may not fit well) (ImageBlock, SegmentationBlock)url

• @barnacl already mentioned the age regression so I won’t mention it again

Okay @barnacl i’m working my way there stop beating me

[arora_aman (Aman Arora)]

Adding to documentation with explanations would be most useful I think.

[barnacl (Rekil Prashanth)]

The age prediction - Image Regression is also a different one from @muellerzr https://github.com/muellerzr/Practical-Deep-Learning-for-Coders-2.0/blob/master/Computer%20Vision/06_Scalar_Regression.ipynb

[arora_aman (Aman Arora)]

We can add explanations to CategoryBlock, TransformBlock, ImageBlock etc and also add multiple examples from the excellent work done by @muellerzr already.

[barnacl (Rekil Prashanth)]

Right probably we should convert this to a wiki to add examples at top. for example with the Bengali AI we write a one liner like:
Given an image predict 4 different labels( ImageBlock,CategoryBlock,CategoryBlock,CategoryBlock) (link)[https://github.com/muellerzr/Practical-Deep-Learning-for-Coders-2.0/blob/master/Computer%20Vision/06_Multimodal_Head_and_Kaggle.ipynb]
Something like that

[arora_aman (Aman Arora)]

@muellerzr can make this a wiki for us

[muellerzr (Zachary Mueller)]

@barnacl

Something like that

We’re on the exact same wavelength right now, I don’t know how we managed that

[muellerzr (Zachary Mueller)]

@arora_aman

make this a wiki

Can I? I suppose so, it’s done

[barnacl (Rekil Prashanth)]

haha didn’t meant to (or did i ). Just had your AMAZING repo open (this videolectureusc btw on youtube who was bugging you with questions as usual ) Have to checkout the permutation importance stuff

[arora_aman (Aman Arora)]

Okay, now that this is a wiki, peeps spread out and get to work and let’s bring back multiple examples haha

I will get to humback whale competition then.

[muellerzr (Zachary Mueller)]

@arora_aman

let’s bring back multiple examples haha

Just edited my above post with the format you asked, for, feel free to copy and paste, I need to get back to that blog homework

[arora_aman (Aman Arora)]

Thanks. See ya later!

[muellerzr (Zachary Mueller)]

One example to get creative minds thinking is a DataBlock example that combines (and needs to combine): get_x, get_y, and get_items. Just FFT, I’ll return to this later if there’s questions

[muellerzr (Zachary Mueller)]

@arora_aman

CategoryBlock, TransformBlock, ImageBlock

FYI, simply passing in TransformBlock() used to be how to do regression, but now we have RegressionBlock() (thag could be a good example of whats different between the base class (TransformBlock) and the RegressionBlock

[Moody (Sarada Lee)]

It’s come to my attention that the ImageBlock is no longer available under TransformBlock. Is it being renamed or deleted accidentally?