General course chat

@Danrohn Some late answers to a couple of your questions.

The latest version of pillow can handle more image data types, including 32-bit signed integer pixels mode='I', which I imagine covers the raw images. If not, I created TensorImage compatible item transforms for when I needed more high-bit channels then pillow supported at the time. You might be able to use those out of the box, or as inspiration for your own implementation.

I created the Simple Profiler Callback to help debug these types of issues. There’s currently no pip package, so you’ll have to install from git, but once installed add these two lines to profile your training run:

from fastxtend.callback.simpleprofiler import *

# define learner like normal

learn.profile()

# train for at least an epoch to profile
4 Likes

Thank you for the Simple Profiler tip! I’ll look at it.

Regarding the Pillow: Yes but actually no (as far as I understand… I might be wrong).

PIL objects are indeed able to read files that have 16bit,24bit or even 32bit like you mentioned, but their mode doesn’t support RGB format - which is to say 3x16bits.

Their I mode supports only one channel.

When I even tried to change their source code’s dictionary, and add that missing key of 3x16bit (or 3x24bit), I encountered annoying errors that I couldn’t handle or bypass.

The most annoying part was, personally, that even when they have that BMP format class, to support BMP files (3x24bits), their object reads the file but stores its values in a matrix of mapped values in between 0-255 (meaning, it was converted to 3x8bits RGB mode).

I’ve already finished with the TensorRawImage (and got some better results! I’m quite satisfied), but please enlighten me about that PIL supporting 3x16 (or higher) modes… I’d be horribly happy to know. :wink:

Thank you so much for watching and for the kind words :pray:

1 Like

This paper which claims to increase the performance of classifiers across the board using what they call as “PolyLoss” looks interesting. Going trough it now and sharing the same to have other’s views on the claim and implementation.

1 Like

fastai supports 3-channel 16-bit tiff files fyi:

2 Likes

I’m really loving the top-down teaching approach of fast ai and would like to take in as much as I can during these 7 weeks.
Is there a list of what the topics are for the rest of the short course?
Week 2 looks to be about NLP, is it possible to ask what Weeks 3-7 are going to be?
I’d like to supplement the week’s learning with fastbook and plan my weeks ahead so I can keep up with the learnings as we go.

I apologise in advance if this has been mentioned somewhere and I overlooked it.

Thank you.

1 Like

That’s neat!
So bad that I didn’t know it earlier. But well,
maybe I developed something that is useful for other purposes.
I’ve flicked through that link of yours, and it indeed seems to support 16bits depth.

I’ll try a bit later to load some RGB files with it. As far as I remember, TIFF files are of 1 channel (I may be wrong, again).

I think that TensorRawImage and RAWImage can only contribute in addition to TensorDicom and PILDicom(accordingly). With RawPy it’s possible to support many formats of files, such as *.ARW, *.ORG, *.DNG and anything that RawPy does support in.

I don’t claim to speak for Jeremy, but the 2020 course (generally) followed the fastbook chapters. The link to fastbook is in the post for the first lecture and you can find it in the fastai github as one of their repos.

HTH

Now after looking at the source code of PILDicom, I will try to create my own PILRawpy, hopefully, that PIL will store the data of those 3x16bits RGB modes without converting them to 3x8bits RGB. If so, then maybe I can optimize my implementation so that TensorImage will support RAW files through PIL objects as well as TensorRawImage actually does.

Telling aircraft apart is tricky. I’m guessing my training set (from DDG) didn’t have any 737 MAX 10 with Boeing and 737 written on them in very large letters. :slight_smile: Maybe we need a “classic fails” thread?

The failure was mine - initially - and after correcting my mistake of identifying a BaldEagle as an Embraer E175 - my model decided the 737 was an Airbus 320… 65% probability.

What we cover each week depends a bit on what we get thru in previous weeks, and what comes up along the way. Broadly speaking, however, the next couple of lessons will largely cover stuff from chapters 1-4 of the book.

2 Likes

I’m confused (and curious.) … did you train your classifier on birds and planes or just birds (or just planes)?

1 Like

Thanks for clarifying Jeremy. Will go through and start experimenting with chapters 1-4 of the book so i can keep up with the lessons.

1 Like

:slight_smile: I was comparing just birds (different eagles) then switched to try comparing small jets. I just left an old reference to my test Bald Eagle after duplicating the notebook - and it most closely matched the Embraer. Who knew!

1 Like

Ah ok, so for the second experiment you used planes to train the network but it still sees some similarities. Based on what I’ve seen, it can only make predictions based on stuff it has “seen” … but I think in chapter 3 we’ll learn how to make them say things like “this is out of my domain”

1 Like

The probabilities will always add to one in this case - so ‘most similar’ may still be very different. Birds are like planes, but I’m sure zucchini would have scored around 35% to at least one of my three planes.

(Edited) I just had to try - and quite suprised by the result and why zucchini, even when there are two look so much more like a Boeing 737 than the other two jets. 80%

1 Like

Zucchini, not zuccini :slight_smile:

2 Likes

While the MAX 10 has had test flights, it is still a yet to be certified airplane. I would agree with your guess that the training probably does not have any images, or enough samples to identify the slight changes between other MAX aircraft.

1 Like

Hey,
I have a batch of 4 pairs, with each pair (item,item) to have two items (input,target) of images (TensorImage).

Now I’m trying to add this augment of RandomErasing() only on the input image of each pair (I don’t want that to apply on the target image).

This is the augment:

# Cell
def cutout_gaussian(x, areas):
    "Replace all `areas` in `x` with N(0,1) noise"
    print(x.shape)
    chan,img_h,img_w = x.shape[-3:]
    for rl,rh,cl,ch in areas: x[..., rl:rh, cl:ch].normal_()
    return x

# Cell
def _slice(area, sz):
    bound = int(round(math.sqrt(area)))
    loc = random.randint(0, max(sz-bound, 0))
    return loc,loc+bound

# Cell
class RandomErasing(RandTransform):
    "Randomly selects a rectangle region in an image and randomizes its pixels."
    order = 100 # After Normalize
    def __init__(self, p=0.5, sl=0., sh=0.3, min_aspect=0.3, max_count=1):
        store_attr()
        super().__init__(p=p)
        self.log_ratio = (math.log(min_aspect), math.log(1/min_aspect))

    def _bounds(self, area, img_h, img_w):
        r_area = random.uniform(self.sl,self.sh) * area
        aspect = math.exp(random.uniform(*self.log_ratio))
        return _slice(r_area*aspect, img_h) + _slice(r_area/aspect, img_w)

    def encodes(self,x:TensorImage):
        count = random.randint(1, self.max_count)
        _,img_h,img_w = x.shape[-3:]
        area = img_h*img_w/count
        areas = [self._bounds(area, img_h, img_w) for _ in range(count)]
        return cutout_gaussian(x, areas)

I tried to adjust this line:

for rl,rh,cl,ch in areas: x[..., rl:rh, cl:ch].normal_()

into:

for rl,rh,cl,ch in areas: x[0, :, rl:rh, cl:ch].normal_()

Which then applies the augment only on the first pair (both input and target) out of the 4 pairs (in the batch).

How can I change that function or line so that erasing would apply only on the input item of each pair in the batch?

Thanks

I’d usually say Courgette (I’m en-GB, not en-US). But I was referencing from a cucumber v zucchini thread.

1 Like