Question about Tiramisu segm generator


(alex) #1

Hi everyone,

I have a question about the segm generator as part of the tiramisu network:

My understanding is that the paper says to train and then fine-tune, where training is on random crops and flips and fine-tuning is on the full sized-images - but I can’t see where this is implemented in Jeremy’s (awesome!) notebook…

It may just be that I don’t really understand how segm generator works but where do you turn on/off the random crops?

class segm_generator(object):
def __init__(self, x, y, bs=64, out_sz=(224,224), train=True):
    self.x, self.y, self.bs, self.train = x,y,bs,train
    self.n, self.ri, self.ci, _ = x.shape
    self.idx_gen = BatchIndices(self.n, bs, train)
    self.ro, self.co = out_sz
    self.ych = self.y.shape[-1] if len(y.shape)==4 else 1

def get_slice(self, i,o):
    start = random.randint(0, i-o) if self.train else (i-o)
    return slice(start, start+o)

def get_item(self, idx):
    slice_r = self.get_slice(self.ri, self.ro)
    slice_c = self.get_slice(self.ci, self.co)
    x = self.x[idx, slice_r, slice_c]
    y = self.y[idx, slice_r, slice_c]
    if self.train and (random.random()>0.5): 
        y = y[:,::-1]
        x = x[:,::-1]
    return x, y

def __next__(self):
    idxs = next(self.idx_gen)
    items = (self.get_item(idx) for idx in idxs)
    xs,ys = zip(*items)
    return np.stack(xs), np.stack(ys).reshape(len(ys), -1, self.ych)

As far as I can tell, setting train=true only shuffles the order of the images returned by the generator, it doesn’t turn cropping on/off?

Thanks in advance!

p.s. bonus question: to do the train phase then the fine-tune phase, do I just setup the generator with cropping then set up a new generator and train the model some more with that new generator?


(Davide Boschetto) #2

I’m having the same (or a similar) issue. I’m dealing with a VERY little dataset (150 1024x1024 images). Random crops are great, but mixing the segm_generator with ImageDataGenerator would be even better.
Any suggestion on how to do this? Can’t use two generators in “cascade”, and can’t insert imagedatagenerator within segm_generator by myself… It’s been more than 30 minutes, sadly, so I have to ask!


(alex) #3

Not the most elegant solution but you can precompute the random crops + image transformations and just use that instead of a generator…