About checkerboard artifacts and PixelShuffling.
Jeremy explained in detail how to avoid checkerboard artifacts using pixel shuffling. I still have some details I don’t understand: When using scale=2 I couldn’t observe checkerboard, however with scale=4 I do. So, obviously there are some parameters I need to tune.
I found in enhance.ipynb the following:
def upsample(ni, nf, scale):
layers = []
for i in range(int(math.log(scale,2))):
layers += [conv(ni, nf*4), nn.PixelShuffle(2)]
return nn.Sequential(*layers)
Jeremy mentioned in the lecture, that the 4 is really 2*2, because of scale 2. So, after some consideration I thought for scales!=2 upsampling should rather be:
def upsample(ni, nf, scale):
layers = []
for i in range(int(math.log(scale,scale))):
layers += [conv(ni, nf*scale**2), nn.PixelShuffle(scale)]
return nn.Sequential(*layers)
There still is something fishy, however. I can train a network using this upsample function, but I still get a checkerboard. I also really do not understand the for loop. math.log(scale,scale) comes out to always 1, so that probably isn’t the original intention. If I use math.log(scale,2) I get two conv+pixelshuffle blocks for scale=4. Why would I need that? Wouldn’t the model upscale too much then? So, I tried as above, but that kind of makes the for loop obsolete and still I get artifacts.
Did anyone try scale>2 and could comment about how to upscale and pixel shuffle correctly?