Super resolution network result in 'random' noise images


(sbhhbs) #1

I’m trying to training a super resolution network following the tutorial here: https://github.com/fastai/courses/blob/master/deeplearning2/neural-sr.ipynb

However, after training for half day, the predicted result looks like random noise images and the content loss decrease are almost flat.

The input image is:
45

The noised image looks like this:
20

And when I inspect each channel in RGB, however, it looks like it is working. The details and edges are somehow recovered:

R channel:
27

the B and G channel looks just like R channel.

So my question is, what could the possible reason be? Or is it just a normal thing that indicate I should continue training?

If I should just continue training, then how could I continue decease the loss? Currently the loss doesn’t seems going down anymore. I’m using adam and the learning rate is already at 1e-4.


(David Gutman) #2

Might be a color channel scale issue.

Try to bring all three channels back to int 0-255.

Also try posting an image like this to get a better sense of what is in the three channels (Assuming channel is last dim):


for i in range(3):
    plt.subplot(1,3,i+1).imshow(img[...,i])

The images should have similar objects with different intensities if it is working.


(sbhhbs) #3

32

And I managed to get the result right. Here’s what I did. I trained it for another half day with a even smaller learning rate. And the result still is noise. And thanks to @davecg, your answer enlighten me. The scale are correct, I’m using the ‘tanh’ activation followed by a Lamda layer just like the teacher did, so the range is 0-255. However the data type seems to be wrong. After I force convert the result to int, the result showed.

Here’s the result when I force to int:
(Due to new user can only post 1 image per reply constraint, I’ll post this later)

And here’s the result just use as float:
(Due to new user can only post 1 image per reply constraint, I’ll post this later)

I’m not sure the root cause of this however, probably something to do with the plot library?


(sbhhbs) #4

The force covered to int result:
51


(sbhhbs) #5

And the result using float as is:

09


(David Gutman) #6

Matplotlib expects different ranges for float and int images.

Float should be [0.0,1.0]

Int [0,255]

Anything outside of those ranges will look strange.