Super resolution network result in 'random' noise images

I’m trying to training a super resolution network following the tutorial here: https://github.com/fastai/courses/blob/master/deeplearning2/neural-sr.ipynb

However, after training for half day, the predicted result looks like random noise images and the content loss decrease are almost flat.

The input image is:
45

The noised image looks like this:
20

And when I inspect each channel in RGB, however, it looks like it is working. The details and edges are somehow recovered:

R channel:
27

the B and G channel looks just like R channel.

So my question is, what could the possible reason be? Or is it just a normal thing that indicate I should continue training?

If I should just continue training, then how could I continue decease the loss? Currently the loss doesn’t seems going down anymore. I’m using adam and the learning rate is already at 1e-4.

Might be a color channel scale issue.

Try to bring all three channels back to int 0-255.

Also try posting an image like this to get a better sense of what is in the three channels (Assuming channel is last dim):


for i in range(3):
    plt.subplot(1,3,i+1).imshow(img[...,i])

The images should have similar objects with different intensities if it is working.

32

And I managed to get the result right. Here’s what I did. I trained it for another half day with a even smaller learning rate. And the result still is noise. And thanks to @davecg, your answer enlighten me. The scale are correct, I’m using the ‘tanh’ activation followed by a Lamda layer just like the teacher did, so the range is 0-255. However the data type seems to be wrong. After I force convert the result to int, the result showed.

Here’s the result when I force to int:
(Due to new user can only post 1 image per reply constraint, I’ll post this later)

And here’s the result just use as float:
(Due to new user can only post 1 image per reply constraint, I’ll post this later)

I’m not sure the root cause of this however, probably something to do with the plot library?

The force covered to int result:
51

And the result using float as is:

09

Matplotlib expects different ranges for float and int images.

Float should be [0.0,1.0]

Int [0,255]

Anything outside of those ranges will look strange.