I download “trn_resized_72.bc” and “trn_resized_288.bc” from here, the color of these images are very weird.
plt.figure(figsize=(7,7))
plt.imshow(low_res_imgs[img_idx])
plt.figure(figsize=(7,7))
plt.imshow(high_res_imgs[img_idx])
Results
I guess maybe it is because the order of channels
plt.figure(figsize=(7,7))
plt.imshow(low_res_imgs[img_idx][:,:,::-1])
plt.figure(figsize=(7,7))
plt.imshow(high_res_imgs[img_idx][:,:,::-1])
Results
Looks better, but still weird. I try to deprocess it with some try, but all fail
deproc = lambda x: np.clip(x[:,:,:,::-1] + rmean, 0,255)
deproc = lambda x: np.clip(x[:,:,:,::-1] - rmean, 0,255)
However, this is not the weirdest part, the most incredible part is, the neural net know how to upscale the image and do some “fix” on the color of the input image.
top_model = Model(low_res_inp, output)
plt.figure(figsize=(7,7))
plt.imshow(low_res_imgs[img_idx])
p = top_model.predict(np.expand_dims(low_res_imgs[img_idx], 0))
plt.figure(figsize=(7,7))
plt.imshow(p[0].astype('uint8'))
Result
Try it with random image download from google search(I bet it is not in the training set)
img = Image.open('img/fish.jpg')
img = img.resize((72,72))
plt.figure(figsize=(7,7))
plt.imshow(img)
img_array = np.expand_dims(np.array(img),0)
p = top_model.predict(img_array)
plt.figure(figsize=(7,7))
plt.imshow(p[0].astype('uint8'))
Output image become sharper and color of the fishes changed. I am amazed the network can learn how to “fix” the color even the color of the training set are weird. These results looks interesting to me, I guess it is because vgg16 already learned the info colors, edges, shapes and other features I do not know.