Lesson 8 Discussion

do let me know if you get the Style extraction part working in Keras 2.0. I am struggling with this, can’t see what’s my undoing :frowning:

Will do! Are you getting errors?

Nope. The loss function drops from 10,000 to 20 over10 iterations. But the random image input still looks random, can’t see the style being extracted

Thank you!

Hi,
You can use VGG-16 directly from Keras.


Here is an example on how to use it with tensorflow

Please let me know if you have any questions,
Yannet

1 Like

Note: the complete collection of Part 2 video timelines is available in a single thread for keyword search.
Part 2: complete collection of video timelines

I did a timeline of the Lesson 8 video as I found them very practical in Part 1 wiki.
There are many links, probably there was a lot of “happy noise” in the first class :upside_down:
I expect future timelines to be shorter.

Lesson 8 video timeline:

0:00 : Intro and review of Part 1

08:00 : moving to Python 3

10:30 : moving to Tensorflow and TF Dev Summit videos

22:15 : moving to PyTorch

27:30 : from Part 1 “best practices” to Part 2 “new directions”

31:40 : time to build your own box

36:20 : time to start reading papers

39:30 : time to start writing about your work in this course

41:30 : what we’ll study in Part 2

40:40 : artistic style (or neural style) transfer

52:10 : neural style notebook

54:15 : Mendeley Desktop

56:15 : arXiv-Sanity.com

59:00 : Jeremy on twitter.com and reddit.com/r/MachineLearning/

1:01:15 : neural style notebook (continued)

1:04:05 : broadcasting, APL as “A Programming Language”, and Jsoftware

1:07:15 : broadcasting with Keras

1:12:00 : recreate input with a VGG model

1:22:45 : optimize the loss function with a deterministic approach

1:33:25 : visualize the iterations through a short video

1:37:30 : recreate a style

1:44:05 : transfer a style

4 Likes

Hi all! Just wanted to point out few issues that I had with first part of lesson 8.

Task: Get the bird out of noise, using only tensorflow 1.2 and its contrib.keras without keras .That is, content transfer. (afaik from TF Dev videos, standalone keras will be deprecated some time in the future).

I had to replace the inports on vgg_16_average.py, and handcode the loss keras.backend.mean(keras.backend.square(content_model.output - content_target)). Other than that, most of the “porting” is straight forward and/or easily solved with a bit of doc surfing.

However, I fought a lot with a nasty bug (feature) in scikit-image. I used skimage.transform.rescale() to get my image size down. Default, this function also scales your image to [0,1]. VERY nasty feature, nothing worked. I got my MSE down but the out image is pure noise. It took me a while to track down this issue. So, use preserve_range=True parameter.

Lessons learned:

  • LL1: VGG is VERY sensitive both to centering the values (subtracting mean) and with the “standard deviation” of the input. It expects the range between -128 – 128 (ish)
  • LL2: The Evaluator class is needed so you don’t have to run the network twice (once for the loss and second for the gradients). If smb has a more elegant (pythonic) way of doing this please post!
  • LL3: Accidentally noted that the range of the initial random image does not matter much. Smaller values give smoother images and larger values (close to the input dynamic range) yields interesting images.

Huh, time for style transfer!

1 Like

why we never preprocessed our data when we used vgg16 in part 1 and now it is necessery?

why we needed to make k.function to loss and gradiant and then seperate them?
why couldnt we do 2 k.functions? one for the loss and one for the gradient

Sir,
is their any laptop available that have light at bottom of it. I mean it gives glowing look same as blue light under my car.best laptop in travel with perfect vlogging.

K.mean should be lower than K.sum because mean divides the sum by the number of elements. So the lower loss isn’t meaningful; it’s just scaled differently.

Worked for me too, thanks!

In the Super Resolution Network part of the neural-style.ipynb, it appears that a bunch of code has been added since the video. For instance, there is now a up_block defined that makes use of keras UpSampling2D() layer. The deconv_block is defined but no longer used in creating the up-sampled network. I cannot tell from keras’ documentation if the UpSampling2D() layer followed by Convolution2D() layers does the same thing as a Deconvolution2D() layer.

I think it is helpful for @jeremy to print out the keras version at the start of the notebook such that we will save time to debug due to keras version changes.

I’m getting an error generating the random image: ValueError: Floating point image RGB values must be in the 0…1 range.

If I change the range from -2.5, 2.5 to 0, 100 it works well, but not sure if I’ll get the same result as the original.

How long is it taking to run the first solve_image function? Here with a 1080ti it’s taking 5 mins.

It’s even better, as explained in Lesson 9, as I’m sure you’ve figured out by now :slight_smile:

1 Like

When you say

the first solve_image function

do you mean the 10 iterations? I have a 1080 Ti as well and it takes about 20 seconds tops. When you run

watch -n 1 nvidia-smi

what % GPU Util. are you seeing while running those 10 iterations? It should get above 90%.

As regards the random image, I couldn’t for the life of me figure out why it was -2.5 to 2.5, so I changed my function to the following:

rand_img = lambda shape: np.random.uniform(0, 255, shape)
x = rand_img(img_shape)

Don’t know if this helps at all, but let me know if you’re still running into issues

Thanks @corbin! It wasn’t using my gpu, I had to reinstall Tensorflow and not it’s taking about 8 seconds, way better.

The only thing that I’ve noticed is that the quality generated seems worst than with CPU.

Check out some images:

  • Content Extraction from block5_conv1, 10 iterations:
    res_at_iteration_9
  • Style extraction from Starry Night. 10 iterations:
    res_at_iteration_9
  • Result with 10 iterations:
    res_at_iteration_9

Any idea?

That looks very odd. Are you running the notebook exactly as provided? What version of keras are you using?

One thing you can try is to clone my implementation of Jeremy’s code and see if it works better, then we can use that to figure out what’s wrong. (Mine is modified for Keras 2.0.6, however)

Just skip down to the section titled “Bringing it all together”