do let me know if you get the Style extraction part working in Keras 2.0. I am struggling with this, can’t see what’s my undoing
Will do! Are you getting errors?
Nope. The loss function drops from 10,000 to 20 over10 iterations. But the random image input still looks random, can’t see the style being extracted
Thank you!
Hi,
You can use VGG-16 directly from Keras.
Here is an example on how to use it with tensorflow
Please let me know if you have any questions,
Yannet
Note: the complete collection of Part 2 video timelines is available in a single thread for keyword search.
Part 2: complete collection of video timelines
I did a timeline of the Lesson 8 video as I found them very practical in Part 1 wiki.
There are many links, probably there was a lot of “happy noise” in the first class
I expect future timelines to be shorter.
Lesson 8 video timeline:
0:00 : Intro and review of Part 1
08:00 : moving to Python 3
10:30 : moving to Tensorflow and TF Dev Summit videos
22:15 : moving to PyTorch
27:30 : from Part 1 “best practices” to Part 2 “new directions”
31:40 : time to build your own box
36:20 : time to start reading papers
39:30 : time to start writing about your work in this course
41:30 : what we’ll study in Part 2
40:40 : artistic style (or neural style) transfer
52:10 : neural style notebook
54:15 : Mendeley Desktop
56:15 : arXiv-Sanity.com
59:00 : Jeremy on twitter.com and reddit.com/r/MachineLearning/
1:01:15 : neural style notebook (continued)
1:04:05 : broadcasting, APL as “A Programming Language”, and Jsoftware
1:07:15 : broadcasting with Keras
1:12:00 : recreate input with a VGG model
1:22:45 : optimize the loss function with a deterministic approach
1:33:25 : visualize the iterations through a short video
1:37:30 : recreate a style
1:44:05 : transfer a style
- https://youtu.be/cRjPVN3oo4s?t=1h44m5s[details=Summary]This text will be hidden[/details]
Hi all! Just wanted to point out few issues that I had with first part of lesson 8.
Task: Get the bird out of noise, using only tensorflow 1.2 and its contrib.keras without keras .That is, content transfer. (afaik from TF Dev videos, standalone keras will be deprecated some time in the future).
I had to replace the inports on vgg_16_average.py, and handcode the loss keras.backend.mean(keras.backend.square(content_model.output - content_target))
. Other than that, most of the “porting” is straight forward and/or easily solved with a bit of doc surfing.
However, I fought a lot with a nasty bug (feature) in scikit-image. I used skimage.transform.rescale()
to get my image size down. Default, this function also scales your image to [0,1]. VERY nasty feature, nothing worked. I got my MSE down but the out image is pure noise. It took me a while to track down this issue. So, use preserve_range=True
parameter.
Lessons learned:
- LL1: VGG is VERY sensitive both to centering the values (subtracting mean) and with the “standard deviation” of the input. It expects the range between -128 – 128 (ish)
- LL2: The
Evaluator
class is needed so you don’t have to run the network twice (once for the loss and second for the gradients). If smb has a more elegant (pythonic) way of doing this please post! - LL3: Accidentally noted that the range of the initial random image does not matter much. Smaller values give smoother images and larger values (close to the input dynamic range) yields interesting images.
Huh, time for style transfer!
why we never preprocessed our data when we used vgg16 in part 1 and now it is necessery?
why we needed to make k.function to loss and gradiant and then seperate them?
why couldnt we do 2 k.functions? one for the loss and one for the gradient
Sir,
is their any laptop available that have light at bottom of it. I mean it gives glowing look same as blue light under my car.best laptop in travel with perfect vlogging.
K.mean
should be lower than K.sum
because mean divides the sum by the number of elements. So the lower loss isn’t meaningful; it’s just scaled differently.
Worked for me too, thanks!
In the Super Resolution Network part of the neural-style.ipynb, it appears that a bunch of code has been added since the video. For instance, there is now a up_block
defined that makes use of keras UpSampling2D()
layer. The deconv_block
is defined but no longer used in creating the up-sampled network. I cannot tell from keras’ documentation if the UpSampling2D()
layer followed by Convolution2D()
layers does the same thing as a Deconvolution2D()
layer.
I think it is helpful for @jeremy to print out the keras version at the start of the notebook such that we will save time to debug due to keras version changes.
I’m getting an error generating the random image: ValueError: Floating point image RGB values must be in the 0…1 range.
If I change the range from -2.5, 2.5
to 0, 100
it works well, but not sure if I’ll get the same result as the original.
How long is it taking to run the first solve_image function? Here with a 1080ti it’s taking 5 mins.
It’s even better, as explained in Lesson 9, as I’m sure you’ve figured out by now
When you say
the first solve_image function
do you mean the 10 iterations? I have a 1080 Ti as well and it takes about 20 seconds tops. When you run
watch -n 1 nvidia-smi
what % GPU Util. are you seeing while running those 10 iterations? It should get above 90%.
As regards the random image, I couldn’t for the life of me figure out why it was -2.5 to 2.5, so I changed my function to the following:
rand_img = lambda shape: np.random.uniform(0, 255, shape)
x = rand_img(img_shape)
Don’t know if this helps at all, but let me know if you’re still running into issues
Thanks @corbin! It wasn’t using my gpu, I had to reinstall Tensorflow and not it’s taking about 8 seconds, way better.
The only thing that I’ve noticed is that the quality generated seems worst than with CPU.
Check out some images:
- Content Extraction from block5_conv1, 10 iterations:
- Style extraction from Starry Night. 10 iterations:
- Result with 10 iterations:
Any idea?
That looks very odd. Are you running the notebook exactly as provided? What version of keras are you using?
One thing you can try is to clone my implementation of Jeremy’s code and see if it works better, then we can use that to figure out what’s wrong. (Mine is modified for Keras 2.0.6, however)
Just skip down to the section titled “Bringing it all together”