Improve Neural Style Tranfer - Lesson 13

[James_Ying]

Tricks to improve Neural Style Tranfer

1. Change MaxPool to AvgPool
   

2. noise_image = noise_image + content_image
   
   you can play with noise_ratio, default is 0.6.
   Normally use 0-1, But you can try 100 or -100.

3. don’t use relu layer, use conv layer
   if relu layers are [ 5 12 22 32 42]
   conv layers are [ 3 10 20 30 40]
   

4. each conv layer multiple different scalar.
   each layer not the same weight - [1, 1, 1, 1, 1]
   but different weights - [5,4,3,2,1]
   
   

   

5. change the style_rate higher can make result more abstrach.
   

6. Don’t use
   opt_img = scipy.ndimage.filters.median_filter(opt_img, [8,8,1])

My daughter’s printing

[radek]

I might be wrong, but this seems to be making a significant difference? If so, maybe this could be added to the lesson 13 notebook, maybe even as a sidenote at the bottom if not inline?

Really nice looking results

[James_Ying]

I try and add many other tricks, but these two are most effective.
I will post more details here.

[Even (Even Oldridge)]

These are great tips, thanks for sharing!

Something that might work well with the images you use (high frequency, complex patterns) that I explored is to dramatically increase the ratio between style loss and content loss. I found a really interesting result when I went up to 1000:1. What happens is that the network takes much longer to train and converge, but I thought the results were much better looking.

I posted it on the fora in the 2017 course. Here’s a link to the results and a corresponding explanation.

Lesson 8 homework assignments

After significant experimentation I found much more interesting results when I changed the ratio of style loss to content loss by a few orders of magnitude and allowed the style starting point to slowly converge to the content image. As an example the style layer below (my own artwork): [image] produces results that I would say match the palette but not the style of the image: [image] But when I modified the relative weighting of the content to be /2000 rather than /50 the results are (at …

It’s something I meant to explore more deeply, but haven’t had the chance.

[James_Ying]

In my code if more than 100:1, the image will be ruined.

[Even (Even Oldridge)]

Have you tried training for longer? If you look at the thread, within the first 10 epochs you don’t really see the content, it only really starts to appear more slowly after that point. It takes ~75 epochs to get a good result. That’s way higher than the more balanced, which looks somewhat good even after a single epoch and generally converges in 5-10 epochs.

[James_Ying]

I usually train at least 600 epochs, the loss will lower than 4000, depend on the style image, sometimes the loss was high, but looks good.
and for some style images I will train for 2000 epoch.