Improve Neural Style Tranfer - Lesson 13

Tricks to improve Neural Style Tranfer

  1. Change MaxPool to AvgPool

  2. noise_image = noise_image + content_image
    you can play with noise_ratio, default is 0.6.
    Normally use 0-1, But you can try 100 or -100.

  3. don’t use relu layer, use conv layer
    if relu layers are [ 5 12 22 32 42]
    conv layers are [ 3 10 20 30 40]

  4. each conv layer multiple different scalar.
    each layer not the same weight - [1, 1, 1, 1, 1]
    but different weights - [5,4,3,2,1]

  5. change the style_rate higher can make result more abstrach.

  6. Don’t use
    opt_img = scipy.ndimage.filters.median_filter(opt_img, [8,8,1])

My daughter’s printing


I might be wrong, but this seems to be making a significant difference? If so, maybe this could be added to the lesson 13 notebook, maybe even as a sidenote at the bottom if not inline?

Really nice looking results :slight_smile:

I try and add many other tricks, but these two are most effective.
I will post more details here.

1 Like

These are great tips, thanks for sharing!

Something that might work well with the images you use (high frequency, complex patterns) that I explored is to dramatically increase the ratio between style loss and content loss. I found a really interesting result when I went up to 1000:1. What happens is that the network takes much longer to train and converge, but I thought the results were much better looking.

I posted it on the fora in the 2017 course. Here’s a link to the results and a corresponding explanation.

It’s something I meant to explore more deeply, but haven’t had the chance.

1 Like

In my code if more than 100:1, the image will be ruined.

Have you tried training for longer? If you look at the thread, within the first 10 epochs you don’t really see the content, it only really starts to appear more slowly after that point. It takes ~75 epochs to get a good result. That’s way higher than the more balanced, which looks somewhat good even after a single epoch and generally converges in 5-10 epochs.

I usually train at least 600 epochs, the loss will lower than 4000, depend on the style image, sometimes the loss was high, but looks good.
and for some style images I will train for 2000 epoch.