Part 2 Lesson 13 Wiki


(Satender S Saroha) #171

Hi,
I have finished re-watching the lesson 13 video and was working through the jupyter notebook “style-transfer.ipynb” discused in last class. I am getting an error towards the end during execution of iterations on optimizer in style transfer

n_iter=0
while n_iter <= max_iter: optimizer.step(partial(step,comb_loss))

here is the error:
“RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.”

Has anybody else run into this issue?

looking on google seems to indicate an issue with “backward” method in ~/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/optim/lbfgs.py

"loss.backward() is trying to back-propagate all the way through to the start of time, which works for the first batch but not for the second because the graph for the first batch has been discarded. "

Any pointers on fixing it will be super helpful.

thanks!


(Satender S Saroha) #172

If I modify the step method implementation to change the following:
loss.backward()
with
loss.backward(retain_graph=True)
there is no runtime error but the loss never comes down across the iterations:
n_iter=0
while n_iter <= max_iter: optimizer.step(partial(step,comb_loss))

Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625
Iteration: n_iter, loss: 7216.890625

still trying to figure out best way to resolve this.


#173

Been trying to implement the Deep painterly harmonization paper in pytorch and it has been a real pain :frowning: It doesn’t give a lot of details concerning the implementation, but more than that, they are bits in the three algorithms they explain that are done completely differently in code.

I think I have their stage 1 right, even if I’m getting better result than them on this one, so I may have done something differently. I’m not quite where they are on stage 2 but still, here are some results:

Will share the notebook next week when I’ve figured out how to get the top of my Eiffel tower back :wink:


(Arvind Nagaraj) #174

The under-construction version still looks fantastic!


(Jason McGhee) #175

I tested out the original notebook as well, to play with hyperparameters etc. One thing that worked for me was not closing the forward hook after the style transfer section.


(Igor Kasianenko) #176

Hi, I have two questions:

  1. Why not to use target image instead of randomly initialized image? In this case loss will be less from the first step. I found this idea was discussed in last years course Lesson 9.
  2. How to run style transfer for batch of images? Every image will have own loss function, so it’s confusing, how to iterate in this case.

(Bart Fish) #177

Jeremy with Style
source


(Satender S Saroha) #178

thanks @jsonm ! not closing the forward hook did the trick.


(blake west) #179

Yeah, just gonna say, their code is actually really great. Not surprised, cause it comes from engineers, and not academics. But looking at the models they wrote and the way the did the training is really interesting: https://github.com/tkarras/progressive_growing_of_gans/blob/master/networks.py . I may try to re-implement in PyTorch!


(Igor Kasianenko) #180

Hi Alena @alenas,
How did you get this great ratio between style and structure? I’m trying to play around and make full set of parameters that effect the result. So far using all as in notebook I get results that inherit too much style and not enough structure.
For example input image


becomes almost unrecognizable after style transfer.

I’d like to make exploration and write python script to run it with range of parameters to find out what works best for different style images.


(Bart Fish) #181

The Structure is really hard, I think it’s because in this incarnation, the gradients are not learning structure, we’re just doing a bunch of convs. What is learned seems to me to be color and broad shape variations, just my two cents.


(Bart Fish) #182

Check out the cover ofJim Gaffigan’s new comedy albumn…Style Transfer


(Ravi Sekar Vijayakumar) #183

I have a naive question. I tried giving the content as input image instead of starting with random noisy image and could still see it getting stylized. What is the intent of having random noisy image as input to begin with ?


(Jeremy Howard) #184

It’s an excellent question! In fact, a followup paper tried exactly what you did, and found it worked better :smiley:


(Ravi Sekar Vijayakumar) #185

Thanks Jeremy. I am yet to look up papers on style transfer but how does one compare results of this task though ? It’s like comparing art pieces.


(Jeremy Howard) #186

Human perception, I believe :slight_smile:


(Sam) #187

@jsonm, @satender, @jeremy
I encountered a different error in style-transfer.ipynb.
This maybe because of pytorch 0.4
At cell after definition of actn_loss2, I got an error RuntimeError: element 0 of variables does not require grad and does not have a grad_fn
I overcame that error by modifying the definition of actn_loss2 as:
out = V(sf.features, requires_grad=True)
Now I see that the loss does not reduce:

Iteration: n_iter, loss: 4.30463981628418
Iteration: n_iter, loss: 4.30463981628418
Iteration: n_iter, loss: 4.30463981628418
Iteration: n_iter, loss: 4.30463981628418

If you have upgraded to pytorch 0.4, can you please rerun the notebook for me and confirm my observation?

Any suggestions to fix the code are welcome


(Ralph Brooks) #188

Thanks @jsonm . I had the same problem. Not closing the forward hook at the end of the “Style Match” section fixed the problem.


#189

Hello, everybody!
I download horse2zebra.zip file to …/fastai/courses/dl2/data0/datasets/cyclegan/horse2zebra and I get error ‘%s is not a valid directory’. I work on paperspace. I have explored all the code and I can’t find any answer. Could you help to figure out this pls?


(James_Ying) #190

I am 0.4 too, got The same problem. Did you figure it out?