Part 2 Lesson 13 Wiki


(Sam) #191

@James_Ying,
It has been a while, so I do not remember but I think I changed definition of actn_loss2:

I changed the line out = V(sf.features)

to out = V(sf.features, requires_grad=True)

I model did not converge. see if it does for you


(James_Ying) #192

I found the reason.
Change one line in core.py in fastai/dl2

will solve
RuntimeError: element 0 of variables does not require grad and does not have a grad_fn

from

to


(Sam) #193

@James_Ying
When I make the change in my local core.py,
I get an error at
optimizer = optim.LBFGS([opt_img_v], lr=0.5)

ValueError: can’t optimize a non-leaf Tensor

instead in definition actn_loss2 I changed
line out = V(sf.features)
to out = torch.tensor(sf.features, requires_grad=True)

Now the model converges nicely

I also changed line in definition of style_loss
outs = [V(o.features) for o in sfs] to outs = [torch.tensor(o.features, requires_grad=True) for o in sfs]

line in definition of comb_loss
outs = [V(o.features) for o in sfs] to outs = [torch.tensor(o.features, requires_grad=True) for o in sfs]


(James_Ying) #194

Thanks for that.
run on comb_loss is no error
but run on before ones like 1 Style transfer 2 forward hook 3 Style match. will have errors

But, only opt_img_v need require_grad=True.
all the weights in every layers should be fixed.

There are three images
0 noise image
1 original image (content image)
2 style image
We need fix all the neural network, but update the noise image very step.
Let the noise image looks similar with the original image and style image.
Normal network is update weights, but now we update the 0 noise image.

outs is equal (input image and get the result on every block_ends layers)

How about your image looks like, can you show me your result?


(Sam) #195

The images look good…similar to what jeremy has in his notebook


(Phil Weslow) #196

I am having the same issue.

I have downloaded the data to '…/fastai/courses/dl2/data0/datasets/cyclegan/horse2zebra'. Since that did not work, I also placed a copy of /data0/datasets/... in my home directory.

Can’t seem to figure out what’s causing the hang-up.

Update:

The problem was caused by the fact that I was using a Docker container that did not have access to my entire file system. Here is my solution.


(Phil Weslow) #197

I would absolutely love to check out your notebook! Glad you’re working on the project, it’s another gem in the “Fast.ai Image Manipulation Canon”.


(Anand Pawara) #198

I am getting error ‘cuda runtime error (10) : invalid device ordinal at torch/csrc/cuda/Module.cpp:88’
I am running style transfer file and i am getting this error on line - torch.cuda.set_device(3)
If i set to torch.cuda.set_device(0) then i get same error on line- targ_t = m_vgg(VV(img_tfm[None]))
need help


(Theodoros Galanos) #199

Just bumping this in case there’s a notebook out there for us to try!

Thanks!

Regards,
Theodore.


#200

Yes, there is a blog post and a notebook now.


(Christian Tutivén Gálvez) #201

‘cuda runtime error (10) : invalid device ordinal at torch/csrc/cuda/Module.cpp:88’
I have the same problem


(Theodoros Galanos) #202

Thanks Sylvain much appreciated!


(Marc) #203

Jeremy mentions multiple times that “it takes 5 seconds” and the image ist generated.
Are those 5 seconds a figure of speech or ist that the real time it takes on his nvidia 1080ti?
I have 2 measurements here for the final style transfer optimizer (1000 iterations of comb_loss)

  • GTX 1050 w/ 4GB RAM: 96 seconds (Laptop GPU)
  • GTX 1060 w/ 6GB RAM: 55 seconds

So is the 1080ti actually 11 times faster then the 1060?
(I am aware the RAM-size is irrelevant here, but I put it there to show that those are “real” cards, both GPUs exist as cheaper versions with smaller RAM sizes (and I assume downgraded performance) )


(Stas Bekman) #204

comment out the line at the top of the notebook (or set it to 0 if you have just one GPU):

#torch.cuda.set_device(3)

Note, it’s the same issue in almost all dl2 notebooks.


(Stas Bekman) #205

Looks like a missing cell in the notebook. Should be fixed by this PR.