Cycle GAN Art -- completing visual loop :-)

(Alena Harley) #1

Hi All,

Here are some of my experiments with unpaired image-to-image translation using Cycle GAN models:

Experiment 1 Photo portrait to Artistic sketch:
(left - original photo, middle - artistic sketch generated, right - reconstructed original)

(left - original sketch, middle - “photo realistic” portrait generated, right - reconstructed original)

Experiment 2 Landscape photo to Stained glass art:

Experiment 3 Landscape photo to Watercolor painting:

After listening to Jeremy’s lecture 13 on style transfer it is interesting to think that cycle GAN can be used for the same purpose end-to-end with a bit more training data required, still need to evaluate and study mode collapse in those experiments. Thank you Jeremy for another awesome lecture!! @jeremy

Thanks!
Alena

40 Likes

Part 2 Lesson 12 wiki
(Arvind Nagaraj) #2

Amazing! Love it!!
Could you please share more details on dataets, training, etc.?

0 Likes

(Alena Harley) #3

Hi Arvind,

datasets were scraped from google image searches (using a script). I also added a few photos of my kids for fun :slight_smile:

Total datasets are small ~400-500 images in each direction. I modified multiplicative factor of cyclic loss to assign more importance to cyclic loss rather than to the discrimination loss.

I also experimented with dropout, and number of ResNet blocks, mostly to learn the code details. Trained for 200 epochs (~12 hrs on aws p3 instance).

Thanks!
Alena

9 Likes

(Arvind Nagaraj) #4

Nice!

0 Likes

(Yesu Feng) #5

Hi Alena,

Really impressive! do you add any constraint when you scrape images from google, for example, add ‘portrait’ to artistic sketch to confine the images to be portrait and also confine the photos to be only portrait?

Yesu

0 Likes

(Rachel Thomas) #6

Hi Alena, this is great! I hope you write a blog post about your work (and please post it here if you do)

3 Likes