Text generation with GAN

Hi.

I’ve seen vision.gan which is “All the modules and callbacks necessary to train a GAN” at https://docs.fast.ai/vision.gan.html

Is there a similar facility for text GAN? I understand that it’s less mature than visual GAN, but still, it’d be nice to get even partial results.

2 Likes

I’m also very interested in this.

It’s not exactly a GAN for text generation but maybe you’ll find ELECTRA interesting. It follows a similar idea since it’s using generator and discriminator models for language model pre-training.

@Richard-Wang recently did some amazing work reimplementing ELECTRA from scratch. He probably can say more about it. See the thread below:

1 Like

Yes, as @stefan-ai said, ELECTRA is actually not a GAN (b/c it doesn’t backprop through discriminator and generator), but it is inspired by GAN, and successfully applies GAN-like “idea” on NLP pretraining.

The authors have also mention the difficulty of applying GAN to text by citing Language GANs Falling Short. Although I haven’t found time to read it :joy:, but it should be related to your question.

1 Like