Could someone please direct me to a lecture (if any) that discusses English text generation? Was working through the course sequentially, but now a professor has recruited me for a research project and am on a short deadline.
Jeremy discusses text generation a little bit in lesson 4 starting around the 18 minute mark. However, the takeaway from the lesson is not to generate text but to leverage transfer learning from a text corpus (wikipedia) to get better results in classifying movie review sentiments as positive or negative.
Out of interest, you might see OpenAI’s recent result in text generation: https://blog.openai.com/better-language-models/
Thanks @Tom2718. I am working on a problem that involves (1) text generation from a network pretrained on a corpus and (2) replacing the loss function of the network with a custom loss function. I checked out the OpenAI model, but it does not allow us to do the latter. textgenrnn does not allow this either. Lesson 6 in fastai v2 does discuss text generation and allows us to replace the loss function. I’m guessing they removed this from v3?
1 Like