And here is a link to the paper about perceptual loss and real-time style transfer.
L1 loss is the mean of the absolute value of your outputs - targets. The f1 score is veeeeeeeeeeery different.
So yes.
3 Likes
Makes sense. Image similarity with generative model!
Very impressive model !
I had to update fastai specifying version, I don’t know why only let me do it with 1.0.34 and not any more recent one. “conda install -c fastai fastai==1.0.34”. It works now and recognize plot_multi
There was an issue with dependency conflicts in pypi fastai-1.0.35+36, if you still have a problem in 1.0.37 which was just released please post your report to Fastai v1 install issues thread thanks.
2 Likes
GAN on NLP - No.
But generating using gated conv or RNN - we can use to augment data.
One popular tech I remember from Kaggle on NLP augment. Is translation. Translate English-French. Then French-Original Language(English) - Add that set to augment.
how do Unet and GAN relate with each other…
may be very basic q…
1 Like
can u explain more on the intuition for using the same weights for hidden layers? can understand for embedding (green lines) but not the orange ones!
My understanding is that Unet is a kind of architecture that output an image from an image, and a GAN is the process of having 2 networks that go back and forth together : one to generate images (like Unet), and the other to discrimate those images (are they generated or real ?)
I’m a bit lost on the lesson7-human-numbers section: http://nbviewer.jupyter.org/github/fastai/course-v3/blob/master/nbs/dl1/lesson7-human-numbers.ipynb - what is the goal with this model? What are we trying to achieve?
Word prediction
This s a language model, so we’re trying to predict what the next word is, having read all the previous ones.
We are predicting the next number in a sequence where numbers are represented by words.
This is correct. A unet outputs an image from an image, and a GAN has a generator and a discriminator that go back and forth. In the superres gan notebook Jeremy showed, the generator for the GAN was a unet.
3 Likes
I need to point out- one of the pieces of advice there may be outdated. It says " 11: Dont balance loss via statistics (unless you have a good reason to)". We’re actually doing that exact thing with this line:
switcher = partial(AdaptiveGANSwitcher, critic_thresh=0.65)
That switcher is using a simple heuristic- keep training the critic until it’s firmly “ahead” of the generator by making sure its loss is sufficiently low. This is all pretty new (like…last two weeks) but it appears this is helpful for pretrained networks in particular because it helps to bring the critic in balance with the generator (important- because you don’t want a critic not knowing what it’s doing!)
9 Likes
For the orange ones, the idea is that going from the nth word to the (n+1)th word is the same (regardless of n).
so we use Unet for GAN ?? bu our objective is to make image to look as good like hi res image by resolving the crapiness in the image
but shudnt we learn something new from the each new sequence of word embedding compared to previous ones!! by having same weights, arent we forcing to learn same features from every set of word embedding!? or am i missing something?
I think the main problem I’m having here is that predicting the next word against this data set of english words for numbers is a bit dull - I’d find this section a lot more compelling if it was working against a more interesting word corpus (like multi-word movie names or something)