Lesson 10 Discussion

(Kent) #41

In order to make it actually using the preproc, this is what I did:

vgg = VGG16(include_top=False)
preproc_inp =Lambda(preproc, name=‘preproc_inp’) (vgg_inp)
preproc_outp = Lambda(preproc, name=‘preproc_outp’) (outp)
vgg_content = Model (vgg.input, [get_outp(vgg, o) for o in [1,2,3]])
vgg1 = vgg_content(preproc_input)
vgg2 = vgg_content(preproc_outp)

I had to read the Keras source codes in order to understand why the above code works. Essentially when calling the model with the preproc tensor, Keras creates a new node connecting it to the model (by replacing the default inputs and recalculating the outputs)

However, The result is more or less the same with or without preproc. That also made me wonder why … :innocent:

(David Woo) #42

I’m going through the DCGAN notebook.

I get the follow error when i run the MLP GAN section. I think i need to define m as function of the discriminator and generator but was not sure how. Anyone have any advice?

----> 8         gl.append(m.train_on_batch(noise(bs), np.zeros([bs])))
      9         make_trainable(D, True)
     10     return dl,gl

AttributeError: 'int' object has no attribute 'train_on_batch'

(Thundering Typhoons) #43

As explained by @Matthew at the top of this thread…

(David Woo) #44

Thanks for this, very helpful! works now

(Kent) #45

I am encountering a word2vec error when executing the imagenet notebook at this point:

model = word2vec.Word2Vec.load_word2vec_format(w2v_path+’.bin’, binary=True)

Here is the error I got:

DeprecationWarning Traceback (most recent call last)
in ()
1 model = word2vec.Word2Vec.load_word2vec_format(w2v_path+’.bin’, binary=True)
2 model.save_word2vec_format(w2v_path+’.txt’, binary=False)

anaconda3/lib/python3.6/site-packages/gensim/models/word2vec.py in load_word2vec_format(cls, fname, fvocab, binary, encoding, unicode_errors, limit, datatype)
1305 limit=None, datatype=REAL):
1306 “”“Deprecated. Use gensim.models.KeyedVectors.load_word2vec_format instead.”""
-> 1307 raise DeprecationWarning(“Deprecated. Use gensim.models.KeyedVectors.load_word2vec_format instead.”)
1309 def save_word2vec_format(self, fname, fvocab=None, binary=False):

DeprecationWarning: Deprecated. Use gensim.models.KeyedVectors.load_word2vec_format instead.

It no longer moves on after the warning message. Does anybody know why?

(layla.tadjpour) #46

I am trying to undestand the imagenet_process notebook ( i can not run it on my machine though) and I can’t understand why @jeremy concatenate two label arrays (“vecs”) in the “Create Model” section of the notebook. Does any body know the answer? Thanks.

v = np.concatenate([vecs, vecs])
lm.evaluate(features_last, v, verbose=0)
vecs = np.stack([syn2wv[l] for l in labels]); vecs.shape

(David Woo) #47

Hi recently installed keras 2 and realized mode in batch normalization is no longer available


was wondering if there was an intuitive explanation of what mode does?

From the link below mode seems to specify if more than one input is used in the model. what i gather this might be sharing layers or filters across inputs.

(Kent) #48

@kelvin I am using your method to generate the classids.txt file. The number of entries I got is exactly the same as what you posted. However I got an error later at this line:

vecs = np.stack([syn2wv[l] for l in labels]); vecs.shape

The error is:

ipython-input-131-3265cbead269 in listcomp(.0)
----> 1 vecs = np.stack([syn2wv[l] for l in labels]); vecs.shape
KeyError: ‘n01682714’

That’s because the classids.txt contain words like ‘american-chameleon <-> n01682714’ that are not found in GoogleNews-vectors-negative300.bin.gz (i.e. ‘n01682714’ is mapped to ‘american-chameleon’, but the word ‘american-chameleon’ does not exist in syn2wv, lc_w2v, or w2v_dict).

Did I miss anything here? Or is there a preprocessed classids.txt file that I can download?


(kelvin) #49

If you read through the notebook, jeremy elides all the training images w/ categories that couldn’t be matched.

(kelvin) #50

it’s because the training set is doubled. i.e. horizontally flipped. so the validation set is the concatenation of the same vectors.

(layla.tadjpour) #51

why is doubled? Did we do data augmentation or some thing?

(kelvin) #52

yes it is augmented by flipping it horizontally.

(Constantin) #53

you need to go something like m=Model([generator, discriminator]) and pass that to train()

(Kent) #54

Thanks Kelvin. A few lines were commented out in the notebook so I did not notice them. I moved away the files with categories not found, cutting down number of images from 1.2 million to 990k. Now I am able to continue running the notebook.

(rajendra koppula) #55

I implemented SR transfer network using MSE. I formulated yhat as flattened vector of feature maps. This allowed me to: 1) remove custom Lambda layer that computes MSE 2) pass MSE as the loss function instead of MAE 3) remove the targ=0 hack.

I trained the model on MS COCO dataset (80K images). The results compare well with Jeremy’s implementation (at least on the training set). My notebook is here if you want to look at the details.

However, I do see terrible performance when the model is used on test images. I am curious to know if anyone used their SR models on test images. Images below (also found in the notebook) show what I mean.

What could be the cause of poor test set performance?

  1. The number of images/object categories in MS COCO dataset is not sufficient?
  2. Image artifacts?
  3. Need to train more epochs?
  4. Other possibilities ?

From Training set:

  • LR input

  • Original HR image

  • Generated HR image

From Test set

  • LR input

  • Original HR image

  • Generate HR image

(Mariya) #56

I would love this too in order to follow along. Haven’t built my deep learning box yet and AWS P2 failed to properly download LSUN dataset for the wgan code which is much smaller :frowning:

(sravya8) #57

I see similar results. Results are usually pretty dark and look bad. Have not spent much time figuring out. One thing I was going to try is train on center cropped data rather than black padded training data.

(rajendra koppula) #58

I used center cropping. So looks like different issues. For your issue @sravya8, retraining with center cropping makes sense… Anyone have weights from Jeremy’s training (on imagenet) ? The stanford paper doesn’t mention if the results they show are from test or training sets. So we haven’t really seen anyone apply SR to test images successfully…

(sravya8) #59

I think we are seeing the same issue. Then, it’s probably not due to center cropping. Here are my results btw if you want to take a look: https://github.com/sravya8/DL/blob/master/Super%20resolution/SuperResolution.ipynb

(Xinxin) #60

+1. experienced similar difficulties in LSUN dataset with AWS P2