Error in DCGAN code?

thunderingtyphoons · October 1, 2017, 6:54pm

Hi -

I am wondering if there is an error in the DCGAN code given here:

jph00/part2/blob/master/DCGAN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generative Adversarial Networks in Keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [

This file has been truncated. show original

The problem is in the train() function which is reproduced below:
def train(D, G, m, nb_epoch=5000, bs=128):
dl,gl=[],[]
for e in tqdm(range(nb_epoch)):
X,y = data_D(bs//2, G)
dl.append(D.train_on_batch(X,y))
make_trainable(D, False)
gl.append(m.train_on_batch(noise(bs), np.zeros([bs])))
make_trainable(D, True)
return dl,gl

When we train the generator, we freeze the weights of the Discrimator by calling make_trainable(D, False). make_trainable inturn sets the trainable property of the layers to False. But according to Keras documentation, this does not have any impact unless you recompile the model.
https://keras.io/getting-started/faq/#how-can-i-freeze-keras-layers

I created some standalone test code to test if the weights of the Discrimator change even when trainable is False. And it seems like it does change. The test code is here

gist.github.com

https://gist.github.com/anonymous/d0cb46b439fd01d59d27fc0bef50ec49

DCGAN.py

import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam



(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train.shape

This file has been truncated. show original

I print difference in norm of the weights and max weight difference and it seems to change a lot.

Epoch:11 Equals:False Weights:0.06788859516382217 Weights_after:0.06796538084745407 Norm Difference: -0.0003449287833063863 Wd:[0.00010568835, 0.00010551496, 0.00010599568, 0.00010225992, 0.00010524131, 9.7832213e-05]
Epoch:12 Equals:False Weights:0.06788820773363113 Weights_after:0.06797180324792862 Norm Difference: -0.000371000736777205 Wd:[0.00010656146, 0.00010630933, 0.0001069624, 0.00010348298, 0.00010639802, 9.7506672e-05]
Epoch:13 Equals:False Weights:0.0678870752453804 Weights_after:0.06797675043344498 Norm Difference: -0.00039942773582879454 Wd:[0.00010691024, 0.0001066145, 0.00010803342, 0.00010484131, 0.00010755658, 9.7170225e-05]
Epoch:14 Equals:False Weights:0.06788603216409683 Weights_after:0.06798047572374344 Norm Difference: -0.00042060539999511093 Wd:[0.00010801479, 0.00010782964, 0.00010912865, 0.00010589813, 0.00010856241, 9.6798307e-05]
Epoch:15 Equals:False Weights:0.06788482517004013 Weights_after:0.06798316538333893 Norm Difference: -0.00043875075061805546 Wd:[0.00010926649, 0.00010903378, 0.0001102984, 0.00010688975, 0.00010944903, 9.6418866e-05]
Epoch:16 Equals:False Weights:0.06788352131843567 Weights_after:0.06798528879880905 Norm Difference: -0.0004569145094137639 Wd:[0.00011052645, 0.00011032331, 0.00011154823, 0.00010765018, 0.0001100339, 9.603033e-05]
Epoch:17 Equals:False Weights:0.06788225471973419 Weights_after:0.06798683851957321 Norm Difference: -0.0004908795162918977 Wd:[0.00011127442, 0.00011120248, 0.00011280365, 0.00010826113, 0.00011083484, 9.5637639e-05]
Epoch:18 Equals:False Weights:0.06788089871406555 Weights_after:0.0679883137345314 Norm Difference: -0.0005122870206832886 Wd:[0.00011219084, 0.00011159436, 0.00011406932, 0.00010886264, 0.00011220947, 9.5237046e-05]