Building a simple Generative Adversarial Network (GAN) using TensorFlow

tutorial

#21

Hello Aadil

Thank you very much for your great help. yes what you understand is completely right
Could you please let me know after i training the model, how can i do the test.
for example if i want to generate row vector of 1000 column giving one score(number between 0 and 1), what I should write ?. or where i can find the generated data . I mean in case of condtional gan , how i can give condtion and generate data. I really appreciate your help. Thanks


#22

Hi Saida,
At the test time you just need the output of the Generator. So just sample the random noise vector, combine it with the score and run the Generator output for one iteration.

Thanks,
Aadil


#23

Thank you very mych Aadil, Yes it works thank you I would like to aks if my generator

like
def generator(Z, y, hsize=[16, 16, 16], reuse=False):
with tf.variable_scope(“GAN/Generator”,reuse=reuse):
inputs = tf.concat(axis=1, values=[Z, y])
h1 = tf.layers.dense(inputs,hsize[0],activation=tf.nn.relu)
h2 = tf.layers.dense(h1,hsize[1],activation=tf.nn.log_softmax)
h3 = tf.layers.dense(h2, hsize[2], activation=tf.nn.sigmoid)
out = tf.layers.dense(h3,1000)
return out
I need the generated vector(1000 column) to be binary. Do you have an suggestion?


#24

Thank you for a great tutorial. Really helped me get a strong understanding and knowledge of how to code the GAN.

My question is regarding the next step: i.e. after training GAN, how we can use it to generate synthetic data samples from a given data set. For instance, if we have a numeric data frame of 200,000 rows and 8 features, how can we use the trained GAN to create new synthetic data samples? Can we use same data for training GAN and then for new data generation process ?

Thanks,
Umaie


#25

Hi Saida,
Sorry for the late reply.
There is a very general trick when it comes to generating binary vectors using neural networks, you just add the sigmoid activation and round the outputs of the activation. It is same as what we do for classification problems.

Thanks,
Aadil


#26

Hi Umair,
Thank you for the support.
As I have explained in my reply to Saida, at test time you just need the output of the trained Generator network which just requires random noise as the input. The output of the generator network is the synthetic data that you require.

Thanks,
Aadil


#27

Dear Aadil -

First of all, thank you for this beautiful tutorial. Pardon me for being dumb… I don’t understand some pieces of code very well. In particular it is the following lines of code which I am curious about :

G_sample = generator(Z)
r_logits, r_rep = discriminator(X)
f_logits, g_rep = discriminator(G_sample,reuse=True)

disc_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=r_logits,labels=tf.ones_like(r_logits)) + tf.nn.sigmoid_cross_entropy_with_logits(logits=f_logits,labels=tf.zeros_like(f_logits)))
gen_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=f_logits,labels=tf.ones_like(f_logits)))

Question :
1.) What are these variables r_logits, r_rep, f_logits and g_rep ? I can understand that they are outputs of the discriminator. But what really are these outputs ? How can they be conceptually related to Goodfellow’s paper on GAN ?
2.) Until reading this post, I was of the understanding that the discriminator does not process the true data (G_sample in your case). I believed that it only cross verifies the generated data with true data with some sort of cross entropy error. But here I see, you are also making the discriminator process the true data… Why should we do this ?
3.) What are we actually doing in the loss functions disc_loss and gen_loss ? I understand that GAN’s have two losses (discriminator loss and generator cross-entropy loss). I do not understand how does that relate to this code !

I am just beginning to understand neural networks. Please forgive my dumbness. :slight_smile:


#28

Hi Aadil, Thanks a lot for your response. I understand the concept, but since I am relatively new to the tf framework, I am having some problem getting the test data to execute on the trained generator.

So Let me explain it in a few sentences. I am using your program with required changes to train it on my 8 dimensional (8 features / columns) numeric data set. The GAN seems to train quite well. Now for the test phase, first I am generating random noise with the following commands:

Z_test_batch = sample_Z(100, 8) – this generates 100 samples of 8 dimensional test data set.

After that, I pass it through the trained generator function. This is the step which I think I am doing incorrectly.

G_Test_Batch=generator(Z_test_batch)

The error received is: “AttributeError: ‘tuple’ object has no attribute ‘ndims’”.

I need your guidance in how to change the test data set to work with the generator. And secondly, if just calling the generator function like I am doing will actually correspond to the trained GAN Network?

Thanks in advance,
Umair


#29

Hi Umair,
I think you have confused the tensorflow operations with python functions.
The generator function given in the code return the tensorflow graph node for the last layer.
To get the output you need to run this node by feeding the placeholder for random noise.
Please refer to the following line in the code I have shared:

Thanks,
Aadil


#30

Hi Arun,
Thank you for your support. Please find the answers below and feel free to ask if anything is unclear.

If you notice the discriminator function it returns 2 things: firstly the output of the discriminator which is the logit output for the given input and secondly the activation of the last layer of the discriminator. To get the different output for the true dataset and generated samples we create different tensorflow data flow nodes. r_logits and f_logits are the logits output for the true data and generated data respectively. r_rep and g_rep represent the activation of last layer of the discriminator for the real data and generated data respectively. The representation variables have nothing to do with GANs training they are just used to visualize what is the transformation that the discriminator is learning to discriminate between two types of data.

I don’t think it is true that discriminator does not process the true data. A discriminator is just like any other neural network based binary classifier it needs sample from both the classes (real and generated data in our case) to learn the spatial transformation that separates the two classes.

disc_loss is just the cross entropy loss for the discriminator to correctly classify the true data samples as 1 and fake data samples as 0 while gen_loss is the cross entropy loss for the discriminator to classify fake data as 1. It is exactly same as the GANs losses. The discriminator loss is trying to minimize the classification error of the discriminator and generator loss is trying to maximize the classification error of the discriminator.

I hope that was helpful. Let me know if you have any further questions.

Thanks,
Aadil


#31

Hi Aadil,

I read GAN is widely used to generate classification related samples like images. But shall we apply the same GAN techniques to generate regression samples?

For example, I am using DNN for a regression problem. I have 8 input features and 1 target (which is continuous valued label). I just modified your GAN code and converted it into LSGAN. If I generate fake data using this LSGAN by feeding all the 9 features (8+1), will I get reliable result?

The question arisen in my mind after reading the conference paper (https://openreview.net/pdf/90f0e675457dac151517a5ed5c0f15d856366494.pdf), where author proposes Reg-GAN to generate samples for regression problems. Can’t we use existing DCGAN or LSGAN for regression problems?