Building a simple Generative Adversarial Network (GAN) using TensorFlow

tutorial

#1

Generative Adversarial Networks or GANs are one of the most active areas in deep learning research and development due to their incredible ability to generate synthetic results. In this blog, we will build out the basic intuition of GANs through a concrete example. This post is broken down in following way:


This is a companion discussion topic for the original entry at https://blog.paperspace.com/implementing-gans-in-tensorflow

#2

Thank you so much for this detailed explanation of the GAN code. All the previous blog post on GANs have been good at explaining the idea but very hand-wavy at the actual code, which makes it very hard to apply it to your own problem. Quick question: is the loss function used in this post the definition of a GAN, or can it be modified to possibly strengthen a GAN? Which attributes of a GAN’s code can be changed when the GAN doesn’t result in satisfactory output?


#3

Hi Winyl,
Thank you for your support.
The loss function used here can be modified in various ways depending on the use case. We might cover that in future. To get the satisfactory results you can modify a lot of parts depending on the reasons:

  • Complexity (# of layers and/or # of units) of the neural network if the data-set is more complex.
  • Changing how many iterations of generator should be run for every iteration of discriminator (This is not advised although it works in some specific use cases)
  • Changing the learning rate and optimizer.
  • Adding noise to the inputs
  • Changing batch size
  • Changing latent dimension

#4

Hi Aadil Hayat,

Found the article very informative, thanks. In the example you provide for GAN how would one change the latent dimension of it? Would you mind elaborating on what would be considered latent dimensions in this context?


#5

Hi Christian Contreras,
Thank you for your support. You can easily change the latent dimension by changing the dimension of the latent variable Z placeholder’s shape and correspondingly change the shape of Z while generating the samples of Z.

Z = tf.placeholder(tf.float32,[None, latent_dimension])
...
Z_batch = sample_Z(batch_size, latent_dimension)

In the example code provided the latent dimensions are coded to be 2.

Actually, the insights about what would be considered latent dimensions in this context have to be seen by plotting the correlation between Z and generated X. Seems like a good exercise for this particular dataset because it might be easy to interpret because of the simplicity of the dataset.


#6

Hello hayataadil,

Hope you are doing good.

I am very new to GAN and tensorflow, I was wondering what the sample_Z() function is doing.
Also I am not finding any reference for sess.run(), is there any specific imports I need?

If possible can you provide the complete code file on Github or in comments?
I want to debug it and understand the functionality of GAN.

I really appreciate your help.
Many many thanks.

Kaitav Metha


#7

Thanks Aadil for the clarification and about how to gain insight as to what’s to be considered the latent dimensions by plotting the correlation between “Z” and the generated “X”.

Cheers,
Christian


#8

Hi Kaitav,
Thank you for your support.
The link to the code is already given in the first few lines of the blog. I am reposting it https://github.com/aadilh/blogs/tree/new/basic-gans/basic-gans/code

sample_Z() function is used to sample from the uniform distribution of the required shape.
sess is just a normal tensorflow session object assigned using tf.Session() with the required configuration.
Please refer to the code link provided above.

Thanks,
Aadil


#9

HI hayat aadil

is it neccesary to biuld one GAN for each quadratic network(say GAN1 for 10+x^2, GAN2 for 3*x^2 etc) , can conditional GANs get around this problem.


#10

Hi Govind Narasimman,
No it is not necessary to build one GAN for each quadratic network. When you have mixture of such distributions (depending on the type of dataset) it is advised to use an improvement of GANs such as Conditional-GANs, InfoGANs, MAD-GANs, etc. because vanilla GANs suffer from mode-collapse problem. I am planning cover these topics in future.

Thanks,
Aadil


#11

Hi,
Thanks so much for this detailed GAN tutorial.
I’ve read many examples about GAN implementations which are mostly about images. This code runs GAN for simple 2D data. Now, I have a question about this example. Is it possible to use GAN for prediction? My data has 130 features and I want to use GAN to make regression for my data. Is GAN useful for this aim?
Thanks in advance.


#12

Hi Leily,
Sorry for late reply.
Yes it is possible to use GANs for predictions. You can use it for regression by using conditional GANs where you condition the input of generator and discriminator on your input features, You generate your regression prediction using the generator network and discriminator network distinguishes between real prediction and generated prediction. Although it is possible I think this should be only used for very complex dataset where other methods are failing because GANs are inherently unstable and have mode collapse problem.

I hope that helps.

Thanks,
Aadil


#13

Hello,
Thank you for this really good article.
I got issue understanding what is “rep” ? And how do you plot the feature transform : I mean what are the axes?
Thank you for your help
Best Regards


#14

Hello there,

It is a pretty good article. But I tried the same program on a cubic function however the generative model was not able to generate the copy of real image. Can you suggest me any idea to make it work?


#15

Hi Pauline,
Thank you for your support.
‘rep’ is a short form of representation. If you notice in the discriminator network I have intentionally made the last hidden layer to have 2 units. This is basically the feature transformation that is being learned by the discriminator just before it is classifying the input into real or fake. This is what I plot in the end to show how separable are the 2 categories of data in the transformed space.

Let me know if further clarification is needed.

Thanks,
Aadil


#16

Hi Abodh,
Thank you for the support. This same code works for learning MNIST handwitting dataset with few changes so I do not see why it would not work for cubic function. It is possible that we may need to modify some parameters like number of layers, number of units, etc. but ya it should work.

Thanks,
Aadil


#17

Thank you for your great article, I would like to ask if i want to use tha same code but as conditional gans, I want generate vector with one dimension and my condition is one scalar value. DO you think it can work ? Thanks


#18

Hi Saida,
Thank you for your support.
Yes, conditional GANs can be easily implemented by making a few modifications to the given code.
I am not sure about your specific dataset but I have used it for some datasets and it works.

Thanks,
Aadil


#19

Hello Adail

Than you for you reply. I am very new to Gans . i am trying to use that code. but my datset is 10000 row vectors and every vector is 1000 dim and every vector has score values (scalar between zero and one). and i want to generate vectors giving a score.

I saw that code

it is the same as you implemented , but you explain everything. But i do not understand what is X_dim and y_dim and Z_dim?


#20

Hi Saida,

From what I understand x_dim is the dimension of the data that you want to generate as the output of Generator, y_dim is the dimension of the labels of the data which you would use as the conditional inputs and z_dim is the dimension of the noise distribution.

In your case, x_dim = 1000, y_dim = 1 and z_dim is whatever you choose as the dimension of the noise.

Thanks,
Aadil