The model which is used for generating samples

The deep learning can be used to produce hierarchical models which illustrate probability distributions over the types of data that we come across in artificial intelligence applications, such as robotics, speech recognition, and symbols in natural language processing 1. Deep generative models have had a small effect, due to the difficulty of approximating many uncontrollable probabilistic calculations arising in maximum likelihood estimation and associated approaches, and due to the difficulty of having the advantages of piecewise linear units in the generative conditions 2. Generative Adversarial Networks (GANs) provide an attractive alternative to maximum likelihood techniques. GANs have great capabilities and have been applied even for extracting features and for classifying different tasks 3. These tasks are generally performed by including feature matching technique for training the generator network and multitask training of the discriminator network, which plays an extra role as a classifier too. GAN’s contain the generative model which is used for generating samples by adding random noise through a multilayer perceptron (class of feedforward artificial neural network), and it also consists of the discriminative model which is also a multilayer perceptron. The combined system is called the adversarial nets. Training is performed on both the models using backpropagation for optimising the weights and also utilize the dropout algorithms and sample from the generative model using forward propagation. They do not require the approximate inference or Markov chains as required by classical Boltzmann machines. GANs have their objective to achieve an equilibrium between a generator and a discriminator; whereas VAEs have their goal to maximize a lower bound of the data log-likelihood. Deep generative models, such as Deep Belief Networks (DBNs) and Deep Boltzmann Machines (DBMs), Restricted Boltzmann Machines (RBM) used MCMC-based algorithms for training their networks 4, 5. In these approaches, the Markov Chain Monte Carlo (MCMC) methods compute the gradient of log-likelihood which adds on imprecision as training progresses. This is because samples from the Markov Chains are unable to mix between modes fastly. Several generative models have been developed and trained via direct back-propagation and avoid the difficulties that come with MCMC training 6. Application of the structural similarity index as an autoencoder (AE) reconstruction metric for grey-scale images was applied in 7. Simple VAEs have been reformed to even importance weighted VAEs to obtain a more stringent lower bound 8. Several new forms of GANs have been developed, even involving a combination of VAEs for improved forms and generations. The adversarial principle has found the application in generation setting and also been applied to other factors such as domain adaptation and Bayesian inference which uses implicit variational distributions in VAEs and encourage the adversarial method for optimization 9.