Variational Auto Encoders

Variational Auto Encoders are a bayesian perspective to basic auto encoders (the usual encoder — decoder perspective) , thereby treating the inputs, hidden representations, and reconstructed outputs as probabilistic random variables within a directed graphical model.

They perceive the encoder as a bayesian inference network mapping the observed inputs ( given samples)to approximate posterior distribution over a latent space ( hidden state). The decoder is then perceived to be a generative model which when samples from the posterior distribution and maps the data to original input space.

The advantage with such a model is in its beauty to parse the information spread thinly over the high-dimensional observed world and condense the most meaningful features into a structured distribution over reduced latent dimensions through a bottleneck.

This is the major part which differentiates them from a traditional auto encoder paradigm : their condition on encoder part of the network to be constraint enough to compress to latent vectors from a manifold that follows unit gaussian distribution. They are evaluated using a combination of generative loss ( How close are their generated samples to the original input space) and Latent loss ( How closely are they able to follow the above constraint ) . For the generative loss we can use mean squared error between the input and the generated samples or the binary cross entropy loss while for the latent loss we add in the form of KL Divergence between the distribution created by the encoder and the prior distribution ( which we take as standard normal distribution).

Adversarial Auto Encoders : One of the cons of using KL divergence is that it does not have a closed form analytical solutions barring particular distributions . Also it is not tractable to use discrete distributions for the latent code Z .

Related posts:

La vida es una carrera que exige mucha resistencia, ya que a lo largo del camino se nos presentan muchos desafíos. Pero he aprendido en el transcurso de ella que uno elige qué camino tomar; ya sea el…

SaaS or Software as a service is an application available through a network and works on a subscription or billed as per usage. It is critical that the application is usable and provides an excellent…

From New York drug busts to brutal revenge in Newcastle, these thrillers look better than ever fifty years on.