Abstract:

Variational Autoencoders (VAEs) have garnered substantial attention as generative models for producing lower-dimensional representations of high-dimensional data. The $\beta$-VAE model employs the hyperparameter $\beta$ to strike a balance between reconstruction accuracy and disentanglement. This study exclusively targets the enhancement of reconstruction accuracy in the linear Gaussian $\beta$-VAE model by introducing three variants: $\gamma$-VAE with both arbitrary and diagonalized $\Sigma_{Z}$, as well as $\gamma\lambda$-VAE with diagonalized $\Sigma_{Z}$. We commence by deriving closed-form solutions for all three proposed frameworks using gradient-based and iterative methods. This demonstration of consistency between approaches highlights the robustness of our findings. Subsequently, we perform comprehensive numerical experiments employing the Blahut-Arimoto algorithm. These experiments underscore the benefits of utilizing a diagonalized positive definite $\Sigma_{Z}$ over an arbitrary one, leading to more informative numerical outcomes and augmented control over reconstruction accuracy. Furthermore, the introduction of an additional hyperparameter $\lambda$ offers an avenue for further refining reconstruction accuracy control. In conclusion, the introduction of these three variants to the $\beta$-VAE model, combined with analytical and numerical analyses, underscores the potential for improved reconstruction accuracy through the strategic incorporation of additional hyperparameters and nuanced adjustments to the foundational framework.