Invertible Gaussian Reparameterization and Logistic-Normal approximation of Dirichlet #2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR focuses on implementing Invertible Gaussian Reparameterization trick with corresponding distribution, and custom Logistic-Normal distribution with two-side approximation with Dirichlet distribution.
Some details about closed-form Laplace bridge for Logistic-Normal and Dirichlet distributions
We implement a
LogisticNormalSoftmax
distribution, which is a transformed distribution from theNormal
one. In contrast to originalLogisticNormal
frompyro
ortorch
, this one usesSoftmaxTransform
, instead ofStickBreakingTransform
that allows us to remain in the same dimensionality.We implement two distinct functions, each of them has a distribution on input (Dirichlet or Logistic-Normal), and returns an approximation distribution.