ConcreteDropout

Bayesian Methods for Machine Learing Course Project Skoltech 2018

Table of content

• Description
• Authors
• Results
• Usage
• References
• License

Description

We replicate the results of the recent paper Concrete Droput by Gal et al. and extend the results to new experiments.

Basic:

• Understand and discuss model implementation
• Reproduce experiments on: MNIST, Computer vision task and Reinforcement Learning

Extensions:

• Try different RL environments
• Evaluate the algorithm performance for NLP tasks
• Implement the Concrete Droupout for Recurrent Layers

Authors

• Valerii Likhosherstov
• Alfredo de la Fuente

Results

Computer Vision - Segmentation Task

---

UCI - Regression Task

---

Wine dataset

Boston dataset

MNIST - Classification Task

---

Usage

git clone https://github.com/Alfo5123/ConcreteDropout.git

References

Papers:

• Concrete Dropout
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Categorical Reparameterization with Gumbel-Softmax
• Variational Dropout and the Local Reparameterization Trick
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Improving PILCO with Bayesian Neural Network Dynamics Models
• Recurrent Neural Network Regularization

Repositories:

• Main Author Implementation
• Variational Dropout
• Probabilistic Inference for Learning Control

Blogs:

• The Gumbel-Softmax Trick for Inference of Discrete Variables
• Reparametrization Trick
• A review of Dropout as applied to RNNs
• Dropout in Recurrent Networks — Part 1

License

MIT License