Week 10

Lecture

• Lecture #18: Automatic Differentiation
  • Lecture Video
  • Lecture Notes
• Lecture #19: Variational Inference in Context
  • Lecture Video
  • Lecture Notes
  • Lecture Summary

Activities

• In-Class Exercise #18: Automatic Differentiation
• In-Class Exercise #19: Evaluating and Comparing Approximate Inference
• In-Class Exercise #19: Solutions

Reading

Applications and Broader Impact:

1. Risk versus Uncertainty in Deep Learning: Bayes, Bootstrap and the Dangers of Dropout
2. It’s Too Easy to Hide Bias in Deep-Learning Systems  Artificial intelligence makes it hard to tell when decision-making is biased
3. The security threat of adversarial machine learning is real
4. Would you trust a machine to pick a vaccine?

Automatic Differentiation

1. Automatic Differentiation in Machine Learning: a Survey

Approximate Inference for Deep Bayesian Models

1. (Using an Alternate Divergence Measure for VI) Black-Box α-Divergence Minimization
2. (Using an Alternate Variational Family) Variational Inference with Normalizing Flows
3. (Approximating the Posterior in a Subspace) Subspace Inference for Bayesian Deep Learning

Evaluating Approximate Inference for Deep Bayesian Models

1. (Why are BNN Posteriors So Difficult?) Visualizing the Loss Landscape of Neural Nets
2. (How to Evaluate the Epistemic Uncertainty of Approximate Posteriors of BNNs) Quality of Uncertainty Quantification for Bayesian Neural Network Inference
3. (Hard Limitations of Mean-Field VI for BNNs) On the Expressiveness of Approximate Inference in Bayesian Neural Networks

Alternate Models for Deep Bayesian Networks

1. (Neural Linear Models) Learned Uncertainty-Aware (LUNA) Bases for Bayesian
2. (Manifold Gaussian Processes) Manifold Gaussian Processes for Regression