Week 7

Lecture

• Lecture #12: Logistic Regression and Gradient Descent
  • Lecture Video
  • Lecture Notes
  • Lecture Summary
• Lecture #13: Stochastic Gradient Descent and Simulated Annealing
  • Lecture Video
  • Lecture Notes
  • Lecture Summary

Activities

• In-Class Exercise #12: Interpreting and Evaluating Logistic Regression Models
• In-Class Exercise #12: Solutions
• In-Class Exercise #13: Evaluating and Interpreting Hierarchical GLMs
• In-Class Exercise #13: Solutions
• Homework #6
• Homework #6 Solutions

Reading

Applications and Broader Impact

1. FTC Declares Racially Biased Algorithms in Artificial Intelligence Unfair and Deceptive, Prohibited by Law
2. Legal requirements on explainability in machine learning
3. Blind Justice: Fairness with Encrypted Sensitive Attributes
4. Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination
5. Equal Protection Under the Algorithm: A Legal-Inspired Framework for Identifying Discrimination in Machine Learning
6. The Problematic Nature of Racial and Ethnic Categories in Higher Education
7. Gender (mis)measurement: Guidelines for respecting gender diversity in psychological research

Gradient Descent and Stochastic Gradient Descent 

1. (Introductory) Gradient Descent Algorithm and Its Variants
2. (In-Depth) Stochastic Gradient “Descent” Algorithm
3. (Advanced) On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes
4. (Advanced) On the Stability and Convergence of Stochastic Gradient Descent with Momentum
5. (Practical) Stochastic Gradient Descent Tricks
6. (In-Depth) Convex Optimization and Approximation

Model Selection for Bayesian Models

1. Example of Bayesian model selection from Statistical Rethinking

Simulated Annealing and Non-Convex Optimization

1. (In-Depth) Simulated Annealing
2. (Advanced) The Theory and Practice of Simulated Annealing
3. (Advanced) Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing