5. Applications and Broader Impact: Foundations of Inference

Draft chapter

This chapter collects the broader-impact discussion and readings that accompanied Part I in AM207 (Harvard, Fall 2021). It will be expanded into a full chapter.

5.1. Broader Impact

From the Chapter 1 problem set you’ve already seen that model evaluation is far more than just computing the MSE or, indeed, any metric. Interpreting the model in the domain application is equally important:

1. “Gut checking” the clinical meaning of the slope and y-intercept of a linear model can tell us if the model is appropriate.

• Why is it hard to “gut check” model for the dataset in the Chapter 1 problem set. What does this say about our typical ML model building pipeline?
  
  

• What if the model is not easy to interpret by looking at the parameters themselves? How do we know that our interpretations of models are accurate and objective? What is an “accurate” interpretation of a model anyways? Can models be “objectively” interpreted? (Reading: Towards A Rigorous Science of Interpretable Machine Learning)
  
  

• What if the data has been preprocessed like in the Chapter 1 problem set? What does this say about ML development pipelines where the ML experts are removed from the data cleaning, preprocessing step?

2. We’ve also seen that for risk-averse applications it’s crucial to get predictive uncertainty from our models.

• Why is predictive uncertainty useful?

• What is predictive uncertainty useful for?

• Under what real-life conditions would we need predictive uncertainty?

(Reading: Second opinion needed: communicating uncertainty in medical machine learning, Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty)

5.2. Readings

Promises of Machine Learning

• (Introductory) From Diagnosis to Holistic Patient Care, Machine Learning is Transforming Healthcare

• (Introductory) The future of agriculture is computerized

• (In-Depth) Tackling Climate Change with Machine Learning

• (In-Depth) Machine Learning for the Developing World

• (In-Depth) AI for social good: unlocking the opportunity for positive impact

Potential Dangers of Machine Learning

• (Introductory) Predictive policing algorithms are racist. They need to be dismantled.

• (Introductory) When Good Algorithms Go Sexist: Why and How to Advance AI Gender Equity

• (Introductory) Millions of black people affected by racial bias in health-care algorithms

• (Introductory) Medicine’s Machine Learning Problem

• (Introductory) Algorithms Are Making Economic Inequality Worse

• (Introductory) AI needs to face up to its invisible-worker problem

• (In-Depth) A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions

Machine Learning and Ethics

• (Introductory) Artificial Intelligence in Health: Ethical Considerations for Research and Practice

• (In-Depth) Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward

• (In-Depth) Data Science as Political Action: Grounding Data Science in a Politics of Justice

Applications and Broader Impact

• (Introductory) Second opinion needed: communicating uncertainty in medical machine learning

• (Introductory) Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty

• (Introductory) Interpretability (Chapter 2 of Interpretable Machine Learning: A Guide for Making Black Box Models Explainable)

• (In-Depth) Towards A Rigorous Science of Interpretable Machine Learning

• Bayesian Modeling of Uncertainty in Ensembles of Climate Models

• All together now: the most trustworthy covid-19 model is an ensemble