27. Applications and Broader Impact: Bayesian Deep Learning#
Draft chapter
This chapter collects the broader-impact discussion and readings that accompanied Part VI in AM207 (Harvard, Fall 2021). It will be expanded into a full chapter.
27.1. Interpretable Deep Learning#
We might be tempted to conclude that neural networks are uninterpretable due to their complexity. But just because we can’t understand neural networks by inspecting the value of the individual weights, it does not mean that we can’t understand them.
In The Mythos of Model Interpretability, the authors survey a large number of methods for interpreting deep models.
27.2. Can Machine Learning Models Make Use of Human Concepts?#
(with Anita Mahinpei, Justin Clark, Ike Lage, Finale Doshi-Velez)
What if instead building complex non-linear models based on raw inputs, we instead build simple linear models based on human interpretable concepts? We use a neural network to predict concepts from inputs and then use a linear model to predict the outcome from the concepts. We interpret the relationship between the outcome and the concepts via the linear model. These models are called concept bottleneck models.
In The Promises and Pitfalls of Black-box Concept Learning Models, we examine the advantages and drawbacks of these models.
27.3. Can Machine Learning Models Learn to Explore Hypothetical Scenarios?#
(with Michael Downs, Jonathan Chu, Wisoo Song, Yaniv Yacoby, Finale Doshi-Velez)
Rather than explaining why the model made a decision, it’s often more helpful to explain how to change the data in order to change the model’s decision. This modified input is a counter-factual. In CRUDS: Counterfactual Recourse Using Disentangled Subspaces, we study how to automatically generate counter-factual explanations that can help users achieve a favorable outcome from a decision system.
27.4. Right for the Right Reasons?#
In An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets, the authors build a neural network model to detect acute intracranial haemorrhage (ICH) and classifies five ICH subtypes.
Model classifications are explained by highlighting the pixels that contributed the most to the decision. The highlighted regions tends to overlapped with ‘bleeding points’ annotated by neuroradiologists on the images.
27.5. The Perils of Explanations#
In How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection, the authors found that clinicians interacting with incorrect recommendations paired with simple explanations experienced significant reduction in treatment selection accuracy.
Take-away: Incorrect ML recommendations may adversely impact clinician treatment selections and that explanations are insufficient for addressing overreliance on imperfect ML algorithms.
27.6. Readings#
Applications and Broader Impact