Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead
Black box machine learning models are currently being used for high stakes decision-making throughout society, causing problems throughout healthcare, criminal justice, and in other domains. People have hoped that creating methods for explaining these black box models will alleviate some of these problems, but trying to \textit{explain} black box models, rather than creating models that are \textit{interpretable} in…
Wexler R. When a Computer Program Keeps You in Jail: How Computers are Harming Criminal Justice. New York Times. 2017 June 13;.
McGough M. How bad is Sacramento’s air, exactly? Google results appear at odds with reality, some say. Sacramento Bee. 2018 August 7;.
Varshney KR, Alemzadeh H. On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big Data. 2016 10;5.
Freitas AA. Comprehensible classification models: a position paper. ACM SIGKDD Explorations Newsletter. 2014 Mar;15(1):1–10.
Kodratoff Y. The comprehensibility manifesto. KDD Nugget Newsletter. 1994;94(9).
Huysmans J, Dejaeger K, Mues C, Vanthienen J, Baesens B. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Systems. 2011;51(1):141–154.
Rüping S. Learning Interpretable Models. Universität Dortmund; 2006.
Gupta M, Cotter A, Pfeifer J, Voevodski K, Canini K, Mangylov A, et al. Monotonic calibrated interpolated look-up tables. Journal of Machine Learning Research. 2016;17(109):1–47.
Lou Y, Caruana R, Gehrke J, Hooker G. Accurate Intelligible Models with Pairwise Interactions. In: Proceedings of 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM; 2013. .
Miller G. The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review. 1956;63:81–97.
Cowan N. The magical mystery four how is working memory capacity limited, and why? Current directions in psychological science. 2010;19(1):51–57.
Wang J, Oh J, Wang H, Wiens J. Learning Credible Models. In: Proceedings of 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM; 2018. p. 2417–2426.
Rudin C. Please Stop Explaining Black Box Models for High Stakes Decisions. In: Proceedings of NeurIPS 2018 Workshop on Critiquing and Correcting Trends in Machine Learning; 2018. .
Holte RC. Very simple classification rules perform well on most commonly used datasets. Machine Learning. 1993;11(1):63–91.
Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases. AI Magazine. 1996;17:37–54.
Chapman P, et al. CRISP-DM 1.0 - Step-by-step data mining guide. SPSS; 2000.
Agrawal D, Bernstein P, Bertino E, Davidson S, Dayal U, Franklin M, et al. Challenges and Opportunities with Big Data: A white paper prepared for the Computing Community Consortium committee of the Computing Research Association; 2012. Available from: http://cra.org/ccc/resources/ccc-led-whitepapers/.
Defense Advanced Research Projects Agency. Broad Agency Announcement, Explainable Artificial Intelligence (XAI), DARPA-BAA-16-53; 2016. Published August 10. Available from https://www.darpa.mil/attachments/DARPA-BAA-16-53.pdf.
Hand D. Classifier Technology and the Illusion of Progress. Statist Sci. 2006;21(1):1–14.
Rudin C, Passonneau R, Radeva A, Dutta H, Ierome S, Isaac D. A Process for Predicting Manhole Events In Manhattan. Machine Learning. 2010;80:1–31.
Rudin C, Ustun B. Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice. Interfaces. 2018;48:399–486. Special Issue: 2017 Daniel H. Wagner Prize for Excellence in Operations Research Practice September-October 2018.
Chen C, Lin K, Rudin C, Shaposhnik Y, Wang S, Wang T. An Interpretable Model with Globally Consistent Explanations for Credit Risk. In: Proceedings of NeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services: the Impact of Fairness, Explainability, Accuracy, and Privacy; 2018. .
Mittelstadt B, Russell C, Wachter S. Explaining Explanations in AI. In: In Proceedings of Fairness, Accountability, and Transparency (FAT*); 2019. .
Flores AW, Lowenkamp CT, Bechtel K. False Positives, False Negatives, and False Analyses: A Rejoinder to “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals”. Federal probation. 2016 September;80(2):38–46.
Angwin J, Larson J, Mattu S, Kirchner L. Machine Bias. ProPublica; 2016. Available from: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
Larson J, Mattu S, Kirchner L, Angwin J. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica; 2016. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.
Rudin C, Wang C, Coker B. The age of secrecy and unfairness in recidivism prediction. arXiv e-prints 1811 00731 $[$applied statistics$]$. 2018 Nov;.
Brennan T, Dieterich W, Ehret B. Evaluating the Predictive Validity of the COMPAS Risk and Needs Assessment System. Criminal Justice and Behavior. 2009 January;36(1):21–40.
Zeng J, Ustun B, Rudin C. Interpretable classification models for recidivism prediction. Journal of the Royal Statistical Society: Series A (Statistics in Society). 2017;180(3):689–722.
Tollenaar N, van der Heijden PGM. Which method predicts recidivism best?: a comparison of statistical, machine learning and data mining predictive models. Journal of the Royal Statistical Society: Series A (Statistics in Society). 2013;176(2):565–584.
Angelino E, Larus-Stone N, Alabi D, Seltzer M, Rudin C. Certifiably optimal rule lists for categorical data. Journal of Machine Learning Research. 2018;19:1–79.
Mannshardt E, Naess L. Air quality in the USA. Significance. 2018 Oct;15:24–27.
Zech JR, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018;15(e1002683).
Chang A, Rudin C, Cavaretta M, Thomas R, Chou G. How to Reverse-Engineer Quality Rankings. Machine Learning. 2012 September;88:369–398.
Goodman B, Flaxman S. EU regulations on algorithmic decision-making and a ‘right to explanation’. AI Magazine. 2017;38(3).
Wachter S, Mittelstadt B, Russell C. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology. 2018;1(2).
Quinlan JR. C4. 5: programs for machine learning. vol. 1. Morgan Kaufmann; 1993.
Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and regression trees. CRC press; 1984.
Auer P, Holte RC, Maass W. Theory and Applications of Agnostic PAC-Learning with Small Decision Trees. In: Machine Learning Proceedings 1995. San Francisco (CA): Morgan Kaufmann; 1995. p. 21 – 29.
Wang F, Rudin C. Falling Rule Lists. In: Proceedings of Machine Learning Research Vol. 38: Artificial Intelligence and Statistics (AISTATS); 2015. p. 1013–1022.
Chen C, Rudin C. An optimization approach to learning falling rule lists. In: Proceedings of Machine Learning Research Vol. 84: Artificial Intelligence and Statistics (AISTATS); 2018. p. 604–612.
Burgess EW. Factors determining success or failure on parole; 1928. Illinois Committee on Indeterminate-Sentence Law and Parole Springfield, IL.
Ustun B, Rudin C. Optimized Risk Scores. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD); 2017. .
Ustun B, Rudin C. Supersparse linear integer models for optimized medical scoring systems. Machine Learning. 2015;p. 1–43.
Carrizosa E, Martín-Barragán B, Morales DR. Binarized support vector machines. INFORMS Journal on Computing. 2010;22(1):154–167.
Sokolovska N, Chevaleyre Y, Zucker JD. A Provable Algorithm for Learning Interpretable Scoring Systems. In: Proceedings of Machine Learning Research Vol. 84: Artificial Intelligence and Statistics (AISTATS); 2018. p. 566–574.
Ustun B, et al. The World Health Organization Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5. JAMA Psychiatry. 2017;74(5):520–526.
Chen C, Li O, Tao C, Barnett A, Su J, Rudin C. This Looks Like that: Deep Learning for Interpretable Image Recognition. In: Neural Information Processing Systems (NeurIPS); 2019. .
O’Malley D. Clay-colored Sparrow; 2014. Public domain image. https://www.flickr.com/photos/62798180@N03/11895857625/.
ksblack99. Clay-colored Sparrow; 2018. Public domain image. https://www.flickr.com/photos/ksblack99/42047311831/.
Schmierer A. Clay-colored Sparrow; 2017. Public domain image. https://flic.kr/p/T6QVkY.
Schmierer A. Clay-colored Sparrow; 2015. Public domain image. https://flic.kr/p/rguC7K.
Schmierer A. Clay-colored Sparrow; 2015. Public domain image. https://www.flickr.com/photos/sloalan/16585472235/.
Li O, Liu H, Chen C, Rudin C. Deep Learning for Case-based Reasoning through Prototypes: A Neural Network that Explains its Predictions. In: Proceedings of AAAI Conference on Artificial Intelligence (AAAI); 2018. p. 3530–3537.
Gallagher N, et al. Cross-Spectral Factor Analysis. In: Proceedings of Advances in Neural Information Processing Systems 30 (NeurIPS). Curran Associates, Inc.; 2017. p. 6842–6852.
Wang F, Rudin C, Mccormick TH, Gore JL. Modeling recovery curves with application to prostatectomy. Biostatistics. 2018;p. kxy002. Available from: http://dx.doi.org/10.1093/biostatistics/kxy002.
Lou Y, Caruana R, Gehrke J. Intelligible Models for Classification and Regression. In: Proceedings of Knowledge Discovery in Databases (KDD). ACM; 2012. .
Semenova L, Parr R, Rudin C. A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning; 2018. In progress.
Razavian N, et al. Population-Level Prediction of Type 2 Diabetes From Claims Data and Analysis of Risk Factors. Big Data. 2015;3(4).
Ustun B, Spangher A, Liu Y. Actionable Recourse in Linear Classification. In: ACM Conference on Fairness, Accountability and Transparency (FAT*); 2019. .
Su G, Wei D, Varshney KR, Malioutov DM. Interpretable Two-Level Boolean Rule Learning for Classification. In: Proceedings of ICML Workshop on Human Interpretability in Machine Learning; 2016. p. 66–70.
Dash S, Günlük O, Wei D. Boolean Decision Rules via Column Generation. In: 32nd Conference on Neural Information Processing Systems (NeurIPS); 2018. .
Wang T, Rudin C, Doshi-Velez F, Liu Y, Klampfl E, MacNeille P. A Bayesian Framework for Learning Rule Sets for Interpretable Classification. Journal of Machine Learning Research. 2017;18(70):1–37.
Rijnbeek PR, Kors JA. Finding a Short and Accurate Decision Rule in Disjunctive Normal Form by Exhaustive Search. Machine Learning. 2010 Jul;80(1):33–62.
Goh ST, Rudin C. Box Drawings for Learning with Imbalanced Data. In: Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD); 2014. .
Built from what the sources declared and what the gates observed.
Nothing absent has been filled in here.
1 value read out of the text by the enrichment rules and 1 link to or from other resources — a lab's files, the pages it links, the works it cites stand under their elements, marked inferred, and are kept apart in the exports.
DCMI Metadata Terms. Dublin Core has no element that separates the original file from the text extracted out of it, and none for LOM's educational characterisation. Both survive here as provenance statements and in the record itself, not in the projection.
the standard ↗
The groups below are this library's, for reading. DCMI Terms itself has no categories; each term keeps its standard name.
Black box machine learning models are currently being used for high stakes decision-making throughout society, causing problems throughout healthcare, criminal justice, and in other domains. People have hoped that creating methods for explaining these black box models will alleviate some of these problems, but trying to \textit{explain} black box models, rather than creating models that are \textit{interpretable} in the first place, is likely to perpetuate bad practices and can potentially cause catastrophic harm to society. There is a way forward -- it is to design models that are inherently interpretable. This manuscript clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare, and computer vision.
The abstract, and the depositor's additional notes after it when the source has a field for them.
CC-BY-SA-4.0 — assessed as SHARE_ALIKE by the harvester's licence gate. Conditions: attribution required, adaptations must carry the same licence. open but ShareAlike/copyleft — derivatives keep the license
Nature Machine Intelligence, Vol 1, May 2019, 206-215
Where the work was published, in the source's own words: a journal with its volume and pages, a conference, an imprint.
Relations and custody
1/9
Is part ofdcterms:isPartOf
none found — the source did not declare it
The repository, book or record this resource was found inside.
Has partdcterms:hasPart
none found — the source did not declare it
What this resource is made of, when the source lists its parts.
Is version ofdcterms:isVersionOf
none found — the source did not declare it
Has versiondcterms:hasVersion
none found — the source did not declare it
Referencesdcterms:references
inferred
read its reference list, line 315citation
doi:10.1093/biostatistics/kxy002
Works this one cites, when the source declares them as relations. What its text links to and its reference list cites is inferred, and stands under it apart.
Is referenced bydcterms:isReferencedBy
none found — the source did not declare it
What links to or cites this one, when a source declares it; inferred from the texts held otherwise, and set apart.
Requiresdcterms:requires
none found — the source did not declare it
What this resource needs in order to be used, when a source declares it. The files a lab works on are inferred, and stand under it apart.
Is required bydcterms:isRequiredBy
none found — the source did not declare it
What needs this resource, when a source declares it. The lab a component belongs to is inferred, and stands under it apart.
Provenancedcterms:provenance
this engine
source
conversion
latexml-html
Retrieved from arXiv on 2026-10-09 in response to the search string “(all:"artificial intelligence" OR all:"machine learning" OR all:"generative AI" OR all:"deep learning" OR all:"reinforcement learning" OR all:"large language model") AND (all:"AI concepts" OR all:"types of AI" OR all:"AI fundamentals" OR all:"recognizing AI" OR all:"recognising AI" OR all:"general versus narrow AI" OR all:"narrow AI" OR all:"general AI" OR all:"machine intelligence" OR all:"AI strengths and weaknesses" OR all:"traditional software" OR all:"rule-based systems" OR all:"introduction to AI" OR all:"introduction to artificial intelligence" OR all:"artificial intelligence introduction" OR all:"AI primer" OR all:"foundations of artificial intelligence" OR all:"overview of AI" OR all:"understanding AI" OR all:"history of AI" OR all:"AI essentials" OR all:"AI terminology" OR all:"metaphors for AI" OR all:"AI fundamental concepts" OR all:"AI key concepts" OR all:"philosophy of AI" OR all:"critical AI literacy")”. arXiv served the resource and is not asserted to be its publisher or author.
Text extracted from latexml-html to Markdown by arxiv-html; the original is retained unchanged beside it.
Where it was collected from, what was converted, and what container it came out of — the custody statements that would otherwise be mistaken for authorship.
IEEE 1484.12.1 Learning Object Metadata. LOM has no element for an SPDX identifier or a licence URI, so both are written into 6.3 Rights.Description. Flattening this record into simple Dublin Core would lose more again, which is why the two projections exist side by side rather than one being generated from the other.
the standard ↗
Black box machine learning models are currently being used for high stakes decision-making throughout society, causing problems throughout healthcare, criminal justice, and in other domains. People have hoped that creating methods for explaining these black box models will alleviate some of these problems, but trying to \textit{explain} black box models, rather than creating models that are \textit{interpretable} in the first place, is likely to perpetuate bad practices and can potentially cause catastrophic harm to society. There is a way forward -- it is to design models that are inherently interpretable. This manuscript clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare, and computer vision.
The abstract, and the depositor's additional notes after it as a second LangString when the source has a field for them.
Role, entity and date per declared contribution. A role outside LOM's vocabulary is reported in the entry's description instead.
3 Meta-metadata
4/4
Identifier3.1
this engine
resource_id
URI: tag:aim-pro.eu,2026:oer/ac97f0396dda/record
The identifier of this metadata record — the resource's own, with /record after it, because the record is a description of the resource and not the resource.
Contribute3.2
this engine
source
AIM-PRO WP3 OER harvester (arXiv) — creator
Who generated this record and when — a statement about the record, not about the resource.
Metadata schema3.3
LOMv1.0
aimpro-oer-profile/1
LOMv1.0, and the profile this was built against.
Language3.4
language gate
read the declared field
en
4 Technical
3/7
Format4.1
conversion
latexml-html
text/markdown
One value per form held: the original as the source published it, and the Markdown this engine extracted.
Size4.2
conversion
latexml-html
84815
Bytes. The original's, because the resource is the file and not our conversion of it.
Location4.3
this engine
resource_id
https://arxiv.org/abs/1811.10154
Where the source serves it.
Requirement4.4
not collected — this library does not fill it
Software or hardware needed to use it. No source declares it.
Installation remarks4.5
not collected — this library does not fill it
No source declares it.
Other platform requirements4.6
not collected — this library does not fill it
No source declares it.
Duration4.7
not collected — this library does not fill it
Playing time, for audio and video. The corpus holds neither.
5 Educational
1/11
Interactivity type5.1
not collected — this library does not fill it
Active, expositive or mixed. A judgement about how the resource is used; no source declares it.
Yes unless the licence reserves nothing — attribution is a restriction. The conditions after the dash are the licence gate's reading; the export carries LOM's bare term.
CC-BY-SA-4.0 · http://creativecommons.org/licenses/by-sa/4.0/. Conditions: adaptation permitted, adaptations must carry the same licence, attribution required.
The SPDX id, the licence URI and the conditions. LOM has no element for any of the three, so this is where they survive.
7 Relation
2/2
Kind7.1
ispartof
Resource7.2
inferred
read its reference list, line 315citation
Nature Machine Intelligence, Vol 1, May 2019, 206-215
references: doi:10.1093/biostatistics/kxy002
The container a file was found inside, and the Markdown extracted from the original. What a lab requires, and the lab a component belongs to, are inferred and stand apart.
8 Annotation
0/3
Entity8.1
not collected — this library does not fill it
Comments on the resource's educational use, by whoever made them. The platform's review grades competencies, which are classification (9), and writes no comment here.
Date8.2
not collected — this library does not fill it
Description8.3
not collected — this library does not fill it
9 Classification
0/4
Purpose9.1
not collected — this library does not fill it
Empty in the record: no source declares a competency. The alignment reads the resource for them and stands beside the record, never in it, and a taxon path derived from the search string that found it would be a claim about the query.
Taxon path9.2
not collected — this library does not fill it
Where the competency framework goes. Empty in the record for the reason above.
Description9.3
not collected — this library does not fill it
Keyword9.4
not collected — this library does not fill it
What could not be established3
Where the source's metadata could not be carried
over as it was — missing, contradictory, with no matching term in the
standard, restructured, or taken from the repository — and what was done
instead. Without these notes, an empty element would look like something
the harvester missed.
Status
Field
Why
Not available
rights_holder
no rights holder is named at source; the licence is recorded without one rather than attributed to the platform that served it
Not available
publisher
the source named no publisher of the work; where it was collected from is recorded as collection provenance instead, which is a different claim
Not available
educational
the source declared no educational metadata — no resource type, audience, context, difficulty or learning time. Nothing here estimates them