research-article

Open access

Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives

Authors:

Zhiwei Steven Wu,

Haiyi ZhuAuthors Info & Claims

DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference

Pages 1245 - 1257

https://doi.org/10.1145/3357236.3395528

Published: 03 July 2020 Publication History

Abstract

Artificial intelligence algorithms have been used to enhance a wide variety of products and services, including assisting human decision making in high-stake contexts. However, these algorithms are complex and have trade-offs, notably between prediction accuracy and fairness to population subgroups. This makes it hard for designers to understand algorithms and design products or services in a way that respects users' goals, values, and needs. We proposed a method to help designers and users explore algorithms, visualize their trade-offs, and select algorithms with trade-offs consistent with their goals and needs. We evaluated our method on the problem of predicting criminal defendants' likelihood to re-offend through (i) a large-scale Amazon Mechanical Turk experiment, and (ii) in-depth interviews with domain experts. Our evaluations show that our method can help designers and users of these systems better understand and navigate algorithmic trade-offs. This paper contributes a new way of providing designers the ability to understand and control the outcomes of algorithmic systems they are creating.

Supplementary Material

PDF File (disfp8274aux.pdf)

Download
352.77 KB

References

[1]

2019. Introduction - Machine Learning - Human Interface Guidelines - Apple Developer. (2019). https://developer.apple.com/design/human-interface-guidelines/machine-learning/overview/introduction/

[2]

2019. People + AI Guidebook. (2019). https://design.google/ai-guidebook

[3]

Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI conference on human factors in computing systems. ACM, 582.

Digital Library

[4]

Alekh Agarwal, Alina Beygelzimer, Miroslav Dud�k, John Langford, and Hanna Wallach. 2018. A reductions approach to fair classification. arXiv preprint arXiv:1803.02453 (2018).

[5]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, and others. 2019. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[6]

Don A Andrews, James Bonta, and J Stephen Wormith. 2006. The recent past and near future of risk and/or need assessment. Crime & Delinquency 52, 1 (2006), 7--27.

[7]

Or Biran and Courtenay Cotton. 2017. Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI), Vol. 8. 1.

[8]

Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77--91.

[9]

Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. " Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1--24.

Digital Library

[10]

Shan Carter and Michael Nielsen. 2017. Using artificial intelligence to augment human intelligence. Distill 2, 12 (2017), e9.

[11]

Kathy Charmaz. 2014. Constructing grounded theory. sage.

[12]

Hao-Fei Cheng, Ruotong Wang, Zheng Zhang, Fiona O'Connell, Terrance Gray, F Maxwell Harper, and Haiyi Zhu. 2019. Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 559.

Digital Library

[13]

Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153--163.

[14]

Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Conference on Fairness, Accountability and Transparency. 134--148.

[15]

Eric S Chung, Jason I Hong, James Lin, Madhu K Prabaker, James A Landay, and Alan L Liu. 2004. Development and evaluation of emerging design patterns for ubiquitous computing. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques. 233--242.

Digital Library

[16]

Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 797--806.

Digital Library

[17]

Cynthia L Corritore, Beverly Kracher, and Susan Wiedenbeck. 2003. On-line trust: concepts, evolving themes, a model. International journal of human-computer studies 58, 6 (2003), 737--758.

Digital Library

[18]

Mark Craven and Jude Shavlik. 1999. Rule extraction: Where do we go from here. University of Wisconsin Machine Learning Research Group working Paper 99 (1999).

[19]

Jean-Charles de Borda. 1781. M�moire sur les �lections au scrutin. Histoire de l'Acad�mie Royale des Sciences (1781).

[20]

William Dieterich, Christina Mendoza, and Tim Brennan. 2016.comPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpoint Inc (2016).

[21]

Graham Dove, Kim Halskov, Jodi Forlizzi, and John Zimmerman. 2017. Ux design innovation: Challenges for working with machine learning as a design material. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 278--288.

Digital Library

[22]

Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances 4, 1 (2018), eaao5580.

[23]

Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. 2018. Decoupled classifiers for group-fair and efficient machine learning. In Conference on Fairness, Accountability and Transparency. 119--133.

[24]

Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)(Utrecht).

[25]

Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 80--89.

[26]

Michael Gleicher, Danielle Albers, Rick Walker, Ilir Jusufi, Charles D Hansen, and Jonathan C Roberts. 2011. Visual comparison for information visualization. Information Visualization 10, 4 (2011), 289--309.

Digital Library

[27]

Moritz Hardt, Eric Price, Nati Srebro, and others. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323.

[28]

Karen Holtzblatt, Jessamyn Burns Wendell, and Shelley Wood. 2004. Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier.

Digital Library

[29]

Lauren Kirchner Jeff Larson, Surya Mattu and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm

[30]

Surya Mattu Julia Angwin, Jeff Larson and Lauren Kirchner. 2016. Machine Bias. (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[31]

Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2019a. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 100--109.

Digital Library

[32]

Michael J. Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm�ssan, Stockholm, Sweden, July 10--15, 2018. 2569--2577. http://proceedings.mlr.press/v80/kearns18a.html

[33]

Michael J. Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2019b. An Empirical Study of Rich Subgroup Fairness for Machine Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29--31, 2019. 100--109.

Digital Library

[34]

Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018. Algorithmic fairness. In Aea papers and proceedings, Vol. 108. 22--27.

[35]

Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In 8th Innovations in Theoretical Computer Science Conference, ITCS 2017, January 9--11, 2017, Berkeley, CA, USA. 43:1--43:23.

[36]

Michael S Klinkman, James C Coyne, Susan Gallo, and Thomas L Schwenk. 1998. False positives, false negatives, and the validity of the diagnosis of major depression in primary care. Archives of Family Medicine 7, 5 (1998), 451--461.

[37]

Josua Krause, Adam Perer, and Kenney Ng. 2016. Interacting with predictions: Visual inspection of black-box machine learning models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5686--5697.

Digital Library

[38]

Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1675--1684.

Digital Library

[39]

John Lee and Neville Moray. 1992. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35, 10 (1992), 1243--1270.

[40]

Min Kyung Lee, Anuraag Jain, Hae Jin Cha, Shashank Ojha, and Daniel Kusbit. 2019. Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic mediation. In Proceedings of the ACM: Human-Computer Interaction. ACM, Article 182.

Digital Library

[41]

Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Seong Tae, Xinran Yuan, A. D. C. Chan, Ritesh Noothigattu, Daniel See, Siheon Lee, Christos-Alexandros Psomas, and Ariel D. Procaccia. 2018. WeBuildAI : Participatory Framework for Fair and Efficient Algorithmic Governance.

[42]

Adam W Meade and S Bartholomew Craig. 2012. Identifying careless responses in survey data. Psychological methods 17, 3 (2012), 437.

[43]

Aditya Krishna Menon and Robert C Williamson. 2018. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency. 107--118.

[44]

Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680--5689.

[45]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016a. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386 (2016).

[46]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016b. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1135--1144.

Digital Library

[47]

Georg Simmel. 1950. The sociology of georg simmel. Vol. 92892. Simon and Schuster.

[48]

Jay P Singh, Sarah L Desmarais, Cristina Hurducas, Karin Arbach-Lucioni, Carolina Condemarin, Kimberlie Dean, Michael Doyle, Jorge O Folino, Ver�nica Godoy-Cervera, Martin Grann, and others. 2014. International perspectives on the practical application of violence risk assessment: A global survey of 44 countries. International Journal of Forensic Mental Health 13, 3 (2014), 193--206.

[49]

Jennifer L Skeem and Christopher T Lowenkamp. 2016. Risk, race, and recidivism: Predictive bias and disparate impact. Criminology 54, 4 (2016), 680--712.

[50]

Justin Talbot, Bongshin Lee, Ashish Kapoor, and Desney S Tan. 2009. EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1283--1292.

Digital Library

[51]

Philip van Allen. 2018. Prototyping ways of prototyping AI. interactions 25, 6 (2018), 46--51.

[52]

Michael Veale, Max Van Kleek, and Reuben Binns. 2018. Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the 2018 chi conference on human factors in computing systems. 1--14.

Digital Library

[53]

Lauren Weber and RE Silverman. 2012. Your resume vs. oblivion. The Wall Street Journal 24 (2012).

[54]

Ann Wilkinson, Alison E While, and Julia Roberts. 2009. Measurement of information and communication technology experience and attitudes to e-learning of students in the healthcare professions: integrative review. Journal of advanced nursing 65, 4 (2009), 755--772.

[55]

Abraham J Wyner, Matthew Olson, Justin Bleich, and David Mease. 2017. Explaining the success of adaboost and random forests as interpolating classifiers. The Journal of Machine Learning Research 18, 1 (2017), 1558--1590.

Digital Library

[56]

Qian Yang, Alex Scuito, John Zimmerman, Jodi Forlizzi, and Aaron Steinfeld. 2018. Investigating how experienced UX designers effectively work with machine learning. In Proceedings of the 2018 Designing Interactive Systems Conference. 585--596.

Digital Library

[57]

Qian Yang, Aaron Steinfeld, Carolyn P Ros�, and John Zimmerman. 2020. Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 chi conference on human factors in computing systems.

Digital Library

[58]

Qian Yang, John Zimmerman, Aaron Steinfeld, and Anthony Tomasic. 2016. Planning adaptive mobile experiences when wireframing. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. 565--576.

Digital Library

[59]

Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-sensitive algorithm design: method, case study, and lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 194.

Digital Library

Cited By

Meyer LZhu J(2024)Slide to Explore 'What If': An Analysis of Explainable InterfacesAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685416(1-6)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3677045.3685416
Song YLi CXing WLi SLee H(2024)A Fair Clustering Approach to Self-Regulated Learning Behaviors in a Virtual Learning EnvironmentProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636863(771-778)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636863
Yildirim NZlotnikov SSayar DKahn JBukowski LAmin SRiman KDavis BMinturn JKing ARicketts DTang LSivaraman VPerer APreum SMcCann JZimmerman J(2024)Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care UnitProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641896(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641896
Show More Cited By

Index Terms

Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems Conference

With the growing industry applications of Artificial Intelligence (AI) systems, pre-trained models and APIs have emerged and greatly lowered the barrier of building AI-powered products. However, novice AI application designers often struggle to ...
Finding Insights through Interactive Visualization
ICIT '18: Proceedings of the 6th International Conference on Information Technology: IoT and Smart City

The ability to analyze and turn the data into useful information is becoming increasingly important to businesses as they need to quickly react to changing demands and needs. Data visualization has become progressively important to enable data to be ...
On the Impact of Explanations on Understanding of Algorithmic Decision-Making
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

Ethical principles for algorithms are gaining importance as more and more stakeholders are affected by "high-risk" algorithmic decision-making (ADM) systems. Understanding how these systems work enables stakeholders to make informed decisions and to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference

July 2020

2264 pages

ISBN:9781450369749

DOI:10.1145/3357236

General Chairs:
Ron Wakkary
Simon Fraser University, CA and Eindhoven University of Technology, NL
,
Kristina Andersen
Eindhoven University of Technology, NL
,
Program Chairs:
Will Odom
Simon Fraser University, CA
,
Audrey Desjardins
University of Washington, USA
,
Marianne Graves Petersen
Aarhus University, DK

Copyright � 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

JP Morgan Faculty Award
National Science Foundation

Conference

DIS '20

Sponsor:

SIGCHI

DIS '20: Designing Interactive Systems Conference 2020

July 6 - 10, 2020

Eindhoven, Netherlands

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,727
Total Downloads

Downloads (Last 12 months)363
Downloads (Last 6 weeks)39

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Meyer LZhu J(2024)Slide to Explore 'What If': An Analysis of Explainable InterfacesAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685416(1-6)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3677045.3685416
Song YLi CXing WLi SLee H(2024)A Fair Clustering Approach to Self-Regulated Learning Behaviors in a Virtual Learning EnvironmentProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636863(771-778)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636863
Yildirim NZlotnikov SSayar DKahn JBukowski LAmin SRiman KDavis BMinturn JKing ARicketts DTang LSivaraman VPerer APreum SMcCann JZimmerman J(2024)Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care UnitProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641896(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641896
Portela MCastillo CTolan SKarimi-Haghighi MPueyo A(2024)A comparative user study of human predictions in algorithm-supported recidivism risk assessmentArtificial Intelligence and Law10.1007/s10506-024-09393-yOnline publication date: 15-Mar-2024
https://doi.org/10.1007/s10506-024-09393-y
Awasthi PCortes CMansour YMohri M(2024)A Theory of�Learning with�Competing Objectives and�User FeedbackArtificial Intelligence and Image Analysis10.1007/978-3-031-63735-3_2(10-49)Online publication date: 23-Jul-2024
https://doi.org/10.1007/978-3-031-63735-3_2
Delgado FYang SMadaio MYang Q(2023)The Participatory Turn in AI Design: Theoretical Foundations and the Current State of PracticeProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization10.1145/3617694.3623261(1-23)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3617694.3623261
Johnston CVossler PBlessenohl SVayanos P(2023)Deploying a Robust Active Preference Elicitation Algorithm on MTurk: Experiment Design, Interface, and Evaluation for COVID-19 Patient PrioritizationProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization10.1145/3617694.3623254(1-10)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3617694.3623254
He WGordon MPopowski LBernstein M(2023)Cura: Curation at Social Media ScaleProceedings of the ACM on Human-Computer Interaction10.1145/36101867:CSCW2(1-33)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610186
Sipos LSch�fer UGlinka KM�ller-Birn C(2023)Identifying Explanation Needs of End-users: Applying and Extending the XAI Question BankProceedings of Mensch und Computer 202310.1145/3603555.3608551(492-497)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3608551
Feffer MSkirpan MLipton ZHeidari H(2023)From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future ResearchProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604661(38-48)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604661
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents