Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors

Niels Van Berkel; Jorge Goncalves; Daniel Russo; Simo Hosio; Mikael B. Skov

doi:10.1145/3411764.3445365

Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors

Niels Van Berkel, Jorge Goncalves, Daniel Russo, Simo Hosio, Mikael B. Skov

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

36 Citations (Scopus)

Abstract

The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people’s fairness perception and that the presented scenario, as well as the participant’s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.

Original language	English
Title of host publication	CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems : Making Waves, Combining Strengths
Number of pages	13
Publisher	Association for Computing Machinery
Publication date	6 May 2021
Article number	245
ISBN (Electronic)	978-1-4503-8096-6
DOIs	https://doi.org/10.1145/3411764.3445365
Publication status	Published - 6 May 2021
Event	ACM CHI 2021 conference on human factors in computing systems - Online virtual, Yokohama, Japan Duration: 8 May 2021 → 13 May 2021 https://chi2021.acm.org/

Conference

Conference	ACM CHI 2021 conference on human factors in computing systems
Location	Online virtual
Country/Territory	Japan
City	Yokohama
Period	08/05/2021 → 13/05/2021
Internet address	https://chi2021.acm.org/

Keywords

Ai
Artifcial intelligence
Crowdsourcing
Fairness
Layperson
Machine learning
Ml
Predictor selection
Transparency
Visualisation

Access to Document

10.1145/3411764.3445365

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

Van Berkel, N., Goncalves, J., Russo, D., Hosio, S., & Skov, M. B. (2021). Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors. In CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths Article 245 Association for Computing Machinery. https://doi.org/10.1145/3411764.3445365

@inproceedings{b744c02bf112475b845368309f3bf4aa,

title = "Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors",

abstract = "The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people{\textquoteright}s fairness perception and that the presented scenario, as well as the participant{\textquoteright}s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.",

keywords = "Ai, Artifcial intelligence, Crowdsourcing, Fairness, Layperson, Machine learning, Ml, Predictor selection, Transparency, Visualisation",

author = "{Van Berkel}, Niels and Jorge Goncalves and Daniel Russo and Simo Hosio and Skov, {Mikael B.}",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; ACM CHI 2021 conference on human factors in computing systems ; Conference date: 08-05-2021 Through 13-05-2021",

year = "2021",

month = may,

day = "6",

doi = "10.1145/3411764.3445365",

language = "English",

booktitle = "CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery",

address = "United States",

url = "https://chi2021.acm.org/",

}

Van Berkel, N, Goncalves, J, Russo, D, Hosio, S & Skov, MB 2021, Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors. in CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths., 245, Association for Computing Machinery, ACM CHI 2021 conference on human factors in computing systems, Yokohama, Japan, 08/05/2021. https://doi.org/10.1145/3411764.3445365

Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors. / Van Berkel, Niels; Goncalves, Jorge; Russo, Daniel et al.
CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths. Association for Computing Machinery, 2021. 245.

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

TY - GEN

T1 - Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors

AU - Van Berkel, Niels

AU - Goncalves, Jorge

AU - Russo, Daniel

AU - Hosio, Simo

AU - Skov, Mikael B.

PY - 2021/5/6

Y1 - 2021/5/6

N2 - The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people’s fairness perception and that the presented scenario, as well as the participant’s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.

AB - The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people’s fairness perception and that the presented scenario, as well as the participant’s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.

KW - Ai

KW - Artifcial intelligence

KW - Crowdsourcing

KW - Fairness

KW - Layperson

KW - Machine learning

KW - Ml

KW - Predictor selection

KW - Transparency

KW - Visualisation

UR - http://www.scopus.com/inward/record.url?scp=85106725896&partnerID=8YFLogxK

U2 - 10.1145/3411764.3445365

DO - 10.1145/3411764.3445365

M3 - Article in proceeding

AN - SCOPUS:85106725896

BT - CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

T2 - ACM CHI 2021 conference on human factors in computing systems

Y2 - 8 May 2021 through 13 May 2021

ER -

Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors

Abstract

Conference

Keywords

Access to Document

AUB Link

Other files and links

Fingerprint

Cite this