Evaluating Fairness Metrics

Irfan, Zahid and McCaffery, Fergal and Loughran, Roisin (2023) Evaluating Fairness Metrics. In: International Workshop on Algorithmic Bias in Search and Recommendation, 2 April 2023, Dublin.

PDF Download (695kB)

Abstract

Artificial Intelligence systems add significant value to decision-making. However, the systems must be fair because bias creeps into the system from sources like data and preprocessing algorithms. In this work, we explore fairness metrics discussing the shortfalls and benefits of each metric. The fairness metrics are demographic, statistical, and game theoretic. We find that the demographic fairness metrics are independent of the actual target value and hence have limited use. In contrast, the statistical fairness metrics can provide the thresholds to maximize fairness. The Minimax criterion was used to guide the search and help recommend the best model where the error among protected groups was minimum.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Artificial Intelligence; Fairness; Bias; Game Theory; Minimax; Pareto Front.
Subjects:	Computer Science
Research Centres:	Regulated Software Research Centre
Depositing User:	Sean McGreal
Date Deposited:	04 Sep 2023 08:58
Last Modified:	04 Sep 2023 08:58
License:	Creative Commons: Attribution-Noncommercial-Share Alike 4.0
URI:	https://eprints.dkit.ie/id/eprint/857

Actions (login required)

View Item

Downloads