Evaluating the Impact of Situation Testing on SMOTE-Based Sampling Techniques for Bias Mitigation

Heidarpour-Shahrezei, Maliheh and McDaid, Kevin and Loughran, Roisin (2025) Evaluating the Impact of Situation Testing on SMOTE-Based Sampling Techniques for Bias Mitigation. In: 33rd International Conference on Artificial Intelligence and Cognitive Science (AICS 2025), December 1st & 2nd, 2025, Dublin, Republic of Ireland. (Submitted)

PDF Download (358kB)

Abstract

Sampling bias can be mitigated through preprocessing techniques such as oversampling and undersampling in imbalanced datasets. However, they do not address labeling bias, which can reinforce unfair model behavior. To address this, we established a structured evaluation framework that applies Situation Testing as an additional step after six SMOTE-based sampling techniques. Across 10 datasets and multiple classifiers, our findings reveal that the number of biased samples removed by Situation Testing depends on the dataset, classi-fier, and preprocessing sampling technique applied beforehand. Applying Situa-tion Testing directly to the baseline consistently improved fairness with respect to both Demographic Parity and Equalized Odds, albeit with reduced predictive performance. In contrast, the impact of Situation Testing after sampling varied across strategies, proving to be more effective in group size adjustments based on Equalized Representation than in balanced-size techniques. Overall, the re-sults highlight the trade-off between fairness gains and predictive utility, under-scoring the need to align mitigation strategies with dataset characteristics and fairness objectives.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Bias; Fairness; Mitigation Techniques; Situation Testing.
Subjects:	Computer Science
Research Centres:	Regulated Software Research Centre
Depositing User:	Sean McGreal
Date Deposited:	05 Jan 2026 12:41
Last Modified:	05 Jan 2026 12:42
License:	Creative Commons: Attribution-Noncommercial-Share Alike 4.0
URI:	https://eprints.dkit.ie/id/eprint/993

Actions (login required)

View Item

Downloads