New Metric Assesses Fairness in Private Machine Learning.

This research proposes a novel group fairness criterion called the Privacy-Cost Equity Ratio (PCER) for machine learning systems that employ differential privacy (DP). The core argument is that privacy cost, or the information leakage experienced by different demographic groups, constitutes a form of harm that should be factored into fairness assessments, rather than solely focusing on outcome disparities like accuracy gaps. PCER is derived from a compensatory-fairness framework, suggesting that groups bearing greater privacy exposure should receive proportionally greater benefits from the system. The metric is defined as a group's positive prediction rate normalized by its per-group overfitting gap, which serves as a practical proxy for vulnerability to membership inference attacks. This allows PCER to be used as a post-hoc audit tool, requiring only per-group train and test accuracy. Evaluations across various benchmarks and privacy budgets demonstrate that PCER uncovers fairness issues that traditional outcome-based metrics often miss. For instance, on the COMPAS dataset, PCER revealed a persistent double disadvantage for a protected group, experiencing both higher privacy exposure and worse predictive outcomes, a pattern obscured by demographic parity gap. The findings emphasize the necessity of considering privacy cost as an equity input when auditing privacy-preserving AI systems.