Continuous Probability Functions for Achieving Group and Individual Fairness in Predictive Models

Core Concepts

The authors propose a method for constructing continuous probability functions that satisfy both group fairness (equalised odds) and individual fairness constraints when post-processing the outputs of a predictive model.

Abstract

The authors address the shortcomings of the fixed randomisation approach for achieving group fairness, which can violate individual fairness. They derive a set of closed-form, continuous, and monotonic probability functions parameterized by group-specific thresholds and a probability parameter. These functions preserve group fairness, improve individual fairness, and maintain the predictive power of the underlying model. The key highlights and insights are: Fixed randomisation, while effective in satisfying group fairness, can lead to discontinuities that violate individual fairness and create disparities in individual odds between groups. The authors define a new notion of fairness called "equalised individual odds" to address this issue. They derive a family of continuous probability functions that satisfy both group fairness (equalised odds) and individual fairness constraints. The proposed functions are parameterized only by the group-specific thresholds and a probability parameter, making them easy to compute. The functions are constrained by a maximum derivative, preventing large shifts in classification odds for small changes in the input score and softening the threshold effect. The authors demonstrate the effectiveness of their approach through case studies on credit scoring for loan allocation and risk of recidivism prediction, showing improved individual fairness while preserving group fairness and accuracy.

Stats

"As credit score increases, the likelihood of defaulting decreases." "The rate of these changes, however, is correlated with race."

Quotes

"Fixed randomisation is an effective approach to build a classifier 𝑓ℎ𝑎based on a scoring function 𝑔that satisfies group fairness such as equalised odds. This strategy, however, exhibits a number of undesired properties." "We reconcile all this by constructing continuous probability functions between group thresholds that are constrained by their Lipschitz constant. Our solution preserves the model's predictive power, individual fairness and robustness while ensuring group fairness."

Key Insights Distilled From

Equalised Odds is not Equal Individual Odds: Post-processing for Group and Individual Fairness

by Edward A. Sm... at arxiv.org 04-22-2024

https://arxiv.org/pdf/2304.09779.pdf

Deeper Inquiries

How can the proposed continuous probability functions be extended to handle more complex protected attribute structures, such as intersections of multiple attributes

The proposed continuous probability functions can be extended to handle more complex protected attribute structures, such as intersections of multiple attributes, by incorporating additional thresholds and probabilities for each combination of attributes. For example, if we have multiple protected attributes like race and gender, we can define separate thresholds and probabilities for each subgroup (e.g., white males, white females, black males, black females, etc.). By creating a set of curves for each subgroup based on the specific combinations of attributes, we can ensure that the classification odds are fair and consistent across all intersecting groups. This approach allows for a more nuanced and comprehensive treatment of individuals belonging to different subgroups within the protected attribute structure.

What are the potential trade-offs between the different polynomial forms (linear, quadratic, cubic, 4th order) in terms of computational complexity, interpretability, and fairness guarantees

The potential trade-offs between the different polynomial forms (linear, quadratic, cubic, 4th order) in terms of computational complexity, interpretability, and fairness guarantees are as follows: Computational Complexity: Linear forms are the simplest and computationally least intensive, making them easier to implement and faster to compute. Quadratic, cubic, and 4th order polynomial forms introduce increasing levels of complexity and computational overhead due to the higher degree of the polynomials involved. Interpretability: Linear forms are more interpretable as they have a direct relationship between the input (credit score) and the output (probability of positive classification). Higher-order polynomial forms may be harder to interpret, especially when the curves become more complex and non-linear. Fairness Guarantees: While all polynomial forms can satisfy the fairness constraints, the higher-order polynomials offer more flexibility in capturing the nuances of the data distribution and ensuring individual and group fairness. The choice of polynomial form should be based on the specific requirements of the application, balancing between computational efficiency, interpretability, and fairness guarantees.

Can the framework be adapted to handle dynamic or evolving scoring functions, where the underlying model is updated over time

The framework can be adapted to handle dynamic or evolving scoring functions by incorporating mechanisms for continuous updates and adjustments. This can be achieved by implementing a feedback loop that monitors the performance of the scoring model over time and dynamically adjusts the thresholds and probabilities based on new data and feedback. Some ways to adapt the framework for dynamic scoring functions include: Implementing an automated re-calibration process that periodically updates the thresholds and probabilities based on the latest data. Introducing adaptive learning algorithms that can adjust the curves in real-time as the scoring function evolves. Incorporating mechanisms for feedback from users or domain experts to fine-tune the fairness constraints and ensure that the model remains up-to-date and aligned with the evolving requirements. By making the framework adaptable to changes in the underlying scoring function, it can maintain fairness guarantees and performance accuracy even as the model evolves over time.