The paper explores the sources of inequality in reinforcement learning (RL) problems through a causal lens. It first formulates the RL problem using a structural causal model to capture the agent-environment interaction. The authors then provide a quantitative decomposition of the well-being gap, which measures the difference in long-term returns between demographic groups.
The key contributions are:
Introduction of a novel fairness notion called "dynamics fairness" that captures the fairness of the underlying mechanisms governing the environment. This is distinct from fairness in decision-making or inherited from the past.
Derivation of identification formulas to quantitatively evaluate dynamics fairness without making parametric assumptions about the environment.
Demonstration through experiments that the proposed decomposition accurately explains the sources of inequality and the effectiveness of the dynamics fairness-aware algorithm in promoting long-term fairness.
The paper systematically examines the intricacies of inequality in RL, offering insights into the causal paths responsible for the well-being gap. It highlights the importance of considering the environmental dynamics when studying long-term fairness, beyond just decision-making or historical disparities.
Til et andet sprog
fra kildeindhold
arxiv.org
Vigtigste indsigter udtrukket fra
by Zhihong Deng... kl. arxiv.org 04-18-2024
https://arxiv.org/pdf/2404.10942.pdfDybere Forespørgsler