Core Concepts
The core message of this paper is to develop a robust framework called Rashomon Partitions to estimate and analyze heterogeneity in factorial data, where the outcome of interest varies with combinations of observable covariates. The proposed approach enumerates a set of high posterior probability partitions that offer substantively different explanations for the heterogeneity, allowing for robust conclusions that are not overly sensitive to the choice of a single "optimal" partition.
Abstract
The paper addresses the problem of estimating heterogeneity in factorial data, where the outcome of interest varies with combinations of observable covariates. Existing approaches either search for a single "optimal" partition under assumptions about covariate associations or attempt to sample from the entire set of possible partitions, both of which ignore the reality that many partitions may be statistically indistinguishable despite offering very different implications.
The authors develop an alternative framework called Rashomon Partition Sets (RPSs), which enumerates all partitions that have posterior values near the maximum a posteriori partition. This allows for robust conclusions that incorporate all high posterior probability models, even if they offer substantively different explanations.
Key aspects of the RPS framework:
Uses a prior (the ℓ0 prior) that makes no assumptions about the associations between covariates, making it robust to the complex marginal effects in factorial settings.
Provides bounds on the approximation error of the posterior distribution restricted to the RPS relative to the full posterior.
Characterizes the size of the RPS in terms of the number of features, values per feature, and the prior over the number of distinct pools.
Develops an algorithm to efficiently enumerate the full RPS.
The authors demonstrate the usefulness of the RPS framework through simulation experiments and three empirical applications: price effects on charitable giving, heterogeneity in chromosomal structure, and the introduction of microfinance. The RPS approach allows them to make robust conclusions, including affirmations and reversals of findings from existing literature.