The paper presents a new approach for efficiently enabling suspension in universal probabilistic programming languages (PPLs) that use Monte Carlo inference algorithms. Current state-of-the-art PPL implementations often rely on full continuation-passing style (CPS) transformations to enable suspension, which introduces overhead from unnecessary closure allocations.
The authors make the key observation that PPLs do not require arbitrary suspensions provided by full CPS transformations - most Monte Carlo inference algorithms only require suspension in specific parts of the program. They develop a static suspension analysis that determines the parts of the program that may require suspension for a given inference algorithm. This analysis then guides a selective CPS transformation, where only the necessary parts of the program are transformed to enable suspension.
The suspension analysis is formalized using a core PPL calculus and proven correct. The selective CPS transformation is also formalized and shown to significantly reduce runtime overhead compared to a full CPS transformation, while still enabling the suspensions required by the inference algorithm.
The authors implement the suspension analysis and selective CPS transformation in the Miking CorePPL compiler. They evaluate the approach on a range of real-world models and inference algorithms, demonstrating significant performance improvements across all cases.
To Another Language
from source content
arxiv.org
Deeper Inquiries