Core Concepts
OpenIVM is a SQL-to-SQL compiler that enables incremental maintenance of materialized views by propagating changes in base tables to the views using SQL.
Abstract
The paper presents OpenIVM, a new open-source SQL-to-SQL compiler for Incremental View Maintenance (IVM). The key ideas are:
Leverage existing SQL query processing engines to perform all IVM computations via SQL, rather than implementing IVM functionality in a separate system. This enables integration of IVM in these systems without code duplication.
Support cross-system IVM, where one DBMS (e.g., OLTP) provides insertions/updates/deletes (deltas) that are propagated using SQL into another DBMS (e.g., OLAP) hosting materialized views.
Use the DBSP framework to rewrite relational operators into their incremental forms, generating SQL statements that propagate deltas into the materialized view table.
Implement OpenIVM as a DuckDB extension module that adds IVM functionality to DuckDB, and demonstrate cross-system IVM with PostgreSQL handling updates on base tables and DuckDB hosting materialized views.
The paper discusses the DBSP principles, the SQL-to-SQL compiler architecture, and the integration with DuckDB. It also outlines future work on extending the supported relational operators and optimization strategies.