Smart DuckLake table maintain (adaptive rewrite + merge) and catalog-global expire/cleanup
Maintainer(s):
paulosuzart
Compatibility Matrix
This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
|---|---|---|---|---|---|
| Linux AMD64 | |||||
| Linux ARM64 | |||||
| macOS AMD64 | |||||
| macOS ARM64 | |||||
| Windows AMD64 | |||||
| Windows ARM64 | |||||
| WebAssembly |
Installing and Loading
INSTALL lakemon FROM community;
LOAD lakemon;
Example
INSTALL ducklake;
LOAD ducklake;
INSTALL lakemon FROM community;
LOAD lakemon;
ATTACH 'ducklake:metadata.ducklake' AS lake (DATA_PATH 'data_files');
CALL lakemon_table_stats('lake');
CALL lakemon_maintain('lake');
CALL lakemon_maintain_global('lake');
CALL lakemon_maintain('lake', dry_run => true);
CALL lakemon_policy();
About lakemon
lakemon orchestrates native DuckLake maintenance CALLs with a policy layer:
- Catalog SELECT inventory of data files, delete files, and inlined-delete backlog
- Flush guard (lakemon_flush_inlined_max_delete_files) so a large ended-file delete backlog cannot block rewrite/merge
- Byte-weighted adaptive rewrite ladder (delete fraction DESC; one ducklake_rewrite_data_files CALL per byte-cut rung)
- Merge size bands (micro / small / medium) with per-tier target_file_size
- Table maintain (lakemon_maintain) vs catalog-global expire/cleanup (lakemon_maintain_global)
- CALL / SELECT plan and execute by default (dry_run for plan only)
- One result row per step (ok / skip / error / planned)
- Optional stderr step logs with duration and byte-weighted progress
Requires the core ducklake extension in the same session and an attached DuckLake catalog. Compatible with DuckDB 1.5.* and DuckDB 2.x.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| lakemon_maintain | pragma | NULL | NULL | |
| lakemon_maintain | table | NULL | NULL | |
| lakemon_maintain_global | pragma | NULL | NULL | |
| lakemon_maintain_global | table | NULL | NULL | |
| lakemon_policy | table | NULL | NULL | |
| lakemon_set_policy | table | NULL | NULL | |
| lakemon_table_stats | table | NULL | NULL | |
| lakemon_version | scalar | NULL | NULL |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases | typed_value |
|---|---|---|---|---|---|
| lakemon_flush_inlined_max_delete_files | Skip ducklake_flush_inlined_data when the planned delete-file count exceeds this (0 = no limit). Negatives are rejected. | BIGINT | GLOBAL | [] | 500 |
| lakemon_log_stderr | Write lakemon maintain step start/end lines to stderr | BOOLEAN | GLOBAL | [] | false |
| temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |