Secure remote secrets storage with OPAQUE PAKE authentication, email/password registration, MFA login, and automatic ducklake mounting
Maintainer(s):
dforsber
Installing and Loading
INSTALL boilstream FROM community;
LOAD boilstream;
Example
-- Setup: Install and load boilstream extension
INSTALL httpfs; LOAD httpfs;
INSTALL boilstream FROM community;
LOAD boilstream;
-- Option 1: Login with email/password/MFA (establishes OPAQUE session)
-- NOTE: Remember to clean up your ~/.duckdb/history
PRAGMA boilstream_login('https://api.example.com/[email protected]', 'password', '123456');
┌────────────────────────┬─────────────────────┬────────────────────┐
│ status │ expires_at │ ducklakes_attached │
├────────────────────────┼─────────────────────┼────────────────────┤
│ Session token obtained │ 2025-11-21 10:30:00 │ 2 │
└────────────────────────┴─────────────────────┴────────────────────┘
-- Option 2: Authenticate with bootstrap token (from SSO/web UI)
PRAGMA boilstream_bootstrap_session('https://api.example.com/secrets:TOKEN');
┌────────────────────────┬─────────────────────┬────────────────────┐
│ status │ expires_at │ ducklakes_attached │
├────────────────────────┼─────────────────────┼────────────────────┤
│ Session token obtained │ 2025-11-21 10:30:00 │ 2 │
└────────────────────────┴─────────────────────┴────────────────────┘
-- List all available ducklakes
FROM boilstream_ducklakes();
┌──────────────┬──────────────┬─────────────────┐
│ catalog_id │ catalog_name │ description │
├──────────────┼──────────────┼─────────────────┤
│ 5a805647-... │ my_analytics │ Sales analytics │
└──────────────┴──────────────┴─────────────────┘
-- List all cached secrets with expiration
FROM boilstream_secrets();
┌─────────────────────┬──────────┬─────────────────────┐
│ name │ type │ expires_at │
├─────────────────────┼──────────┼─────────────────────┤
│ my_analytics │ ducklake │ 2026-11-19 20:00:00 │
│ my_analytics_s3 │ s3 │ 2026-11-19 20:00:00 │
│ my_analytics_pg │ postgres │ 2026-11-19 20:00:00 │
└─────────────────────┴──────────┴─────────────────────┘
-- Create new ducklake
PRAGMA boilstream_create_ducklake('new_catalog', 'My new ducklake');
-- Query ducklake (automatically attached)
USE my_analytics;
SHOW TABLES;
About boilstream
Boilstream extension provides enterprise-grade remote secrets storage for multi-tenant DuckDB deployments.
Features:
- 🔐 OPAQUE PAKE Authentication - Password-less authentication (IETF RFC 9807)
- 🔒 End-to-end Encryption - AES-256-GCM/ChaCha20-Poly1305 for secrets and responses
- 🔑 Cryptographic Key Derivation - HKDF-SHA256 with domain separation
- 🛡️ Defense in Depth - HMAC verification before AEAD decryption
- 🎯 Session Protection - Lock-step sequence protocol prevents hijacking
- 🗄️ Ducklake Support - Auto-attach multi-tenant data catalogs
- 🌐 SSO Integration - Bootstrap token exchange for web environments
Security:
- Facebook's audited OPAQUE implementation (Rust + C++ FFI)
- Protects against MITM attacks even if TLS is compromised
- One-time bootstrap tokens with anonymous authentication
- Session resumption with encrypted refresh tokens
- Comprehensive test vectors and security specification
API Functions:
PRAGMA boilstream_register_user(url_with_email, password)- Register new user with MFA (displays QR code)PRAGMA boilstream_verify_mfa(totp_code)- Complete registration and get backup codesPRAGMA boilstream_login(url_with_email, password, mfa_code)- Email/password/MFA loginPRAGMA boilstream_bootstrap_session(url_with_token)- Authenticate with bootstrap tokenboilstream_ducklakes()- List available ducklakesboilstream_secrets()- List cached secrets with expirationboilstream_buckets()- List available storage bucketsPRAGMA boilstream_create_ducklake(name, description)- Create new ducklakePRAGMA boilstream_help- Show all available commands
Requirements:
- httpfs extension (auto-loaded)
- postgres_scanner extension (auto-loaded)
- ducklake extension (auto-loaded)
- Compatible REST API server (see API specification)
For detailed documentation, security specification, and setup instructions, visit the extension repository.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| boilstream_bootstrap_session | pragma | NULL | NULL | |
| boilstream_buckets | table | NULL | NULL | |
| boilstream_create_ducklake | pragma | NULL | NULL | |
| boilstream_ducklakes | table | NULL | NULL | |
| boilstream_help | pragma | NULL | NULL | |
| boilstream_login | pragma | NULL | NULL | |
| boilstream_register_user | pragma | NULL | NULL | |
| boilstream_secrets | table | NULL | NULL | |
| boilstream_verify_mfa | pragma | NULL | NULL | |
| ducklake_add_data_files | table | NULL | NULL | |
| ducklake_cleanup_old_files | table | NULL | NULL | |
| ducklake_current_snapshot | table | NULL | NULL | |
| ducklake_delete_orphaned_files | table | NULL | NULL | |
| ducklake_expire_snapshots | table | NULL | NULL | |
| ducklake_flush_inlined_data | table | NULL | NULL | |
| ducklake_last_committed_snapshot | table | NULL | NULL | |
| ducklake_list_files | table | NULL | NULL | |
| ducklake_merge_adjacent_files | table | NULL | NULL | |
| ducklake_options | table | NULL | NULL | |
| ducklake_rewrite_data_files | table | NULL | NULL | |
| ducklake_scan | table | NULL | NULL | |
| ducklake_set_commit_message | table | NULL | NULL | |
| ducklake_set_option | table | NULL | NULL | |
| ducklake_settings | table | NULL | NULL | |
| ducklake_snapshots | table | NULL | NULL | |
| ducklake_table_changes | table_macro | NULL | NULL | |
| ducklake_table_deletions | table | NULL | NULL | |
| ducklake_table_info | table | NULL | NULL | |
| ducklake_table_insertions | table | NULL | NULL | |
| murmur3_32 | scalar | NULL | NULL | |
| pg_clear_cache | table | NULL | NULL | |
| postgres_attach | table | NULL | NULL | |
| postgres_configure_pool | table | NULL | NULL | |
| postgres_execute | table | NULL | NULL | |
| postgres_query | table | NULL | NULL | |
| postgres_scan | table | NULL | NULL | |
| postgres_scan_pushdown | table | NULL | NULL | |
| read_postgres_binary | table | NULL | NULL |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| ducklake_default_data_inlining_row_limit | Default row limit for data inlining (0 disables inlining) | UBIGINT | GLOBAL | [] |
| ducklake_max_retry_count | The maximum amount of retry attempts for a ducklake transaction | UBIGINT | GLOBAL | [] |
| ducklake_retry_backoff | Backoff factor for exponentially increasing retry wait time | DOUBLE | GLOBAL | [] |
| ducklake_retry_wait_ms | Time between retries | UBIGINT | GLOBAL | [] |
| ducklake_write_deletion_vectors | [EXPERIMENTAL] Write Iceberg V3 deletion vectors (puffin) instead of positional delete files (parquet) | BOOLEAN | GLOBAL | [] |
| pg_array_as_varchar | Read Postgres arrays as varchar - enables reading mixed dimensional arrays | BOOLEAN | GLOBAL | [] |
| pg_connection_cache | Whether or not to use the connection pooling. This option is deprecated, instead to disable the connection pooling use "SET pg_pool_max_connections=0" for newly attached Postgres databases and "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', max_connections=0)" for already attached Postgres databases. | BOOLEAN | GLOBAL | [] |
| pg_connection_limit | The maximum amount of concurrent Postgres connections. This option is deprecated, instead use "SET pg_pool_max_connections = 42" for newly attached Postgres databases and "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', max_connections=42)" for already attached Postgres databases. | UBIGINT | GLOBAL | [] |
| pg_debug_show_queries | DEBUG SETTING: print all queries sent to Postgres to stdout | BOOLEAN | GLOBAL | [] |
| pg_experimental_filter_pushdown | Whether or not to use filter pushdown | BOOLEAN | GLOBAL | [] |
| pg_idle_in_transaction_timeout_millis | Postgres idle in transaction timeout in milliseconds to set on scan connections | UINTEGER | GLOBAL | [] |
| pg_null_byte_replacement | When writing NULL bytes to Postgres, replace them with the given character | VARCHAR | GLOBAL | [] |
| pg_pages_per_task | The amount of pages per task | UBIGINT | GLOBAL | [] |
| pg_pool_enable_reaper_thread | Whether to enable the connection pool reaper thread, that periodically scans the pool to check the 'max_lifetime_millis' and 'idle_timeout_millis' and closes the connection which exceed the specified values. Either 'max_lifetime_millis' or 'idle_timeout_millis' must be set to a non-zero value for this option to be effective. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', enable_reaper_thread=TRUE)" | BOOLEAN | GLOBAL | [] |
| pg_pool_enable_thread_local_cache | Whether to enable the connection caching in thread-local cache. Such connections are getting pinned to the threads and are not made available to other threads, while still taking the place in the pool. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', enable_thread_local_cache=FALSE)" | BOOLEAN | GLOBAL | [] |
| pg_pool_health_check_query | The query that is used to check that the connection is healthy. Setting this option to an empty string disables the health check. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', health_check_query=SELECT 42)" | VARCHAR | GLOBAL | [] |
| pg_pool_idle_timeout_millis | Maximum number of milliseconds the connection can be kept idle in the pool. This number is checked when the connection is taken from the pool. When the connection pool reaper thread is enabled ('pg_pool_enable_reaper_thread' option), then this number is checked in background periodically. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', idle_timeout_millis=300000)" | UBIGINT | GLOBAL | [] |
| pg_pool_max_connections | Maximum number of connections that are allowed to be cached in a connection pool for each attached Postgres database. This number can be temporary exceeded when parallel scans are used. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', max_connections=42)" | UBIGINT | GLOBAL | [] |
| pg_pool_max_lifetime_millis | Maximum number of milliseconds the connection can be kept open. This number is checked when the connection is taken from the pool and returned to the pool. When the connection pool reaper thread is enabled ('pg_pool_enable_reaper_thread' option), then this number is checked in background periodically. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', max_lifetime_millis=600000)" | UBIGINT | GLOBAL | [] |
| pg_pool_wait_timeout_millis | Maximum number of milliseconds to wait when acquiring a connection from a pool where all available connections are already taken. This option only applies to newly attached Postgres databases, to configure a database that is already attached use "FROM postgres_configure_pool(catalog_name='my_attached_postgres_db', wait_timeout_millis=60000)" | UBIGINT | GLOBAL | [] |
| pg_statement_timeout_millis | Postgres statement timeout in milliseconds to set on scan connections | UINTEGER | GLOBAL | [] |
| pg_use_binary_copy | Whether or not to use BINARY copy to read data | BOOLEAN | GLOBAL | [] |
| pg_use_ctid_scan | Whether or not to parallelize scanning using table ctids | BOOLEAN | GLOBAL | [] |
| pg_use_text_protocol | Whether or not to use TEXT protocol to read data. This is slower, but provides better compatibility with non-Postgres systems | BOOLEAN | GLOBAL | [] |