Perform rate and burst limit on filesystem operations
Maintainer(s):
dentiny
Compatibility Matrix
This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
|---|---|---|---|---|---|
| Linux AMD64 | |||||
| Linux ARM64 | |||||
| macOS AMD64 | |||||
| macOS ARM64 | |||||
| Windows AMD64 | |||||
| Windows ARM64 | |||||
| WebAssembly |
Installing and Loading
INSTALL rate_limit_fs FROM community;
LOAD rate_limit_fs;
Example
SELECT rate_limit_fs_wrap('RateLimitFsFakeFileSystem');
SELECT rate_limit_fs_quota('RateLimitFileSystem - RateLimitFsFakeFileSystem', 'read', 1000000, 'blocking');
About rate_limit_fs
This extension implements rate and burst limit for duckdb filesystems. It supports a few key features:
- Support filesystem and operation based limit.
- Support burst limit and GCRA-based rate limit.
- Supports wrap custom filesystems, so users don't need to change their query.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| rate_limit_fs_burst | scalar | Sets the burst capacity for read or write operations on a rate-limited filesystem. | NULL | [SELECT rate_limit_fs_burst('RateLimitFileSystem - LocalFileSystem', 'read', 10485760);] |
| rate_limit_fs_clear | scalar | Clears one or more rate-limit settings; '*' selects every filesystem or operation. | NULL | [SELECT rate_limit_fs_clear('RateLimitFileSystem - LocalFileSystem', 'read');] |
| rate_limit_fs_configs | table | Returns all configured filesystem operation quotas, modes, burst limits, and concurrency caps. | NULL | [SELECT * FROM rate_limit_fs_configs();] |
| rate_limit_fs_list_filesystems | table | Lists the filesystem implementations registered with DuckDB's virtual filesystem. | NULL | [SELECT * FROM rate_limit_fs_list_filesystems();] |
| rate_limit_fs_max_requests | scalar | Sets the maximum number of concurrent requests for a filesystem operation, or -1 for unlimited. | NULL | [SELECT rate_limit_fs_max_requests('RateLimitFileSystem - LocalFileSystem', 'read', 10);] |
| rate_limit_fs_quota | scalar | Sets the sustained rate limit for a filesystem operation in blocking or non-blocking mode. | NULL | [SELECT rate_limit_fs_quota('RateLimitFileSystem - LocalFileSystem', 'read', 1048576, 'blocking');] |
| rate_limit_fs_wrap | scalar | Wraps a registered filesystem with rate limiting and registers the wrapped filesystem. | NULL | [SELECT rate_limit_fs_wrap('LocalFileSystem');] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases | typed_value |
|---|---|---|---|---|---|
| temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |