Search Shortcut cmd + k | ctrl + k
duck_block_utils

Build, transform, validate, and extract content from structured documents using the duck_block type

Maintainer(s): teaguesterling

Compatibility Matrix

This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

Platform 1.4.4 1.4.5 1.5.5 1.5.6 2.0.0
Linux AMD64
Linux ARM64
macOS AMD64
macOS ARM64
Windows AMD64
Windows ARM64
WebAssembly

Installing and Loading

INSTALL duck_block_utils FROM community;
LOAD duck_block_utils;

Example

-- Build a document programmatically
SELECT duck_blocks_assemble([
    duck_block_heading(1, 'Hello World'),
    duck_block_paragraph([
        duck_block_text('This is '),
        duck_block_bold('important'),
        duck_block_text(' content.')
    ]),
    duck_block_code('sql', 'SELECT * FROM documents')
]);

-- Render it as plain text
SELECT duck_blocks_to_text(duck_blocks_assemble([
    duck_block_heading(1, 'Hello World'),
    duck_block_paragraph('Some prose.')
]));

-- Works with the markdown extension for reading and writing files
LOAD markdown;
LOAD duck_block_utils;

-- Table of contents from a markdown file
SELECT * FROM duck_blocks_toc_rows(
    (SELECT list(b) FROM read_markdown_blocks('README.md') b)
);

-- Validate document structure
SELECT duck_blocks_validate(
    (SELECT list(b) FROM read_markdown_blocks('doc.md') b)
);

About duck_block_utils

The Duck Block Utils extension provides tools for building, transforming, validating, and extracting content from structured documents. All functions use the duck_block type - a unified representation for document elements that works across formats.

Documentation: duck-block-utils.readthedocs.io

2.0.0 is a breaking rename

Every db_* function was renamed to duck_block_* (builders) or duck_blocks_* (operations on a list). 54 names present in 1.6.1 are gone and there is no alias path, so 1.6.1 queries must be updated. The mapping is mechanical:

db_heading(1, 'Title')      ->  duck_block_heading(1, 'Title')
db_assemble(blocks)         ->  duck_blocks_assemble(blocks)
db_blocks_to_text(blocks)   ->  duck_blocks_to_text(blocks)
db_page                     ->  page_break

The duck_block Type

A unified struct for block-level, inline and metadata-value elements:

STRUCT(
    kind VARCHAR,                       -- 'block', 'inline' or 'value'
    element_type VARCHAR,               -- 'heading', 'paragraph', 'bold', 'link', etc.
    content VARCHAR,                    -- text, iff the element has a single text child
    level INTEGER,                      -- depth in a depth-first ordering; top is 1 (0 is the
                                        --   optional explicit document root)
    encoding VARCHAR,                   -- 'text', 'json', 'yaml', 'html', 'xml',
                                        --   'latex', 'markdown', 'toml'
    attributes MAP(VARCHAR, VARCHAR),   -- element-specific metadata
    element_order INTEGER               -- position in the document
)

level is structural depth, never a heading rank - a heading's rank lives in attributes['heading_level']. content is populated if and only if an element has a single text child; a heading additionally keeps a flattened title alongside its rich children.

Builders

All builder functions return LIST(duck_block) so they compose by nesting:

SELECT duck_block_paragraph([
    duck_block_text('Click '),
    duck_block_link('https://example.com', 'here'),
    duck_block_text(' for more.')
]);
Function Description
duck_block_heading(level, content) Heading (h1-h6)
duck_block_paragraph(content) Paragraph
duck_block_code(language, content) Fenced code block
duck_block_blockquote(content) Block quote
duck_block_list_block(ordered, items[]) Ordered or unordered list
duck_block_hr() Horizontal rule
duck_block_image(src, alt, title) Image
duck_block_metadata(content) Metadata blob (YAML frontmatter)
duck_block_raw(format, content) Raw content block
duck_block_text(content) Plain text
duck_block_bold(content) Bold
duck_block_italic(content) Italic
duck_block_link(href, content) Hyperlink
duck_block_inline_code(content) Inline code
duck_block_math(content) Math
duck_block_strikethrough(content) Strikethrough
duck_block_superscript(content) / duck_block_subscript(content) Super/subscript

Assembly

Function Description
duck_blocks_assemble(blocks[]) Combine blocks into a document
duck_blocks_document(blocks[]) Alias for assemble
duck_block_section(level, title, children[]) Section with a heading
duck_blocks_concat(a, b) Concatenate block lists
duck_blocks_rebase_levels(blocks, offset) Shift heading levels

Query and extraction

Function Description
duck_blocks_to_text(blocks) Plain text
duck_blocks_headings(blocks) Heading hierarchy
duck_blocks_toc_rows(blocks) Table of contents, one row per entry
duck_blocks_get_section(blocks, pattern) One section by title or id
duck_blocks_sections_like(blocks, term) Sections whose text matches
duck_blocks_page_rows(blocks) Page boundaries, separate from the outline
duck_blocks_get_pages(blocks, first, last) A page range
duck_blocks_links(blocks) Every link
duck_blocks_stats(blocks) / duck_blocks_structure(blocks) Document shape
duck_blocks_diff(before, after) ADDED / REMOVED / MOVED between two versions

Validation

Function Description
duck_blocks_validate(blocks) Spec conformance: valid, plus errors
duck_blocks_repair(blocks) Deterministic fixes for the list-level rules: wrap orphans, rebase, renumber
duck_block_implicit_parent(type, kind) The wrapper a fragment of that element gets, from the spec table
duck_block_is_body(kind, element_type) Per-row half of the body rule: block or inline, and not the metadata blob
duck_blocks_body(blocks) The body rows only: no value or metadata subtree, element_order kept
duck_blocks_lint(blocks) Advisory conformance warnings
duck_blocks_quality(blocks) Document quality, which is NOT conformance
duck_block_spec_version() The spec version this build implements

Rendering

Function Description
duck_blocks_render_ansi(blocks, width) Terminal rendering with themes

Pandoc integration

Convert between Pandoc's JSON AST and duck_block without invoking Pandoc:

Function Description
pandoc_ast_to_blocks(json) Parse a Pandoc AST
duck_blocks_to_pandoc_ast(blocks) Emit a full Pandoc document
duck_blocks_to_pandoc_blocks(blocks) Emit just the block list
read_pandoc_ast(path) / write_pandoc_ast(path, blocks) Read and write AST files

Ecosystem

  • duckdb_markdown - read_markdown_blocks(), and COPY ... TO ... (FORMAT markdown, MARKDOWN_MODE duck_block)
  • duckdb_webbed - HTML parsing and generation
-- Read markdown, append to it, write it back
LOAD markdown;
LOAD duck_block_utils;

COPY (
    SELECT unnest(duck_blocks_concat(
        (SELECT list(b) FROM read_markdown_blocks('doc.md') b),
        duck_blocks_assemble([
            duck_block_hr(),
            duck_block_paragraph('Generated by DuckDB')
        ])
    )) AS block
) TO 'output.md' (FORMAT markdown, MARKDOWN_MODE duck_block);

Added Functions

function_name function_type description comment examples
duck_block scalar Construct a duck_block struct. NULL [duck_block('paragraph', 'Hello world'), duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1)]
duck_block_aliases pragma NULL NULL  
duck_block_attr scalar Get an attribute value by key from a duck_block struct. NULL [duck_block_attr(duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1), 'id')]
duck_block_blockquote scalar NULL NULL  
duck_block_bold scalar NULL NULL  
duck_block_cite scalar NULL NULL  
duck_block_code scalar NULL NULL  
duck_block_content scalar Get the content field of a duck_block struct. NULL [duck_block_content(duck_block('paragraph', 'Hello'))]
duck_block_div scalar NULL NULL  
duck_block_doc_macros pragma NULL NULL  
duck_block_encoding scalar Get the encoding field of a duck_block struct. NULL [duck_block_encoding(duck_block('paragraph', 'Hello'))]
duck_block_encoding_names scalar Return the list of valid duck_block encoding names. NULL [duck_block_encoding_names()]
duck_block_ensure_extension scalar Ensure specified extension is loaded. NULL [duck_block_ensure_extension('json')]
duck_block_heading scalar NULL NULL  
duck_block_hr scalar Build a horizontal rule block (V2). NULL [duck_block_hr()]
duck_block_image scalar NULL NULL  
duck_block_implicit_parent scalar Return the implicit parent element type for an element type. NULL [duck_block_implicit_parent('block', 'paragraph')]
duck_block_inline_code scalar Build an inline code element from text (V2). NULL [duck_block_inline_code('foo()')]
duck_block_inline_image scalar NULL NULL  
duck_block_is_body scalar Return true if the element kind and type represent body content. NULL [duck_block_is_body('block', 'paragraph')]
duck_block_italic scalar NULL NULL  
duck_block_kind_names scalar Return the list of valid duck_block kind names. NULL [duck_block_kind_names()]
duck_block_level scalar Get the level field of a duck_block struct. NULL [duck_block_level(duck_block('paragraph', 'Hello'))]
duck_block_linebreak scalar Build a hard line break inline element (V2). NULL [duck_block_linebreak()]
duck_block_link scalar NULL NULL  
duck_block_list scalar NULL NULL  
duck_block_list_block scalar NULL NULL  
duck_block_list_item scalar NULL NULL  
duck_block_math scalar NULL NULL  
duck_block_metadata scalar Build a document metadata block from YAML string (V2). NULL [duck_block_metadata('title: Document
author: Me')]        
duck_block_note scalar NULL NULL  
duck_block_order scalar Get the element_order field of a duck_block struct. NULL [duck_block_order(duck_block('paragraph', 'Hello'))]
duck_block_paragraph scalar NULL NULL  
duck_block_plain scalar NULL NULL  
duck_block_quoted scalar NULL NULL  
duck_block_raw scalar NULL NULL  
duck_block_raw_inline scalar NULL NULL  
duck_block_render pragma NULL NULL  
duck_block_section scalar NULL NULL  
duck_block_set_content scalar Return a copy of the duck_block struct with content updated. NULL [duck_block_set_content(duck_block('paragraph', 'Hello'), 'World')]
duck_block_set_level scalar Return a copy of the duck_block struct with level updated. NULL [duck_block_set_level(duck_block('paragraph', 'Hello'), 2)]
duck_block_set_order scalar Return a copy of the duck_block struct with element_order updated. NULL [duck_block_set_order(duck_block('paragraph', 'Hello'), 2)]
duck_block_smallcaps scalar NULL NULL  
duck_block_softbreak scalar Build a soft line break inline element (V2). NULL [duck_block_softbreak()]
duck_block_space scalar Build a whitespace space inline element (V2). NULL [duck_block_space()]
duck_block_span scalar NULL NULL  
duck_block_spec_version scalar Return the current duck_blocks specification version. NULL [duck_block_spec_version()]
duck_block_strikethrough scalar NULL NULL  
duck_block_subscript scalar NULL NULL  
duck_block_superscript scalar NULL NULL  
duck_block_terminal_width scalar Detect current terminal width in columns. NULL [duck_block_terminal_width()]
duck_block_text scalar Build a text inline element (V2). NULL [duck_block_text('Hello world')]
duck_block_type scalar Get the element_type field of a duck_block struct. NULL [duck_block_type(duck_block('paragraph', 'Hello'))]
duck_block_type_names scalar Return the list of valid duck_block element type names. NULL [duck_block_type_names()]
duck_block_underline scalar NULL NULL  
duck_block_valid scalar Check if a duck_block struct is valid according to the duck_blocks spec. NULL [duck_block_valid(duck_block('paragraph', 'Hello'))]
duck_blocks_assemble scalar NULL NULL  
duck_blocks_body scalar Extract body blocks excluding non-body/metadata elements. NULL [duck_blocks_body(blocks)]
duck_blocks_code_blocks scalar Extract code blocks from a block list. NULL [duck_blocks_code_blocks(blocks)]
duck_blocks_code_blocks_structs scalar Extract code blocks as list of structs. NULL [duck_blocks_code_blocks_structs(blocks)]
duck_blocks_concat scalar Concatenate two block lists. NULL [duck_blocks_concat(blocks1, blocks2)]
duck_blocks_diff table_macro NULL NULL  
duck_blocks_document scalar NULL NULL  
duck_blocks_exclude scalar Exclude specified element types from a list of duck_blocks. NULL [duck_blocks_exclude(blocks, ['raw', 'metadata'])]
duck_blocks_filter scalar Filter a list of duck_blocks to only include specified element types. NULL [duck_blocks_filter(blocks, ['heading', 'paragraph'])]
duck_blocks_get_pages macro NULL NULL  
duck_blocks_get_pages_text macro NULL NULL  
duck_blocks_get_section macro NULL NULL  
duck_blocks_get_section_text macro NULL NULL  
duck_blocks_headings scalar Extract heading blocks from a block list. NULL [duck_blocks_headings(blocks)]
duck_blocks_headings_structs scalar Extract headings as list of structs. NULL [duck_blocks_headings_structs(blocks)]
duck_blocks_inlines_to_pandoc scalar NULL NULL  
duck_blocks_links scalar Extract links from a block list. NULL [duck_blocks_links(blocks)]
duck_blocks_links_structs scalar Extract links as list of structs. NULL [duck_blocks_links_structs(blocks)]
duck_blocks_lint scalar Lint a block list for structural style and issues. NULL [duck_blocks_lint(blocks)]
duck_blocks_merge scalar Merge two lists of duck_blocks and renumber element orders. NULL [duck_blocks_merge(blocks1, blocks2)]
duck_blocks_normalize scalar Normalize block hierarchy by collapsing lone plain blocks into parent. NULL [duck_blocks_normalize(blocks)]
duck_blocks_page_rows table_macro NULL NULL  
duck_blocks_quality table_macro NULL NULL  
duck_blocks_rebase_levels scalar Rebase hierarchy levels of a list of blocks. NULL [duck_blocks_rebase_levels(blocks, 1)]
duck_blocks_render_ansi scalar NULL NULL  
duck_blocks_reorder scalar Renumber element_order sequentially from 1 for a list of duck_blocks. NULL [duck_blocks_reorder(blocks)]
duck_blocks_repair scalar Repair and normalize block structure and levels. NULL [duck_blocks_repair(blocks)]
duck_blocks_sections_like table_macro NULL NULL  
duck_blocks_sections_like_text table_macro NULL NULL  
duck_blocks_slice scalar Slice a list of duck_blocks by 1-based index range. NULL [duck_blocks_slice(blocks, 1, 5)]
duck_blocks_stamp scalar Stamp duck_blocks list with current spec version metadata. NULL [duck_blocks_stamp(blocks)]
duck_blocks_stats scalar Compute summary statistics of elements in block list. NULL [duck_blocks_stats(blocks)]
duck_blocks_structure scalar Analyze structural composition of a block list. NULL [duck_blocks_structure(blocks)]
duck_blocks_to_match_text macro NULL NULL  
duck_blocks_to_pandoc_ast scalar Convert duck_blocks to complete Pandoc AST struct. NULL [duck_blocks_to_pandoc_ast(blocks)]
duck_blocks_to_pandoc_blocks scalar Convert duck_blocks to Pandoc blocks JSON string. NULL [duck_blocks_to_pandoc_blocks(blocks)]
duck_blocks_to_text scalar NULL NULL  
duck_blocks_toc scalar Extract table of contents as bullet list blocks. NULL [duck_blocks_toc(blocks)]
duck_blocks_toc_rows table_macro NULL NULL  
duck_blocks_toc_structs scalar Extract table of contents as list of structs. NULL [duck_blocks_toc_structs(blocks)]
duck_blocks_validate scalar Validate structural correctness of block list. NULL [duck_blocks_validate(blocks)]
duck_blocks_version scalar Return the stamped spec version of a duck_blocks list. NULL [duck_blocks_version(blocks)]
pandoc_ast table Table function returning Pandoc AST rows. NULL [SELECT * FROM pandoc_ast(blocks)]
pandoc_ast_to_blocks scalar Parse Pandoc AST JSON into duck_blocks. NULL [pandoc_ast_to_blocks('{"blocks":[]}')]
pandoc_inlines_to_db_inlines scalar Parse Pandoc inlines JSON into duck_blocks. NULL [pandoc_inlines_to_db_inlines('[{"t":"Str","c":"hi"}]')]
pandoc_inlines_to_text scalar NULL NULL  
read_pandoc_ast scalar Read Pandoc AST JSON file into duck_blocks. NULL [read_pandoc_ast('document.json')]
to_duck_block scalar Convert a compatible struct to a duck_block struct. NULL [to_duck_block({'element_type': 'paragraph', 'content': 'Hello', 'level': 1, 'encoding': 'text', 'attributes': MAP {}, 'element_order': 1})]
write_pandoc_ast scalar Write duck_blocks to file as Pandoc AST JSON. NULL [write_pandoc_ast('out.json', blocks)]

Overloaded Functions

This extension does not add any function overloads.

Added Types

type_name type_size logical_type type_category internal
duck_block 0 STRUCT COMPOSITE true
duck_block_ext 0 STRUCT COMPOSITE true

Added Settings

name description input_type scope aliases typed_value
temp_directory Set the directory to which temporary files are written. VARCHAR GLOBAL [] post.db.tmp