odbc_begin_transactionodbc_bind_paramsodbc_closeodbc_commitodbc_connectodbc_copyodbc_create_paramsodbc_list_data_sourcesodbc_list_driversodbc_queryodbc_rollback
odbc_begin_transaction
odbc_begin_transaction(conn_handle BIGINT) -> VARCHAR
Sets the SQL_ATTR_AUTOCOMMIT attribute to SQL_AUTOCOMMIT_OFF on the specified connection thus effectively starting an implicit transaction. odbc_commit or odbc_rollback must be called on such connection to complete the transaction. The completion starts another implicit transaction on this connection. See Transactions management for details.
Parameters:
conn_handle(BIGINT): ODBC connection handle created withodbc_connect
Returns:
Always returns NULL (VARCHAR).
Example:
SELECT odbc_begin_transaction(getvariable('conn'))
odbc_bind_params
odbc_bind_params(conn_handle BIGINT, params_handle BIGINT, params STRUCT) -> BIGINT
Binds specified parameter values to the specified parameters handle. Only necessary with 2-step parameters binding, see Query parameters for details.
Parameters:
conn_handle(BIGINT): ODBC connection handle created withodbc_connectparams_handle(BIGINT): parameters handle created withodbc_create_paramsparams(STRUCT): parameters values
Returns:
Parameters handle (BIGINT), the same one that was passed as a second argument.
Example:
SELECT odbc_bind_params(getvariable('conn'), getvariable('params1'), row(42, 'foo'))
odbc_close
odbc_close(conn_handle BIGINT) -> VARCHAR
Closes specified ODBC connection to a remote DB. Does not throw errors if the connection is already closed.
Parameters:
conn_handle(BIGINT): ODBC connection handle created withodbc_connect
Returns:
Always returns NULL (VARCHAR).
Example:
SELECT odbc_close(getvariable('conn'))
odbc_commit
odbc_commit(conn_handle BIGINT) -> VARCHAR
Calls SQLEndTran with SQL_COMMIT argument on the specified connection, completing the current transaction. odbc_begin_transaction must be called on this connection before this call for the completion to be effective. See Transactions management for details.
Parameters:
conn_handle(BIGINT): ODBC connection handle created withodbc_connect
Returns:
Always returns NULL (VARCHAR).
Example:
SELECT odbc_commit(getvariable('conn'))
odbc_connect
odbc_connect(conn_string VARCHAR) -> BIGINT
odbc_connect(conn_string VARCHAR, username VARCHAR, password VARCHAR) -> BIGINT
Opens an ODBC connection to a remote DB.
If username and password (positional) parameters are specified, they are appended to the connection string as UID and PWD.
Parameters:
conn_string(VARCHAR): ODBC connection string, passed to the Driver Manager.
Returns:
Connection handle that can be placed into a VARIABLE. Connection is not closed automatically, must be closed with odbc_close.
Example:
SET VARIABLE conn = odbc_connect('Driver={Oracle Driver};DBQ=//127.0.0.1:1521/XE;UID=scott;PWD=tiger')
SET VARIABLE conn = odbc_connect('Driver={Oracle Driver};DBQ=//127.0.0.1:1521/XE', 'scott', 'tiger')
odbc_copy
odbc_copy(conn_handle BIGINT, [, <optional named parameters>]) -> TABLE
odbc_copy(conn_string VARCHAR, [, <optional named parameters>]) -> TABLE
Copies rows from a DuckDB accessible file or table into the remote DB.
Warning Using
odbc_copyfrom Python Relational API.
odbc_copyis a table function that returns one row for each 2048 copied rows. When it is used from Python withduckdb.sql()the Lazy Evaluation is taking place. Thus no rows will be copied until a method that triggers execution is called on the resulting relation and all result rows are consumed.
Parameters:
conn_handle_or_string(BIGINTorVARCHAR), one of:- ODBC connection handle created with
odbc_connect - ODBC connection string, intended for one-off queries, in this case new ODBC connection will be opened and will be closed automatically after the query is complete
- ODBC connection handle created with
Optional named parameters (source):
Source query is executed using a separate DB instance from the instance on which
odbc_copyis being called. Thussource_querycannot refer to pre-existing in-memory tables and cannot open currently opened DuckDB files. As a workaround, for complex source queries it is suggested to export the query result into a local Parquet file first and then runodbc_copyon that file.
source_conn_string(VARCHAR, default::memory:): DuckDB connection string to the source DB, example:ducklake:postgres:postgresql://username:[email protected]:5432/lake1source_file(VARCHAR): path to a Parquet, CSV or JSON file (remote or local) to be read with DuckDB, example:https://blobs.duckdb.org/nl_stations.csv, equivalent tosource_query='SELECT * FROM '<source_file>'source_query(VARCHAR): DuckDB SQL query to read the data, example:FROM nl_train_stationssource_queries(LIST(VARCHAR)): multiple DuckDB SQL queries executed one by one, last query must return the result set to copy, results of previous queries are discarded, results of all queries are materialized in memory, example:
source_queries=[
'CREATE SECRET s (TYPE s3 [...])',
'FROM nl_train_stations'
],
source_limit(UBIGINT, default:0): the number of records to read from source query/file at once, when this option is specified - the source query is run multiple times appendingLIMIT <limit> OFFSET <offset>to it, must be more or equal to2048,2048must be dividable by it without a remainder
Optional named parameters (destination):
dest_table(VARCHAR): destination table name in remote DB, will be used inINSERTandCREATE TABLEqueries, cannot be specified ifdest_queryis specified; different DBs have different rules regarding case sensitivity and the default case, thus the name of the destination table may need to be specified in upper-case:TAB1or in quoted form:"tab1"(or with a schema name:"schema1"."tab1")dest_query(VARCHAR): query to be executed in remote DB for each source batch, must have the number of ODBC parameter placeholders?equal to thesource_columns_count * batch_size, cannot be specified ifdest_tableis specified, example:CALL import_city(?,?,?,?)dest_query_single(VARCHAR): only used when thebatch_size>0and the number of rows read in the last source batch are less than thebatch_size, in this case used instead ofdest_query, must have the number of ODBC parameter placeholders?equal to thesource_columns_count
Optional named parameters (create table):
create_table(BOOLEAN, default:FALSE): whether to create a table in the destination remote DB using the column names and column types from the source query, effectively implements CTAS (create table as select)column_types(MAP(VARCHAR, VARCHAR)): whencreate_table=TRUEis specified, allows to provide/override the type mapping between source DuckDB types and destination RDBMS types, example:
create_table=TRUE,
column_types=MAP {
'DUCKDB_TYPE_VARCHAR': 'VARCHAR2(10)',
'DUCKDB_TYPE_DECIMAL': 'NUMBER({typmod1},{typmod2})'}
column_quotes(VARCHAR, default:"): quotation character (or string) to be used to quote column names in the generatedCREATE TABLEandINSERTqueriescommit_after_create_table(BOOLEAN, default:FALSE): whether to issue aCOMMITafter executingCREATE TABLE, enabled automatically for Firebird
Optional named parameters (query parameters handling):
decimal_params_as_chars(BOOLEAN, default:false): passDECIMALparameters asVARCHARsintegral_params_as_decimals(BOOLEAN, default:false): pass (unsigned)TINYINT,SMALLINT,INTEGERandBIGINTparameters asSQL_C_NUMERIC.
Optional named parameters (other):
batch_size(UINTEGER, default:16): number of records to be inserted (or executed in case ofdest_query) in a singleSQLExecuteODBC call to remote DB, allowed values:1,2,4,8,16,32,64,128,256,512,1024,2048use_insert_all(BOOLEAN, default:FALSE): generateINSERT ALLbatch insert query instead of batch insert withINSERT ... VALUES (...), (...), ... (...), enabled automatically for Oracleuse_insert_union(BOOLEAN, default:FALSE): generateINSERT ... SELECT FROM .. UNION ALL ...batch insert query instead of batch insert withINSERT ... VALUES (...), (...), ... (...), enabled automatically for Firebirddummy_table_name(VARCHAR): name of the dummy table use forINSERT ALLandINSERT UNIONqueries,dualfor Oraclecopy_in_transaction(BOOLEAN, default:TRUE): begin a transaction in remote DB for this copy call, commit transaction when all rows are processed, roll it back on errormax_records_in_transaction(UBIGINT, default:0): when specified causes the remote transaction to be committed every time after the specified number of rows is processedclose_connection(BOOLEAN, default:false): closes the passed connection after the function call is completed, intended to be used with one-shot invocations of theodbc_copy
Returns:
A table with the following columns:
completed(BOOLEAN): a flag whether this output row is the last row in result setrows_processed(UBIGINT): a number of rows read from the sourceelapsed_seconds(FLOAT): a number of seconds passed after the copy process has startedrows_per_second(FLOAT): a number of rows processed in one secondtable_ddl(VARCHAR): generatedCREATE TABLEquery that was executed in remote DB before starting the copy process
One resulting row is emitted for every 2048 rows read from source. Only the last row has the completed=TRUE and non null table_ddl (only when create_table=TRUE is specified) values.
Examples:
FROM odbc_copy(getvariable('conn'),
source_file='https://blobs.duckdb.org/nl_stations.csv',
dest_table='NL_TRAIN_STATIONS',
create_table=TRUE)
FROM odbc_copy(getvariable('conn'),
source_conn_string='ducklake:postgres:postgresql://username:[email protected]:5432/lake1',
source_queries=[
'CREATE SECRET s (TYPE s3 [...])',
'FROM nl_train_stations'
],
dest_table='NL_TRAIN_STATIONS',
create_table=TRUE,
batch_size=32,
max_records_in_transaction=42);
odbc_create_params
odbc_create_params() -> BIGINT
Creates a parameters handle. Only necessary with 2-step parameters binding, see Query parameters for details.
Parameters:
None.
Returns:
Parameters handle (BIGINT). When the handle is passed to odbc_query it gets tied to the underlying prepared statement and is closed automatically when the statement is closed.
Example:
SET VARIABLE params1 = odbc_create_params()
odbc_list_data_sources
odbc_list_data_sources() -> TABLE(name VARCHAR, description VARCHAR, type VARCHAR)
Returns the list of ODBC data sources registered in the OS. Uses driver manager call SQLDataSources.
Parameters:
None.
Returns:
A table with the following columns:
name(VARCHAR): data source namedescription(VARCHAR): data source descriptiontype(VARCHAR): data source type,USERorSYSTEM
Example:
FROM odbc_list_data_sources()
odbc_list_drivers
odbc_list_drivers() -> TABLE(description VARCHAR, attributes MAP(VARCHAR, VARCHAR))
Returns the list of ODBC drivers registered in the OS. Uses driver manager call SQLDrivers.
Parameters:
None.
Returns:
A table with the following columns:
description(VARCHAR): driver descriptionattributes(MAP(VARCHAR, VARCHAR)): driver attributes as aname->valuemap
Example:
FROM odbc_list_drivers()
odbc_query
odbc_query(conn_handle BIGINT, query VARCHAR[, <optional named parameters>]) -> TABLE
odbc_query(conn_string VARCHAR, query VARCHAR[, <optional named parameters>]) -> TABLE
Runs specified query in a remote DB and returns the query results table.
Parameters:
conn_handle_or_string(BIGINTorVARCHAR), one of:- ODBC connection handle created with
odbc_connect - ODBC connection string, intended for one-off queries, in this case new ODBC connection will be opened and will be closed automatically after the query is complete
- ODBC connection handle created with
query(VARCHAR): SQL query, passed to the remote DBMS
Optional named parameters that can be used to pass query parameters:
params(STRUCT): query parameters to pass to remote DBMSparams_handle(BIGINT): parameters handle created withodbc_create_params. Only used with 2-step parameters binding, see Query parameters for details.
Optional named parameters that can change types mapping:
The extension supports a number of options that can be used to change how the query parameters are passed and how the resulting data is handled. For known DBs these options are set automatically. They also can be passed as named parameters to odbc_query function to override the autoconfiguration:
decimal_columns_as_chars(BOOLEAN, default:false): readDECIMALvalues asVARCHARs that are parsed back intoDECIMALs before returning them to clientdecimal_columns_precision_through_ard(BOOLEAN, default:false): when reading aDECIMALspecify itsprecisionandscalethrough "Application Row Descriptor"decimal_columns_as_ard_type(BOOLEAN, default:false): when reading aDECIMALuseSQL_ARD_TYPEinstead ofSQL_C_NUMERICdecimal_params_as_chars(BOOLEAN, default:false): passDECIMALparameters asVARCHARsintegral_params_as_decimals(BOOLEAN, default:false): pass (unsigned)TINYINT,SMALLINT,INTEGERandBIGINTparameters asSQL_C_NUMERIC.reset_stmt_before_execute(BOOLEAN, default:false): reset the prepared statement (usingSQLFreeStmt(h, SQL_CLOSE)) before executing ittime_params_as_ss_time2(BOOLEAN, default:false): passTIMEparameters as SQL Server'sTIME2valuestimestamp_columns_as_timestamp_ns(BOOLEAN, default:false): readTIMESTAMP-like (TIMESTAMP WITH LOCAL TIME ZONE,DATETIME2,TIMESTAMP_NTZetc) columns with nanosecond precision (with nine fractional digits)timestamp_columns_with_typename_date_as_date(BOOLEAN, default:false): readTIMESTAMPcolumns that have a type nameDATEas DuckDBDATEstimestamp_max_fraction_precision(UTINYINT, default:9): maximum number of fractional digits to use when reading aTIMESTAMPcolumn with nanosecond precisiontimestamp_params_as_sf_timestamp_ntz(BOOLEAN, default:false): passTIMESTAMPparameters as Snowflake'sTIMESTAMP_NTZtimestamptz_params_as_ss_timestampoffset(BOOLEAN, default:false): passTIMESTAMP_TZparameters as SQL Server'sDATETIMEOFFSETvar_len_data_single_part(BOOLEAN, default:false): read longVARCHARorVARBINARYvalues as a single read (used when a driver does not support Retrieving Variable-Length Data in Parts)var_len_params_long_threshold_bytes(UINTEGER, default:4000): a length threshold after thatSQL_WVARCHARparameters are passed asSQL_WLONGVARCHARenable_columns_binding(BOOLEAN, default:false): whether to allow usingSQLBindColinstead ofSQLGetDatafor fixed-size columns
Other optional named parameters:
ignore_exec_failure(BOOLEAN, default:false): when a query, that is run in remote DB, can be prepared successfully, but may or may not fail at execution time (for example, because of schema state like table existence), then this flag can be used to not throw an error when query execution fails. Empty result set is returned if query execution fails.close_connection(BOOLEAN, default:false): closes the passed connection after the function call is completed, intended to be used with one-shot invocations of theodbc_query, example:
FROM odbc_query(
odbc_connect('Driver={Oracle Driver};DBQ=//127.0.0.1:1521/XE', 'scott', 'tiger'),
'SELECT 42 FROM dual',
close_connection=TRUE);
Returns:
A table with the query result.
Example:
FROM odbc_query(getvariable('conn'),
'SELECT CAST(? AS NVARCHAR2(2)) || CAST(? AS VARCHAR2(5)) FROM dual',
params=row('🦆', 'quack')
)
odbc_rollback
odbc_rollback(conn_handle BIGINT) -> VARCHAR
Calls SQLEndTran with SQL_ROLLBACK argument on the specified connection, completing the current transaction. odbc_begin_transaction must be called on this connection before this call for the completion to be effective. See Transactions management for details.
Parameters:
conn_handle(BIGINT): ODBC connection handle created withodbc_connect
Returns:
Always returns NULL (VARCHAR).
Example:
SELECT odbc_rollback(getvariable('conn'))