@flowblade/source-duckdb
Install
npm i @flowblade/source-duckdb @flowblade/core @flowblade/sql-tag @duckdb/node-api
Quick start
Create a duckdb instance
import { type DuckDBConnection, DuckDBInstance } from '@duckdb/node-api';
import { DuckdbDatasource } from '@flowblade/source-duckdb';
// Create a connection to a DuckDB instance
const createConnection = async (): Promise<DuckDBConnection> => {
const instance = await DuckDBInstance.create(':memory:', {
// Choose between READ_ONLY or READ_WRITE
// Note that in READ_WRITE mode concurrency is limited to 1
// See: https://duckdb.org/docs/connect/concurrency.html
access_mode: 'READ_ONLY',
max_memory: '64MB',
});
return await instance.connect();
};
const duckdb = await createConnection();
// Create a duckdb datasource
export const ds = new DuckdbDatasource({ connection: duckdb });
Query the database
import { DuckdbDatasource, sql } from '@flowblade/source-duckdb';
import { ds } from "./config.ts";
const params = {
min: 10,
max: 99,
name: 'test',
createdAt: new Date().toISOString(),
};
type Row = { id: number; name: 'test'; createdAt: Date };
const rawSql = sql<Row>`
WITH products(productId, createdAt)
AS MATERIALIZED (
FROM RANGE(1,100) SELECT
range::INT,
TIMESTAMPTZ '2025-01-01 12:30:00.123456789+01:00'
)
SELECT productId,
${params.name} as name,
createdAt
FROM products
WHERE productId BETWEEN ${params.min}::INTEGER AND ${params.max}::INTEGER
AND createdAt < ${params.createdAt}::TIMESTAMPTZ
`;
const result = await ds.query(rawSql);
// Option 1: The QResult object contains the data, metadata and error
// - data: the result rows (TData or undefined if error)
// - error: the error (QError or undefined if success)
// - meta: the metadata (always present)
const { data, meta, error } = result;
// Option 2: You operate over the result, ie: mapping the data
const { data } = result.map((row) => {
return {
id: row.productId,
key: `key-${row.productId}`
})
if (data) {
console.log(data);
}
Compatibility
Level | CI | Description |
---|---|---|
Node | ✅ | CI for 18.x, 20.x & 22.x. |
Cloudflare | ✅ | Ensured with @cloudflare/vitest-pool-workers (see wrangler.toml |
Browserslist | ✅ | > 95% on 01/2025. Chrome 96+, Firefox 90+, Edge 19+, ios 15+, Safari 15+ and Opera 77+ |
Typescript | ✅ | TS 5.0 + / are-the-type-wrong checks on CI. |
ES2022 | ✅ | Dist files checked with es-check |
Performance | ✅ | Monitored with codspeed.io |
Last updated on