awesome-duckdb

DuckDB toolkit

A curated collection of resources and libraries for using DuckDB in software development projects

🦆 A curated list of awesome DuckDB resources

GitHub
1k stars
39 watching
109 forks
last commit: about 21 hours ago
Linked from 1 awesome list

awesomeawesome-list

Screenshot of davidgasquez/awesome-duckdb website

davidgasquez.github.io/awesome-duckdb/

Awesome DuckDB / Resources
Official Documentation Official DuckDB documentation

Awesome DuckDB / Resources / Official Documentation
Official Blog Official DuckDB blog
DuckDB Clients Client APIs for DuckDB
DuckDB Documentation PDF The DuckDB documentation as a single PDF file

Awesome DuckDB / Resources
docker-duckdb 22 6 months ago Docker image for DuckDB CLI
DuckDB setup GitHub Action to install DuckDB in CI
Serverless DuckDB over S3 170 11 months ago Running DuckDB over a data lake on S3 using lambda
DuckDB snippets Collection of snippets curated by MotherDuck
DuckDB tldr page DuckDB's entry in , available in CLI via the command
DuckDB AWS Lambda layer 97 10 days ago Run DuckDB in AWS Lambda functions
Compatible DuckDB Extensions for AWS Lambda Extensions specifically compiled for the AWS Lambda runtime (GLIBC 2.26)
Serverless DuckDB as API 120 10 days ago Use DuckDB as API with Amazon API Gateway and AWS Lambda
Serverless Parquet Repartitioner 29 10 days ago Use DuckDB to repartition data in S3-based Data Lakes
Observable notebooks Notebooks using DuckDB on the Observable data visualization platform
duckdb-nf 2 10 months ago Example uses of DuckDB with Nextflow
DuckDB version manager (duckman) 14 16 days ago – Cross-platform installer and version manager for DuckDB
DuckERD CLI 70 2 months ago – A CLI tool to create an ER Diagram from DuckDB database files

Awesome DuckDB / Client APIs
C
C++
CLI
Go 725 7 days ago
Julia
Node.js
Python
R
Ruby 159 4 days ago
Rust 505 about 1 month ago
Swift
TypeScript
Wasm
ADBC
ODBC
.NET 437 7 days ago
Common Lisp 36 16 days ago
PowerShell 42 2 months ago
Dart 35 9 days ago

Awesome DuckDB / Tools Powered by DuckDB
Rill Developer 1,723 6 days ago Tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL
Ibis Project A DataFrame API for interacting with DuckDB (and other compute engines)
MotherDuck Serverless data warehouse powered by DuckDB
Boiling Data Serverless data analytics overlay on top of S3 Data Lakes
Hex Dataframe SQL Hex's Dataframe SQL cells are powered by DuckDB
Mode Mode uses DuckDB for their in-memory data engine
VulcanSQL DuckDB can be used as a caching layer or a data connector in VulcanSQL, a Data API framework for data folks to create REST APIs by writing SQL templates
Tad A fast, free, cross-platform tabular data viewer application powered by DuckDB
Honeycomb Maps A browser-based geospatial analysis tool leveraging DuckDB Wasm
Bauplan A serverless data transformation platform for data lakes
Malloy Malloy is an experimental language for describing data relationships and transformations. Malloy connects to BigQuery, Snowflake, Trino, and Postgres, and natively supports DuckDB
Evidence Generate reports using SQL and markdown. The DuckDB connector allows querying across DuckDB, csv, parquet and json
Latitude Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light
Census Census's dataset diffing for incremental syncs is powered by DuckDB
Huey 260 4 days ago Blazing-fast & intuitive pivot tables on .parquet, .csv, .json files and .duckdb tables in the browser based on DuckDB WASM. Open source (MIT). Zero install!
Parquet Explorer Visual Studio Code extension for exploring Parquet files with SQL, powered by DuckDB
DQOps Data quality platform for data engineers, data quality teams and data operations
DatalakeStudio 76 2 months ago Load, explore, transform your datasets and expose them via API. Integration with external APIs, S3, PostgreSQL and ChatGPT
Spice.ai 1,918 4 days ago A unified SQL query interface and portable runtime to locally materialize (using an embedded DuckDB), accelerate, and query datasets from any database, data warehouse, or data lake
Definite Definite pulls all your data into a single place for analytics and dashboards. No engineering or SQL required. Get a managed data warehouse (DuckDB), ELT, data modeling / transformations and BI in a single platform
Amphi ETL 904 6 days ago Low-code data pipelines for structured and unstructured data. SQL transformations are powered by DuckDB
Quackpipe 183 13 days ago Serverless OLAP API/UI built on top of DuckDB with basic ClickHouse API compatibility and Motherduck support
ParadeDB 6,218 6 days ago Postgres for Search and Analytics, powered by DuckDB-embedded-in-Postgres
Crunchy Bridge for Analytics Fully managed DBaaS based in Postgres integrated with DuckDB
UniverSQL 98 8 days ago An implementation of Snowflake API, enables running queries on Snowflake tables locally with DuckDB without a running warehouse
Whereabouts 20 21 days ago Fast, accurate, open-source geocoding in Python, using DuckDB
Phoenix Analytics 270 about 1 month ago Plug and play analytics for Phoenix applications, powered by DuckDB
sqlglot 6,745 4 days ago Python transpiler that translates between 23 different SQL dialects including DuckDB
yato 174 2 months ago The smallest DuckDB SQL orchestrator on Earth
SQLMesh 1,825 2 days ago A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally

Awesome DuckDB / Tools Powered by DuckDB / Web Clients
Online DuckDB Shell Online DuckDB shell powered by WebAssembly
SQL Workbench DuckDB-WASM based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs
Sekuel Playground Query your local parquet, csv, json. Your data will not be sent out of the device you are using
CSVFiddle Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone
Codapi Embed executable code snippets directly into your product documentation, online course or blog post
QuackDB Open-source online DuckDB SQL playground and editor
WhatTheDuck WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data
Sidequery Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data

Awesome DuckDB / Tools Powered by DuckDB / Backends
DuckDB API 40 11 days ago a TypeScript-based Docker image containing DuckDB, and a Hono framework REST API with JSON or streaming Arrow responses
Mosaic DuckDB Server A Python-based server that runs a local DuckDB instance and support queries over Web Sockets or HTTP, returning data in either Apache Arrow or JSON format
duckdb-server A Rust-based server that runs a local DuckDB instance and support queries over Web Sockets or HTTP/HTTPS, returning data in either Apache Arrow or JSON format

Awesome DuckDB / Libraries Powered by DuckDB
Mosaic An extensible framework for linking databases and interactive views
Fugue 2,013 2 months ago A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites
Splink A free Python library for fast, accurate data deduplication and record linkage
Simple-data-analysis 252 6 days ago Easy-to-use and high-performance JavaScript library for data analysis
pg_analytics 380 18 days ago PostgreSQL extension embedding DuckDB-in-Postgres for fast on-disk and remote object storage analytics from Postgres. Built as a Foreign Data Wrapper with full query pushdown to DuckDB
duckdb_fdw 340 18 days ago DuckDB Foreign Data Wrapper for PostgreSQL
@jetblack/duckdb-react 8 9 months ago A context manager for React and duckdb-wasm
QuackOSM 207 17 days ago A Python library for downloading and transforming raw OpenStreetMap data into GeoParquet files
PyGWalker A Pyhon library that turns your dataframe into an interactive UI for data visualization
https://github.com/DataZooDE/flapi] [ API Framework heavily relies on the power of DuckDB and DuckDB extensions. Ready to build performant and cost-efficient APIs on top of BigQuery or Snowflake for AI Agents and Data Apps

Awesome DuckDB / SQL Clients and IDE that Support DuckDB
Harlequin The DuckDB IDE for your terminal. ( )
qStudio A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results
DuckDB SQL Tools Free DuckDB SQL Tools for VS Code IDE. with advanced features
VSCode SQLTools (Free) Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support
DBeaver DBeaver is a universal database access and development tool that can be used to connect almost any type of database
DataGrip Paid SQL IDE by Jetbrains that supports many different database technologies, including DuckDB
Duckling 241 20 days ago A fast viewer for CSV/Parquet files and DuckDB/SQLite, based on Tauri
rsql 127 4 days ago CLI for DuckDB, LibSQL, MariaDB, MySQL, PostgreSQL, SQLite3 and SQL Server
jsqltranspiler 29 10 days ago Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL
jOOQ Type safe querying of DuckDB (and many other RDBMS) from Java. A from and to DuckDB is also available
SQL DATA LENS A lightweight, commercial SQL IDE that supports different DBMS, including DuckDB. The focus on performance and special DBMS features
Dataflare Simple easy-to-use database manager, supports DuckDB, PostgreSQL, MySQL, SQL Server, SQLite etc
manifold-sql (DuckDB for Java) 2,412 8 days ago Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support
Termux DuckB package) DuckDB CLI client for the Termux Android terminal emulator

Awesome DuckDB / Projects Powered by DuckDB
NBA Monte Carlo 451 5 days ago Monte Carlo simulation of the NBA season, leveraging Meltano, dbt, DuckDB and Evidence
Datadex 260 13 days ago Open source and local friendly data platform to collaborate on Open Data using DuckDB, Dagster, dbt, and Quarto
endoflife.date database Daily dumps of endoflife.date data
transfermarkt-datasets 247 6 days ago Curated football datasets from
duckDB-embedding-search 106 22 days ago A search engine for DuckDB that uses embedding vectors to find similar documents
DuckDB PyPI stats live dashboard ( ) - Live dashboard of PyPI downloads using DuckDB, dbt, Evidence and MotherDuck with code source to build your own
Specter 5 about 1 month ago Specter is a CLI tool to search and monitor Databricks audit logs

Awesome DuckDB / Integrations
dbt-duckdb 920 17 days ago DuckDB dbt adapter
data load tool - DuckDB destination Extract and load data from APIs to DuckDB using dlt
target-duckdb Load data to DuckDB based on Singer spec
Airbyte DuckDB destination Load data to DuckDB with Airbyte
Kestra DuckDB plugin Run queries with DuckDB to schedule data transformations and process automations, and run event-driven anomaly detection pipelines
SQLFlite - Arrow Flight SQL Server 205 about 2 months ago An example implementation of the that runs in a client-server setup with DuckDB or SQLite as backends
SQLFlow 69 7 months ago Enables SQL-based stream-processing, powered by DuckDB
nf-sqldb 26 16 days ago This plugin provides support for interacting with SQL databases in Nextflow scripts
MindsDB 26,793 6 days ago The platform for customizing AI from enterprise data. , making data from DuckDB accessible to a diverse range of AI/ML models
Sqlite2Duckdb 9 6 months ago A CLI tool to convert SQLite database to DuckDB
nodbi NoSQL Database Connector for R, providing a common API across Elasticsearch, CouchDB, MongoDB, SQLite, PostgreSQL, and DuckDB
duckplyr Drop-in replacement for dplyr in R that uses DuckDB for performance
kwack 79 6 days ago In-Memory Analytics for Kafka using DuckDB
PSDuckDB 42 2 months ago A PowerShell module for DuckDB integration
duckdb-tableau-connector 6 5 months ago DuckDB Tableau connector
duckdb-power-query-connector 45 about 1 month ago DuckDB Power Query Custom Connector
metabase_duckdb_driver 56 9 days ago Metabase DuckDB Driver shipped as 3rd party plugin

Awesome DuckDB / Extensions / Official Extensions
arrow 34 4 months ago A zero-copy data integration between Apache Arrow and DuckDB
aws 42 20 days ago For handling AWS credentials
azure 50 24 days ago For using the Azure Blob storage
delta 136 7 days ago For Delta Lake support
fts To support full text search
iceberg 158 about 1 month ago For reading Iceberg tables
inet 1 about 1 month ago For storing and handling IPv4 and IPv6 Internet addresses
mysql To read from and write to MySQL databases
postgres To read from and write to PostgreSQL databases
spatial Enables geospatial processing
sqlite To read from and write to SQLite databases
vss Add support for vector similarity search

Awesome DuckDB / Extensions / Community Extensions
chsql 40 14 days ago ClickHouse SQL Dialect macros for DuckDB
crypto 11 about 2 months ago Cryptographic hash functions and HMAC
duckpgq 80 6 days ago Graph workloads that supports the SQL/PGQ standard
evalexpr_rhai 14 9 days ago Evaluates the scripting language as part of SQL
fuzzycomplete 7 about 2 months ago Performs fuzzy string matching for autocompletion
h3 171 17 days ago Adds support for the H3 discrete global grid system
lindel 32 10 days ago Linearization/Delinearization, Z-Order, Hilbert and Morton Curves
prql 260 3 months ago Run PRQL commands directly within DuckDB
scrooge 129 2 months ago A set of aggregation functions and data scanners on financial data
shellfs 56 about 2 months ago Allows shell commands to be used for input and output
ulid 14 5 months ago ULID data type for DuckDB. A ULID is similar to a UUID except that it also contains a timestamp component
gsheets 129 5 days ago Read and write Google Sheets using SQL
httpserver 116 5 days ago DuckDB HTTP API Server and Query Interface

Awesome DuckDB / Extensions / Other Extensions
DuckDB Extension Radar 82 5 days ago Repository that contains DuckDB extensions on GitHub. Refreshed daily
duckdb-bigquery 61 4 days ago Enables seamless integration and querying of BigQuery datasets within DuckDB
duckdb-engine 355 6 days ago SQLAlchemy driver for DuckDB
duckdb-extension-template-zig 22 8 months ago A Zig & Nix toolkit template for building extensions against multiple versions of DuckDB using Zig, C or C++
duckdb-jfr-extension 5 about 1 month ago DuckDB extension to read JFR (Java Flight Recorder) files directly
duckdb-protobuf 17 2 months ago Plugin for querying encoded protobuf messages (both sequences and individual messages per file)
duckdb-pytables 83 7 months ago DuckDB extension to allow running SQL on arbitrary data sources
ERPL DuckDB SAP connector using RFC, ODP, or BICS
Kùzu 1,406 5 days ago Scan DuckDB tables in Kùzu, an embeddable property graph database management system
Lance 3,956 5 days ago Integrate Lance (modern columnar data format for ML implemented in Rust) with DuckDB
ODBC Scanner DuckDB Extension 79 about 1 year ago DuckDB extension to read data directly from databases supporting the ODBC interface
QDuckDB Plugin for reading DuckDB spatial tables in QGIS software
uc_catalog 52 16 days ago Proof-of-concept extension combining the extension with Unity Catalog
duckdb-flockmtl 65 5 days ago Integrate language model (LLM) capabilities directly into your queries and workflows
erpl-web 8 21 days ago ERPL WEB is a DuckDB extension that connects API-based ecosystems via standard interfaces like OData, GraphQL, and REST

Awesome DuckDB / Media / Talks
DuckDB: Crunching data anywhere from laptops to servers @ GOTO Amsterdam 2024 Gábor Szárnyas
DuckDB – Overview and latest developments @ DuckCon #5 Hannes Mühleisen and Mark Raasveldt
DuckCon #5 playlist
DuckCon #4 playlist
DuckCon #3 playlist
In-Process Analytical Data Management with DuckDB @ PyData Amsterdam Hannes Mühleisen
DuckDB: The Power of a Data Warehouse in your Python Process @ PyData Yerevan Gábor Szárnyas
DuckDB: Bringing analytical SQL directly to your Python shell @ EuroPython Pedro Holanda
DuckDB keynote @ Data + AI Summit 2023 Hannes Mühleisen
DuckDB: Bringing Analytical SQL Directly To Your Python Shell @ FOSDEM Pedro Holanda
State of the Duck @ DuckCon #2 Hannes Mühleisen & Mark Raasveldt
DuckDB Extensions @ DuckCon Pedro Holanda & Sam Ansmink
Developing Systems in Academia: The Good, the Bad, and the not-so-Ugly Duckling @ CIDR Hannes Mühleisen
DuckDB An Embeddable Analytical Database @ FOSDEM Hannes Mühleisen
DuckDB tutorials playlist by Learn Data with Mark Mark Needham
DuckDB tutorials playlist by MotherDuck Mehdi Ouazza
Nextflow and database uses: powering data engineering, exploring DuckDB, and beyond Edmund Miller
Why should you care about DuckDB? @ Dublin DuckDB meetup Mihai Bojin
Exploring Monte Carlo Simulations With DuckDB @ Dublin DuckDB meetup James McNeill
DuckDB and recommenders : a lightning fast synergy @ Dublin DuckDB meetup Khalil Muhammad

Awesome DuckDB / Media / Podcasts
Developer Voices: Implementing Hardware-Friendly Databases Hannes Mühleisen
The Geek Narrator: DuckDB Internals Mark Raasveldt
Software Engineering Daily: DuckDB Hannes Mühleisen
Data Engineering Podcast: Move Your Database To The Data And Speed Up Your Analytics With DuckDB Hannes Mühleisen
The Analytics Engineering Podcast: The Personal Data Warehouse Jordan Tigani

Awesome DuckDB / Media / Blog Posts
Modern Data Stack in a Box Fast, free, and open-source Modern Data Stack deployed on a laptop using the combination of DuckDB, Meltano, dbt, and Apache Superset
How to use DuckDB, Motherduck and Kestra for ETL How DuckDB can transform data, mask sensitive PII information, detect anomalies in event-driven workflows, and streamline reporting use cases
DuckDB vs. MotherDuck — how do they compare What are key differences between them, and when to choose each of these options
Building DuckDB Extensions with Zig and Nix For Nix users and Zig developers familiar with DuckDB looking to extend its capabilities with custom extensions
Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts
DuckDB: The Rising Star in the Big Data Landscape
How to Make a DuckDB Extension for a Table Function? How to make a DuckDB extension to fetch data from external sources
Putting DuckDB in Postgres to Query Iceberg How ParadeDB embedded DuckDB in Postgres to achieve fast analytics and Apache Iceberg compatibility from Postgres

Awesome DuckDB / Media / Books
DuckDB in Action DuckDB in Action will show you how to quickly get your hands dirty with DuckDB
Getting Started with DuckDB A practical guide for accelerating your data science, data analytics, and data engineering workflows

Backlinks from these awesome lists:

More related projects: