awesome-duckdb

DuckDB toolkit

A curated collection of DuckDB-related resources and tools for developers

🦆 A curated list of awesome DuckDB resources

GitHub
1k stars
41 watching
111 forks
last commit: 4 months ago
Linked from 1 awesome list

awesomeawesome-list

Screenshot of davidgasquez/awesome-duckdb website

davidgasquez.github.io/awesome-duckdb/

Awesome DuckDB / Resources
Official Documentation Official DuckDB documentation

Awesome DuckDB / Resources / Official Documentation
Official Blog Official DuckDB blog
DuckDB Clients Client APIs for DuckDB
DuckDB Documentation PDF The DuckDB documentation as a single PDF file
DuckDB Documentation MD The DuckDB documentation as a single Markdown file

Awesome DuckDB / Resources
docker-duckdb 22 11 months ago Docker image for DuckDB CLI
DuckDB setup GitHub Action to install DuckDB in CI
Serverless DuckDB over S3 178 over 1 year ago Running DuckDB over a data lake on S3 using lambda
DuckDB snippets Collection of snippets curated by MotherDuck
DuckDB tldr page DuckDB's entry in , available in CLI via the command
DuckDB AWS Lambda layer 98 5 months ago Run DuckDB in AWS Lambda functions
Compatible DuckDB Extensions for AWS Lambda Extensions specifically compiled for the AWS Lambda runtime (GLIBC 2.26)
Serverless DuckDB as API 122 5 months ago Use DuckDB as API with Amazon API Gateway and AWS Lambda
Serverless Parquet Repartitioner 29 5 months ago Use DuckDB to repartition data in S3-based Data Lakes
Observable notebooks Notebooks using DuckDB on the Observable data visualization platform
duckdb-nf 2 about 1 year ago Example uses of DuckDB with Nextflow
DuckDB version manager (duckman) 15 6 months ago – Cross-platform installer and version manager for DuckDB
DuckERD CLI 78 7 months ago – A CLI tool to create an ER Diagram from DuckDB database files
SQLNotebook SQL notebooks by TimerStored powered by DuckDB

Awesome DuckDB / Logos and Icons
Logos DuckDB logos
Iconify Monochrome DuckDB icon in Iconify

Awesome DuckDB / Client APIs
C
C++
CLI
Go 761 4 months ago
Julia
Node.js
Python
R
Ruby 168 4 months ago
Rust 527 6 months ago
Swift
TypeScript
Wasm
ADBC
ODBC
.NET 458 4 months ago
Common Lisp 36 5 months ago
PowerShell 44 7 months ago
Dart 40 5 months ago

Awesome DuckDB / Tools Powered by DuckDB
Rill Developer 1,775 4 months ago Tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL
Ibis Project A DataFrame API for interacting with DuckDB (and other compute engines)
Boiling Data Serverless data analytics overlay on top of S3 Data Lakes
Hex Dataframe SQL Hex's Dataframe SQL cells are powered by DuckDB
Mode Mode uses DuckDB for their in-memory data engine
VulcanSQL DuckDB can be used as a caching layer or a data connector in VulcanSQL, a Data API framework for data folks to create REST APIs by writing SQL templates
Tad A fast, free, cross-platform tabular data viewer application powered by DuckDB
Honeycomb Maps A browser-based geospatial analysis tool leveraging DuckDB Wasm
Bauplan A serverless data transformation platform for data lakes
Malloy Malloy is an experimental language for describing data relationships and transformations. Malloy connects to BigQuery, Snowflake, Trino, and Postgres, and natively supports DuckDB
Evidence Generate reports using SQL and markdown. The DuckDB connector allows querying across DuckDB, csv, parquet and json
Latitude Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light
Census Census's dataset diffing for incremental syncs is powered by DuckDB
Huey 284 4 months ago Blazing-fast & intuitive pivot tables on .parquet, .csv, .json files and .duckdb tables in the browser based on DuckDB WASM. Open source (MIT). Zero install!
Parquet Explorer Visual Studio Code extension for exploring Parquet files with SQL, powered by DuckDB
DQOps Data quality platform for data engineers, data quality teams and data operations
DatalakeStudio 78 5 months ago Load, explore, transform your datasets and expose them via API. Integration with external APIs, S3, PostgreSQL and ChatGPT
Spice.ai 1,946 4 months ago A unified SQL query interface and portable runtime to locally materialize (using an embedded DuckDB), accelerate, and query datasets from any database, data warehouse, or data lake
Definite Definite pulls all your data into a single place for analytics and dashboards. No engineering or SQL required. Get a managed data warehouse (DuckDB), ELT, data modeling / transformations and BI in a single platform
Amphi ETL 933 4 months ago Low-code data pipelines for structured and unstructured data. SQL transformations are powered by DuckDB
Quackpipe 190 5 months ago Serverless OLAP API/UI built on top of DuckDB with basic ClickHouse API compatibility and Motherduck support
UniverSQL 119 4 months ago An implementation of Snowflake API, enables running queries on Snowflake tables locally with DuckDB without a running warehouse
Whereabouts 21 4 months ago Fast, accurate, open-source geocoding in Python, using DuckDB
Phoenix Analytics 278 5 months ago Plug and play analytics for Phoenix applications, powered by DuckDB
sqlglot 6,866 4 months ago Python transpiler that translates between 23 different SQL dialects including DuckDB
yato 182 7 months ago The smallest DuckDB SQL orchestrator on Earth
SQLMesh 1,897 4 months ago A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally

Awesome DuckDB / Tools Powered by DuckDB / Web Clients
Online DuckDB Shell Online DuckDB shell powered by WebAssembly
SQL Workbench DuckDB-WASM based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs
Sekuel Playground Query your local parquet, csv, json. Your data will not be sent out of the device you are using
CSVFiddle Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone
Codapi Embed executable code snippets directly into your product documentation, online course or blog post
QuackDB Open-source online DuckDB SQL playground and editor
WhatTheDuck WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data
Sidequery Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data

Awesome DuckDB / Tools Powered by DuckDB / Backends
DuckDB API 48 5 months ago a TypeScript-based Docker image containing DuckDB, and a Hono framework REST API with JSON or streaming Arrow responses
Mosaic DuckDB Server A Python-based server that runs a local DuckDB instance and support queries over Web Sockets or HTTP, returning data in either Apache Arrow or JSON format
duckdb-server A Rust-based server that runs a local DuckDB instance and support queries over Web Sockets or HTTP/HTTPS, returning data in either Apache Arrow or JSON format

Awesome DuckDB / Libraries Powered by DuckDB
Mosaic An extensible framework for linking databases and interactive views
Fugue 2,021 7 months ago A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites
Splink A free Python library for fast, accurate data deduplication and record linkage
Simple-data-analysis 251 4 months ago Easy-to-use and high-performance JavaScript library for data analysis
pg_analytics 407 4 months ago PostgreSQL extension embedding DuckDB-in-Postgres for fast on-disk and remote object storage analytics from Postgres. Built as a Foreign Data Wrapper with full query pushdown to DuckDB
duckdb_fdw 351 6 months ago DuckDB Foreign Data Wrapper for PostgreSQL
@jetblack/duckdb-react 9 about 1 year ago A context manager for React and duckdb-wasm
QuackOSM 219 4 months ago A Python library for downloading and transforming raw OpenStreetMap data into GeoParquet files
PyGWalker A Python library that turns your dataframe into an interactive UI for data visualization
https://github.com/DataZooDE/flapi] [ API Framework heavily relies on the power of DuckDB and DuckDB extensions. Ready to build performant and cost-efficient APIs on top of BigQuery or Snowflake for AI Agents and Data Apps

Awesome DuckDB / SQL Clients and IDE that Support DuckDB
Harlequin The DuckDB IDE for your terminal. ( )
qStudio A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results
DuckDB SQL Tools Free DuckDB SQL Tools for VS Code IDE. with advanced features
VSCode SQLTools (Free) Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support
DBeaver DBeaver is a universal database access and development tool that can be used to connect almost any type of database
DataGrip Paid SQL IDE by Jetbrains that supports many different database technologies, including DuckDB
Duckling 252 6 months ago A fast viewer for CSV/Parquet files and DuckDB/SQLite, based on Tauri
rsql 135 4 months ago CLI for DuckDB, LibSQL, MariaDB, MySQL, PostgreSQL, SQLite3 and SQL Server
jsqltranspiler 33 4 months ago Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL
jOOQ Type safe querying of DuckDB (and many other RDBMS) from Java. A from and to DuckDB is also available
SQL DATA LENS A lightweight, commercial SQL IDE that supports different DBMS, including DuckDB. The focus on performance and special DBMS features
Dataflare Simple easy-to-use database manager, supports DuckDB, PostgreSQL, MySQL, SQL Server, SQLite etc
manifold-sql (DuckDB for Java) 2,432 5 months ago Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support
Termux DuckDB package DuckDB CLI client for the Termux Android terminal emulator

Awesome DuckDB / Projects Powered by DuckDB
NBA Monte Carlo 461 4 months ago Monte Carlo simulation of the NBA season, leveraging Meltano, dbt, DuckDB and Evidence
Datadex 264 5 months ago Open source and local friendly data platform to collaborate on Open Data using DuckDB, Dagster, dbt, and Quarto
endoflife.date database Daily dumps of endoflife.date data
transfermarkt-datasets 249 4 months ago Curated football datasets from
duckDB-embedding-search 115 6 months ago A search engine for DuckDB that uses embedding vectors to find similar documents
DuckDB PyPI stats live dashboard ( ) - Live dashboard of PyPI downloads using DuckDB, dbt, Evidence and MotherDuck with code source to build your own
Specter 6 6 months ago Specter is a CLI tool to search and monitor Databricks audit logs

Awesome DuckDB / Integrations
dbt-duckdb 944 4 months ago DuckDB dbt adapter
data load tool - DuckDB destination Extract and load data from APIs to DuckDB using dlt
target-duckdb Load data to DuckDB based on Singer spec
Airbyte DuckDB destination Load data to DuckDB with Airbyte
Kestra DuckDB plugin Run queries with DuckDB to schedule data transformations and process automations, and run event-driven anomaly detection pipelines
SQLFlow 73 about 1 year ago Enables SQL-based stream-processing, powered by DuckDB
nf-sqldb 26 5 months ago This plugin provides support for interacting with SQL databases in Nextflow scripts
MindsDB 26,915 4 months ago The platform for customizing AI from enterprise data. , making data from DuckDB accessible to a diverse range of AI/ML models
Sqlite2Duckdb 12 11 months ago A CLI tool to convert SQLite database to DuckDB
nodbi NoSQL Database Connector for R, providing a common API across Elasticsearch, CouchDB, MongoDB, SQLite, PostgreSQL, and DuckDB
duckplyr Drop-in replacement for dplyr in R that uses DuckDB for performance
kwack 85 4 months ago In-Memory Analytics for Kafka using DuckDB
PSDuckDB 44 7 months ago A PowerShell module for DuckDB integration
duckdb-tableau-connector 7 10 months ago DuckDB Tableau connector
duckdb-power-query-connector 49 6 months ago DuckDB Power Query Custom Connector
metabase_duckdb_driver 63 5 months ago Metabase DuckDB Driver shipped as 3rd party plugin
xlDuckDb 50 4 months ago Excel addin to run DuckDB queries in Excel

Awesome DuckDB / Client-Server Setups
Crunchy Bridge for Analytics Fully managed DBaaS based in Postgres integrated with DuckDB
MotherDuck A serverless cloud data warehouse powered by DuckDB
MyDuck Server 329 4 months ago A server wrapping DuckDB with MySQL and PostgreSQL wire protocol support
ParadeDB 6,317 4 months ago Postgres for Search and Analytics, powered by DuckDB-embedded-in-Postgres
pg_duckdb 1,711 4 months ago DuckDB-powered Postgres for high performance apps & analytics
pg_mooncake 256 4 months ago A PostgreSQL extension that adds native column store tables with DuckDB
SQLFlite - Arrow Flight SQL Server 214 7 months ago An example implementation of the that runs in a client-server setup with DuckDB or SQLite as backends

Awesome DuckDB / Extensions / Official Extensions
arrow 36 9 months ago A zero-copy data integration between Apache Arrow and DuckDB
aws 42 6 months ago For handling AWS credentials
azure 52 6 months ago For using the Azure Blob storage
delta 145 4 months ago For Delta Lake support
fts To support full text search
iceberg 175 4 months ago For reading Iceberg tables
inet 1 6 months ago For storing and handling IPv4 and IPv6 Internet addresses
mysql To read from and write to MySQL databases
postgres To read from and write to PostgreSQL databases
spatial Enables geospatial processing
sqlite To read from and write to SQLite databases
vss Add support for vector similarity search

Awesome DuckDB / Extensions / Community Extensions
chsql 42 5 months ago ClickHouse SQL Dialect macros for DuckDB
crypto 12 7 months ago Cryptographic hash functions and HMAC
duckpgq 96 4 months ago Graph workloads that supports the SQL/PGQ standard
evalexpr_rhai 14 4 months ago Evaluates the scripting language as part of SQL
fuzzycomplete 8 7 months ago Performs fuzzy string matching for autocompletion
h3 174 4 months ago Adds support for the H3 discrete global grid system
lindel 37 5 months ago Linearization/Delinearization, Z-Order, Hilbert and Morton Curves
prql 263 8 months ago Run PRQL commands directly within DuckDB
scrooge 133 7 months ago A set of aggregation functions and data scanners on financial data
shellfs 59 4 months ago Allows shell commands to be used for input and output
ulid 16 9 months ago ULID data type for DuckDB. A ULID is similar to a UUID except that it also contains a timestamp component
gsheets 192 5 months ago Read and write Google Sheets using SQL
httpserver 134 4 months ago DuckDB HTTP API Server and Query Interface

Awesome DuckDB / Extensions / Other Extensions
DuckDB Extension Radar 84 4 months ago Repository that contains DuckDB extensions on GitHub. Refreshed daily
duckdb-bigquery 77 5 months ago Enables seamless integration and querying of BigQuery datasets within DuckDB
duckdb-engine 366 4 months ago SQLAlchemy driver for DuckDB
duckdb-extension-template-zig 26 about 1 year ago A Zig & Nix toolkit template for building extensions against multiple versions of DuckDB using Zig, C or C++
duckdb-jfr-extension 5 6 months ago DuckDB extension to read JFR (Java Flight Recorder) files directly
duckdb-protobuf 21 7 months ago Plugin for querying encoded protobuf messages (both sequences and individual messages per file)
duckdb-pytables 84 12 months ago DuckDB extension to allow running SQL on arbitrary data sources
ERPL DuckDB SAP connector using RFC, ODP, or BICS
Kùzu 1,454 4 months ago Scan DuckDB tables in Kùzu, an embeddable property graph database management system
Lance Integrate Lance (modern columnar data format for ML implemented in Rust) with DuckDB
ODBC Scanner DuckDB Extension 80 over 1 year ago DuckDB extension to read data directly from databases supporting the ODBC interface
QDuckDB Plugin for reading DuckDB spatial tables in QGIS software
uc_catalog 60 4 months ago Proof-of-concept extension combining the extension with Unity Catalog
duckdb-flockmtl 85 4 months ago Integrate language model (LLM) capabilities directly into your queries and workflows
erpl-web 8 6 months ago ERPL WEB is a DuckDB extension that connects API-based ecosystems via standard interfaces like OData, GraphQL, and REST

Awesome DuckDB / Media / Talks
DuckDB: Crunching data anywhere from laptops to servers @ GOTO Amsterdam 2024 Gábor Szárnyas
DuckDB – Overview and latest developments @ DuckCon #5 Hannes Mühleisen and Mark Raasveldt
DuckCon #5 playlist
DuckCon #4 playlist
DuckCon #3 playlist
In-Process Analytical Data Management with DuckDB @ PyData Amsterdam Hannes Mühleisen
DuckDB: The Power of a Data Warehouse in your Python Process @ PyData Yerevan Gábor Szárnyas
DuckDB: Bringing analytical SQL directly to your Python shell @ EuroPython Pedro Holanda
DuckDB keynote @ Data + AI Summit 2023 Hannes Mühleisen
DuckDB: Bringing Analytical SQL Directly To Your Python Shell @ FOSDEM Pedro Holanda
State of the Duck @ DuckCon #2 Hannes Mühleisen & Mark Raasveldt
DuckDB Extensions @ DuckCon Pedro Holanda & Sam Ansmink
Developing Systems in Academia: The Good, the Bad, and the not-so-Ugly Duckling @ CIDR Hannes Mühleisen
DuckDB An Embeddable Analytical Database @ FOSDEM Hannes Mühleisen
DuckDB tutorials playlist by Learn Data with Mark Mark Needham
DuckDB tutorials playlist by MotherDuck Mehdi Ouazza
Nextflow and database uses: powering data engineering, exploring DuckDB, and beyond Edmund Miller
Why should you care about DuckDB? @ Dublin DuckDB meetup Mihai Bojin
Exploring Monte Carlo Simulations With DuckDB @ Dublin DuckDB meetup James McNeill
DuckDB and recommenders : a lightning fast synergy @ Dublin DuckDB meetup Khalil Muhammad

Awesome DuckDB / Media / Podcasts
Developer Voices: Implementing Hardware-Friendly Databases Hannes Mühleisen
The Geek Narrator: DuckDB Internals Mark Raasveldt
Software Engineering Daily: DuckDB Hannes Mühleisen
Data Engineering Podcast: Move Your Database To The Data And Speed Up Your Analytics With DuckDB Hannes Mühleisen
The Analytics Engineering Podcast: The Personal Data Warehouse Jordan Tigani

Awesome DuckDB / Media / Blog Posts
Modern Data Stack in a Box Fast, free, and open-source Modern Data Stack deployed on a laptop using the combination of DuckDB, Meltano, dbt, and Apache Superset
How to use DuckDB, Motherduck and Kestra for ETL How DuckDB can transform data, mask sensitive PII information, detect anomalies in event-driven workflows, and streamline reporting use cases
DuckDB vs. MotherDuck — how do they compare What are key differences between them, and when to choose each of these options
Building DuckDB Extensions with Zig and Nix For Nix users and Zig developers familiar with DuckDB looking to extend its capabilities with custom extensions
Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts
DuckDB: The Rising Star in the Big Data Landscape
How to Make a DuckDB Extension for a Table Function? How to make a DuckDB extension to fetch data from external sources
Putting DuckDB in Postgres to Query Iceberg How ParadeDB embedded DuckDB in Postgres to achieve fast analytics and Apache Iceberg compatibility from Postgres

Awesome DuckDB / Media / Books
DuckDB in Action DuckDB in Action will show you how to quickly get your hands dirty with DuckDB
Getting Started with DuckDB A practical guide for accelerating your data science, data analytics, and data engineering workflows

Backlinks from these awesome lists:

More related projects: