awesome-duckdb
🦆 A curated list of awesome DuckDB resources
1k stars
38 watching
106 forks
last commit: 17 days ago
Linked from 1 awesome list
awesomeawesome-list
Awesome DuckDB / Resources | |||
| Official Documentation | Official DuckDB documentation | ||
Awesome DuckDB / Resources / Official Documentation | |||
| Official Blog | Official DuckDB blog | ||
| DuckDB Clients | Client APIs for DuckDB | ||
| DuckDB Documentation PDF | The DuckDB documentation as a single PDF file | ||
Awesome DuckDB / Resources | |||
| docker-duckdb | 17 | 5 months ago | Docker image for DuckDB CLI |
| DuckDB setup | GitHub Action to install DuckDB in CI | ||
| Serverless DuckDB over S3 | 159 | 9 months ago | Running DuckDB over a data lake on S3 using lambda |
| DuckDB snippets | Collection of snippets curated by MotherDuck | ||
| DuckDB tldr page | DuckDB's entry in , available in CLI via the command | ||
| DuckDB AWS Lambda layer | 86 | 18 days ago | Run DuckDB in AWS Lambda functions |
| Compatible DuckDB Extensions for AWS Lambda | Extensions specifically compiled for the AWS Lambda runtime (GLIBC 2.26) | ||
| Serverless DuckDB as API | 103 | 4 months ago | Use DuckDB as API with Amazon API Gateway and AWS Lambda |
| Serverless Parquet Repartitioner | 26 | 12 months ago | Use DuckDB to repartition data in S3-based Data Lakes |
| Observable notebooks | Notebooks using DuckDB on the Observable data visualization platform | ||
| duckdb-nf | 2 | 8 months ago | Example uses of DuckDB with Nextflow |
| DuckDB version manager (duckman) | 13 | 18 days ago | – Cross-platform installer and version manager for DuckDB |
| DuckERD CLI | 69 | 22 days ago | – A CLI tool to create an ER Diagram from DuckDB database files |
Awesome DuckDB / Client APIs | |||
| C | |||
| C++ | |||
| CLI | |||
| Go | 660 | 11 days ago | |
| Julia | |||
| Node.js | |||
| Python | |||
| R | |||
| Ruby | 138 | 13 days ago | |
| Rust | 472 | about 1 month ago | |
| Swift | |||
| TypeScript | |||
| Wasm | |||
| ADBC | |||
| ODBC | |||
| .NET | 407 | 4 days ago | |
| Common Lisp | 35 | 11 days ago | |
| PowerShell | 39 | 26 days ago | |
Awesome DuckDB / Tools Powered by DuckDB | |||
| Rill Developer | 1,642 | 11 days ago | Tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL |
| Ibis Project | A DataFrame API for interacting with DuckDB (and other compute engines) | ||
| MotherDuck | Serverless data warehouse powered by DuckDB | ||
| Boiling Data | Serverless data analytics overlay on top of S3 Data Lakes | ||
| Hex Dataframe SQL | Hex's Dataframe SQL cells are powered by DuckDB | ||
| Mode | Mode uses DuckDB for their in-memory data engine | ||
| VulcanSQL | DuckDB can be used as a caching layer or a data connector in VulcanSQL, a Data API framework for data folks to create REST APIs by writing SQL templates | ||
| Tad | A fast, free, cross-platform tabular data viewer application powered by DuckDB | ||
| Honeycomb Maps | A browser-based geospatial analysis tool leveraging DuckDB Wasm | ||
| Bauplan | A serverless data transformation platform for data lakes | ||
| Malloy | Malloy is an experimental language for describing data relationships and transformations. Malloy connects to BigQuery, Snowflake, Trino, and Postgres, and natively supports DuckDB | ||
| Evidence | Generate reports using SQL and markdown. The DuckDB connector allows querying across DuckDB, csv, parquet and json | ||
| Latitude | Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light | ||
| Census | Census's dataset diffing for incremental syncs is powered by DuckDB | ||
| Huey | 212 | 12 days ago | Blazing-fast & intuitive pivot tables on .parquet, .csv, .json files and .duckdb tables in the browser based on DuckDB WASM. Open source (MIT). Zero install! |
| Parquet Explorer | Visual Studio Code extension for exploring Parquet files with SQL, powered by DuckDB | ||
| DQOps | Data quality platform for data engineers, data quality teams and data operations | ||
| DatalakeStudio | 68 | 27 days ago | Load, explore, transform your datasets and expose them via API. Integration with external APIs, S3, PostgreSQL and ChatGPT |
| Spice.ai | 1,846 | 11 days ago | A unified SQL query interface and portable runtime to locally materialize (using an embedded DuckDB), accelerate, and query datasets from any database, data warehouse, or data lake |
| Definite | pulls all your data into a single place for analytics and dashboards. No engineering or SQL required. Get a managed data warehouse (DuckDB), ELT, data modeling / transformations and BI in a single platform | ||
| Amphi ETL | 791 | 8 days ago | Low-code data pipelines for structured and unstructured data. SQL transformations are powered by DuckDB |
| Quackpipe | 152 | 14 days ago | Serverless OLAP API/UI built on top of DuckDB with basic ClickHouse API compatibility and Motherduck support |
| ParadeDB | 5,922 | 3 days ago | Postgres for Search and Analytics, powered by DuckDB-embedded-in-Postgres |
| Crunchy Bridge for Analytics | Fully managed DBaaS based in Postgres integrated with DuckDB | ||
| UniverSQL | 65 | about 1 month ago | An implementation of Snowflake API, enables running queries on Snowflake tables locally with DuckDB without a running warehouse |
| Whereabouts | 17 | 9 days ago | Fast, accurate, open-source geocoding in Python, using DuckDB |
| Phoenix Analytics | 217 | 11 days ago | Plug and play analytics for Phoenix applications, powered by DuckDB |
Awesome DuckDB / Tools Powered by DuckDB / Web Clients | |||
| Online DuckDB Shell | Online DuckDB shell powered by WebAssembly | ||
| SQL Workbench | DuckDB-WASM based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs | ||
| Sekuel Playground | Query your local parquet, csv, json. Your data will not be sent out of the device you are using | ||
| CSVFiddle | Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone | ||
| Codapi | Embed executable code snippets directly into your product documentation, online course or blog post | ||
| QuackDB | Open-source online DuckDB SQL playground and editor | ||
| WhatTheDuck | WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data | ||
Awesome DuckDB / Libraries Powered by DuckDB | |||
| Mosaic | An extensible framework for linking databases and interactive views | ||
| Fugue | 1,974 | 14 days ago | A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites |
| Splink | A free Python library for fast, accurate data deduplication and record linkage | ||
| Simple-data-analysis | 237 | 12 days ago | Easy-to-use and high-performance JavaScript library for data analysis |
| pg_analytics | 140 | 12 days ago | PostgreSQL extension embedding DuckDB-in-Postgres for fast on-disk and remote object storage analytics from Postgres. Built as a Foreign Data Wrapper with full query pushdown to DuckDB |
| duckdb_fdw | 330 | 5 days ago | DuckDB Foreign Data Wrapper for PostgreSQL |
| @jetblack/duckdb-react | 8 | 7 months ago | A context manager for React and duckdb-wasm |
| QuackOSM | 179 | 11 days ago | A Python library for downloading and transforming raw OpenStreetMap data into GeoParquet files |
| PyGWalker | A Pyhon library that turns your dataframe into an interactive UI for data visualization | ||
Awesome DuckDB / SQL Clients and IDE that Support DuckDB | |||
| Harlequin | The DuckDB IDE for your terminal. ( ) | ||
| qStudio | A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results | ||
| DuckDB SQL Tools | Free DuckDB SQL Tools for VS Code IDE. with advanced features | ||
| VSCode SQLTools (Free) | Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support | ||
| DBeaver | DBeaver is a universal database access and development tool that can be used to connect almost any type of database | ||
| DataGrip | Paid SQL IDE by Jetbrains that supports many different database technologies, including DuckDB | ||
| Duckling | 203 | 22 days ago | A fast viewer for CSV/Parquet files and DuckDB/SQLite, based on Tauri |
| rsql | 93 | about 1 month ago | CLI for DuckDB, LibSQL, MariaDB, MySQL, PostgreSQL, SQLite3 and SQL Server |
| jsqltranspiler | 28 | 25 days ago | Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL |
| jOOQ | Type safe querying of DuckDB (and many other RDBMS) from Java. A from and to DuckDB is also available | ||
| SQL DATA LENS | A lightweight, commercial SQL IDE that supports different DBMS, including DuckDB. The focus on performance and special DBMS features | ||
| Dataflare | Simple easy-to-use database manager, supports DuckDB, PostgreSQL, MySQL, SQL Server, SQLite etc | ||
| manifold-sql (DuckDB for Java) | 2,380 | 5 days ago | Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support |
Awesome DuckDB / Projects Powered by DuckDB | |||
| NBA Monte Carlo | 422 | 12 days ago | Monte Carlo simulation of the NBA season, leveraging Meltano, dbt, DuckDB and Evidence |
| Datadex | 240 | 26 days ago | Open source and local friendly data platform to collaborate on Open Data using DuckDB, Dagster, dbt, and Quarto |
| endoflife.date database | Daily dumps of endoflife.date data | ||
| transfermarkt-datasets | 223 | 12 days ago | Curated football datasets from |
| duckDB-embedding-search | 70 | 7 months ago | A search engine for DuckDB that uses embedding vectors to find similar documents |
| DuckDB PyPI stats live dashboard | ( ) - Live dashboard of PyPI downloads using DuckDB, dbt, Evidence and MotherDuck with code source to build your own | ||
Awesome DuckDB / Integrations | |||
| dbt-duckdb | 870 | 12 days ago | DuckDB dbt adapter |
| data load tool - DuckDB destination | Extract and load data from APIs to DuckDB using dlt | ||
| target-duckdb | Load data to DuckDB based on Singer spec | ||
| Airbyte DuckDB destination | Load data to DuckDB with Airbyte | ||
| Kestra DuckDB plugin | Run queries with DuckDB to schedule data transformations and process automations, and run event-driven anomaly detection pipelines | ||
| Arrow Flight SQL Server | 191 | 26 days ago | An example implementation of the what runs in a client-server setup with DuckDB or SQLite as backends |
| SQLFlow | 62 | 6 months ago | Enables SQL-based stream-processing, powered by DuckDB |
| nf-sqldb | 26 | 10 months ago | This plugin provides support for interacting with SQL databases in Nextflow scripts |
| MindsDB | 26,362 | 4 days ago | The platform for customizing AI from enterprise data. , making data from DuckDB accessible to a diverse range of AI/ML models |
| Sqlite2Duckdb | 7 | 4 months ago | A CLI tool to convert SQLite database to DuckDB |
| nodbi | NoSQL Database Connector for R, providing a common API across Elasticsearch, CouchDB, MongoDB, SQLite, PostgreSQL, and DuckDB | ||
| duckplyr | Drop-in replacement for dplyr in R that uses DuckDB for performance | ||
| kwack | 64 | 13 days ago | In-Memory Analytics for Kafka using DuckDB |
| PSDuckDB | 39 | 26 days ago | A PowerShell module for DuckDB integration |
| duckdb-tableau-connector | 4 | 4 months ago | DuckDB Tableau connector |
| duckdb-power-query-connector | 38 | 4 months ago | DuckDB Power Query Custom Connector |
| metabase_duckdb_driver | 41 | 15 days ago | Metabase DuckDB Driver shipped as 3rd party plugin |
Awesome DuckDB / Extensions / Official Extensions | |||
| arrow | 33 | 3 months ago | A zero-copy data integration between Apache Arrow and DuckDB |
| aws | 40 | about 2 months ago | For handling AWS credentials |
| azure | 47 | about 1 month ago | For using the Azure Blob storage |
| delta | 120 | 11 days ago | For Delta Lake support |
| fts | To support full text search | ||
| iceberg | 132 | about 2 months ago | For reading Iceberg tables |
| inet | 1 | 3 months ago | For storing and handling IPv4 and IPv6 Internet addresses |
| mysql | To read from and write to MySQL databases | ||
| postgres | To read from and write to PostgreSQL databases | ||
| spatial | Enables geospatial processing | ||
| sqlite | To read from and write to SQLite databases | ||
| vss | Add support for vector similarity search | ||
Awesome DuckDB / Extensions / Community Extensions | |||
| chsql | 26 | 12 days ago | ClickHouse SQL Dialect macros for DuckDB |
| crypto | 9 | 11 days ago | Cryptographic hash functions and HMAC |
| duckpgq | 60 | 18 days ago | Graph workloads that supports the SQL/PGQ standard |
| evalexpr_rhai | 12 | 11 days ago | Evaluates the scripting language as part of SQL |
| fuzzycomplete | 5 | 11 days ago | Performs fuzzy string matching for autocompletion |
| h3 | 159 | 11 days ago | Adds support for the H3 discrete global grid system |
| lindel | 23 | 11 days ago | Linearization/Delinearization, Z-Order, Hilbert and Morton Curves |
| prql | 252 | about 1 month ago | Run PRQL commands directly within DuckDB |
| scrooge | 128 | 20 days ago | A set of aggregation functions and data scanners on financial data |
| shellfs | 51 | 11 days ago | Allows shell commands to be used for input and output |
| ulid | 11 | 3 months ago | ULID data type for DuckDB. A ULID is similar to a UUID except that it also contains a timestamp component |
Awesome DuckDB / Extensions / Other Extensions | |||
| DuckDB Extension Radar | 75 | 12 days ago | Repository that contains DuckDB extensions on GitHub. Refreshed daily |
| duckdb-bigquery | 28 | 12 days ago | Enables seamless integration and querying of BigQuery datasets within DuckDB |
| duckdb-engine | 337 | 12 days ago | SQLAlchemy driver for DuckDB |
| duckdb-extension-template-zig | 20 | 7 months ago | A Zig & Nix toolkit template for building extensions against multiple versions of DuckDB using Zig, C or C++ |
| duckdb-jfr-extension | 5 | 10 months ago | DuckDB extension to read JFR (Java Flight Recorder) files directly |
| duckdb-protobuf | 15 | 24 days ago | Plugin for querying encoded protobuf messages (both sequences and individual messages per file) |
| duckdb-pytables | 80 | 5 months ago | DuckDB extension to allow running SQL on arbitrary data sources |
| ERPL | DuckDB SAP connector using RFC, ODP, or BICS | ||
| Kùzu | 1,288 | 11 days ago | Scan DuckDB tables in Kùzu, an embeddable property graph database management system |
| Lance | 3,807 | 11 days ago | Integrate Lance (modern columnar data format for ML implemented in Rust) with DuckDB |
| ODBC Scanner DuckDB Extension | 75 | about 1 year ago | DuckDB extension to read data directly from databases supporting the ODBC interface |
| QDuckDB | Plugin for reading DuckDB spatial tables in QGIS software | ||
| uc_catalog | 42 | 11 days ago | Proof-of-concept extension combining the extension with Unity Catalog |
Awesome DuckDB / Media / Talks | |||
| DuckCon #4 playlist | |||
| State of the Duck @ DuckCon #4 | Hannes Mühleisen and Mark Raasveldt | ||
| In-Process Analytical Data Management with DuckDB @ PyData Amsterdam | Hannes Mühleisen | ||
| DuckDB: The Power of a Data Warehouse in your Python Process @ PyData Yerevan | Gábor Szárnyas | ||
| DuckDB: Bringing analytical SQL directly to your Python shell @ EuroPython | Pedro Holanda | ||
| DuckDB keynote @ Data + AI Summit 2023 | Hannes Mühleisen | ||
| State of the Duck @ DuckCon #3 | Hannes Mühleisen and Mark Raasveldt | ||
| DuckCon #3 playlist | |||
| DuckDB: Bringing Analytical SQL Directly To Your Python Shell @ FOSDEM | Pedro Holanda | ||
| State of the Duck @ DuckCon #2 | Hannes Mühleisen & Mark Raasveldt | ||
| DuckDB Extensions @ DuckCon | Pedro Holanda & Sam Ansmink | ||
| Developing Systems in Academia: The Good, the Bad, and the not-so-Ugly Duckling @ CIDR | Hannes Mühleisen | ||
| DuckDB An Embeddable Analytical Database @ FOSDEM | Hannes Mühleisen | ||
| DuckDB tutorials playlist by Learn Data with Mark | Mark Needham | ||
| DuckDB tutorials playlist by MotherDuck | Mehdi Ouazza (MotherDuck) | ||
| Nextflow and database uses: powering data engineering, exploring DuckDB, and beyond | Edmund Miller | ||
| Why should you care about DuckDB? @ Dublin DuckDB meetup | Mihai Bojin | ||
| Exploring Monte Carlo Simulations With DuckDB @ Dublin DuckDB meetup | James McNeill | ||
| DuckDB and recommenders : a lightning fast synergy @ Dublin DuckDB meetup | Khalil Muhammad | ||
Awesome DuckDB / Media / Podcasts | |||
| Developer Voices: Implementing Hardware-Friendly Databases | Hannes Mühleisen | ||
| The Geek Narrator: DuckDB Internals | Mark Raasveldt | ||
| Software Engineering Daily: DuckDB | Hannes Mühleisen | ||
| Data Engineering Podcast: Move Your Database To The Data And Speed Up Your Analytics With DuckDB | Hannes Mühleisen | ||
| The Analytics Engineering Podcast: The Personal Data Warehouse | Jordan Tigani | ||
Awesome DuckDB / Media / Blog Posts | |||
| Modern Data Stack in a Box | Fast, free, and open-source Modern Data Stack deployed on a laptop using the combination of DuckDB, Meltano, dbt, and Apache Superset | ||
| How to use DuckDB, Motherduck and Kestra for ETL | How DuckDB can transform data, mask sensitive PII information, detect anomalies in event-driven workflows, and streamline reporting use cases | ||
| DuckDB vs. MotherDuck — how do they compare | What are key differences between them, and when to choose each of these options | ||
| Building DuckDB Extensions with Zig and Nix | For Nix users and Zig developers familiar with DuckDB looking to extend its capabilities with custom extensions | ||
| Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit | Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts | ||
| DuckDB: The Rising Star in the Big Data Landscape | |||
| How to Make a DuckDB Extension for a Table Function? | How to make a DuckDB extension to fetch data from external sources | ||
| Putting DuckDB in Postgres to Query Iceberg | How ParadeDB embedded DuckDB in Postgres to achieve fast analytics and Apache Iceberg compatibility from Postgres | ||
Awesome DuckDB / Media / Books | |||
| DuckDB in Action | DuckDB in Action will show you how to quickly get your hands dirty with DuckDB | ||
| Getting Started with DuckDB | A practical guide for accelerating your data science, data analytics, and data engineering workflows | ||
jnv/lists