awesome-database-learning

Database learning resources

A curated list of learning materials for understanding database internals

A list of learning materials to understand databases internals

GitHub

10k stars
207 watching
1k forks
last commit: about 2 years ago
Linked from 1 awesome list

awesomeawesome-listblogsbookscoursedatabasepapers

Database Systems (15-445/645)CMU , thanks to
Advanced Database Systems (15-721)CMU , thanks to
Introduction to Database SystemsUC Berkeley
Database System ImplementationStanford
Introduction to Database SystemsCornell by Prof. Trummer
Let's Build a Simple Database, thanks to
Database Systems: The Complete BookStanford
Designing Data-Intensive Applications,
Database Internals
Foundations of Databases
Readings in Database Systems, 5th Edition
Database Design and Implementation: Second Edition (Data-Centric Systems and Applications)
Principles of Distributed Database Systems, 4th ed
Inside SQLite
Architecture of a Database System
Relational Database Index Design and the Optimizers
Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control
Data Structures and Algorithms for Big Databases
A Journey From A Quick HackTo A High-Reliability Database Engine
How does a relational database work
The Internals of PostgreSQL
Books propose
what is database and its types

Awesome Database Learning / SQL & Relation Algebra

Course Introduction and the Relational Model
Advanced SQL

Awesome Database Learning / Query Optimizer

Database Systems (15-445/645)CMU , thanks to

Awesome Database Learning / Query Optimizer / Database Systems (15-445/645)

Query Planning & Optimization I
Query Planning & Optimization II

Awesome Database Learning / Query Optimizer

数据库内核杂谈, thanks to

Awesome Database Learning / Query Optimizer / 数据库内核杂谈

数据库内核杂谈(七):数据库优化器(上)
数据库内核杂谈(八):数据库优化器(下)

Awesome Database Learning / Query Optimizer

SQL优化器原理 - 查询优化器综述, thanks to

Awesome Database Learning / Query Optimizer / Planner Models

数据库内核杂谈, thanks to

Awesome Database Learning / Query Optimizer / Planner Models / 数据库内核杂谈

数据库内核杂谈(九):开源优化器 ORCA

Awesome Database Learning / Query Optimizer / Planner Models

SQL 查询优化原理与 Volcano Optimizer 介绍, thanks to
Cascades Optimizer, thanks to
Access Path Selection in a Relational Database Management System1979, , SIGMOD
Query Processing in Main Memory Database Management Systems1979, , VLDB
Query Optimization by Simulated Annealing1987, , SIGMOD
Grammar-like Functional Rules for Representing Query Optimization Alternatives1988, , SIGMOD
The Volcano Optimizer Generator- Extensibility and Efficient Search1993, , ICDE
The Cascades Framework for Query Optimization1995, , IEEE Data engineering Bulltin
An Overview of Query Optimization in Relational Systems1998, , PODS
LEO – DB2’s LEarning Optimizer2001, , VLDB
Robust Query Processing through Progressive Optimization2004, , SIGMOD
Orca: A Modular Query Optimizer Architecture for Big Data2014, , SIGMOD
Parallelizing Query Optimization on Shared-Nothing Architectures2016, , VLDB
The MemSQL Query Optimizer: A modern optimizer for real-time analytics in a distributed database2016, , VLDB

Awesome Database Learning / Query Optimizer / Subquery Optimization

SQL 子查询的优化, thanks to
Calcite 子查询处理 - I (RemoveSubQuery), thanks to
Calcite 子查询处理 - II (Decorrelate), thanks to
Orthogonal Optimization of Subqueries and Aggregation2001, , SIGMOD
Enhanced subquery optimizations in Oracle2009, , VLDB
Unnesting Arbitrary Queries2015, , BTW

Awesome Database Learning / Query Optimizer / Join Order Optimization

Analysis of Two Existing and One New Dynamic Programming Algorithm for the Generation of Optimal Bushy Join Trees without Cross Products2006, , VLDB
How Good Are Query Optimizers, Really?2015, , VLDB
Adaptive Optimization of Very Large Join Queries2018, , SIGMOD

Awesome Database Learning / Query Optimizer / Functional Dependency & Physical Properties

Exploiting Functional Dependence in Query Optimization2000,
Fundamental Techniques for Order Optimization1996, , SIGMOD
An Efficient Framework for Order Optimization2004, , ICDE
Incorporating Partitioning and Parallel Plans into the SCOPE Optimizer2010, , ICDE

Awesome Database Learning / Query Optimizer / Cost Model

Modelling Costs for a MM-DBMS1996, , in Real-Time Databases
Approximation Schemes for Many-Objective Query Optimization2014, , SIGMOD
Multi-Objective Parametric Query Optimization2015, , VLDB

Awesome Database Learning / Query Optimizer / Statistics

Accurate Estimation of the Number of Tuples Satisfying a Condition1984, , SIGMOD
Optimal Histograms for Limiting Worst-Case Error Propagation in the Size of Join Results1993, , ACM Trans. on Database Systems
Universality of Serial Histograms1993, , VLDB
Balancing Histogram Optimality and Practicality for Query Result Size Estimation1995, , SIGMOD
Improved Histograms for Selectivity Estimation of Range Predicates1996, , SIGMOD
SEEKing the truth about ad hoc join costs1997, , VLDB
Towards Estimation Error Guarantees for Distinct Values2000, , SIGMOD/PODS
Distinct Sampling for Highly-Accurate Answers to Distinct Values Queries and Event Reports2001, , VLDB
The History of Histograms2003, , VLDB
An Improved Data Stream Summary: The Count-Min Sketch and its Applications2005, , Journal of Algorithms
New Estimation Algorithms for Streaming Data: Count-min Can Do More2007,
Preventing Bad Plans by Bounding the Impact of Cardinality Estimation Errors2009, , VLDB
Histograms Reloaded: The Merits of Bucket Diversity2010, , SIGMOD
Exploiting Ordered Dictionaries to Efficiently Construct Histograms with Q-Error Guarantees in SAP HANA2014, , SIGMOD
Adaptive Statistics in Oracle 12c2017, , VLDB
Pessimistic Cardinality Estimation: Tighter Upper Bounds for Intermediate Join Cardinalities2019, , SIGMOD
Deep Unsupervised Cardinality Estimation2019, , VLDB
NeuroCard: One Cardinality Estimator for All Tables2020, , VLDB
Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches

Awesome Database Learning / Query Execution

Database Systems (15-445/645)CMU , thanks to

Awesome Database Learning / Query Execution / Database Systems (15-445/645)

Query Execution I
Query Execution II

Awesome Database Learning / Query Execution / Execution Framework

Volcano-An Extensible and Parallel Query Evaluation System1994, , IEEE Transactions on Knowledge and Data EngineeringFebruary
Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age2014, , SIGMOD

Awesome Database Learning / Query Execution / Vectorization vs Compilization

Overhead of a Generalized Query Execution Engine40about 5 years ago, from , thanks to the Pivotal Engineering team
MonetDB/X100: Hyper-Pipelining Query Execution2005, , CIDR
Efficiently Compiling Efficient Query Plans for Modern Hardware2011, , VLDB
Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last2017, , VLDB
Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask2018, , VLDB
Adaptive Execution of Compiled Queries2018, , ICDE

Awesome Database Learning / Query Execution / Join

Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited2013, , VLDB
Looking Ahead Makes Query Plans Robust2017, , VLDB

Awesome Database Learning / Query Execution / Hash Table

Database Systems (15-445/645)CMU , thanks to

Awesome Database Learning / Query Execution / Hash Table / Database Systems (15-445/645)

Hash Tables

Awesome Database Learning / Query Execution / Hash Table

Fibonacci Hashing: The Optimization that the World Forgot (or: a Better Alternative to Integer Modulo), thanks to
All hash table sizes you will ever need, thanks to

Awesome Database Learning / Query Execution / Bloom Filter

SuRF: Practical Range Query Filtering with Fast Succinct Tries2018, , SIGMOD

Awesome Database Learning / DDL

Online, Asynchronous Schema Change in F12013, , VLDB

Awesome Database Learning / Relational Model

What is a Relational Database?, thanks to
What is a Relational Database?,thank to

Awesome Database Learning / Relational Model / Codd's Rules

Codd’s Rules for Relational Database Systems, thanks to

Awesome Database Learning / Relational Model / Relational Data Model

Relational model, thanks to

Awesome Database Learning / Relational Model / Relational Algebra

Introduction of Relational Algebra in DBMS, thanks to

Awesome Database Learning / Relational Model / ER to Relational Model

ER Model to Relational Model, thanks to

Awesome Database Learning / Relational Model / SQL - Overview

An Overview of SQL Text Functions, thanks to

Awesome Database Learning / Transaction / Isolation Levels

一致性模型, thanks to
A Critique of ANSI SQL Isolation Levels1995, , SIGMOD
Generalized Isolation Level Definitions2000, , Proceedings of 16th International Conference on Data Engineering

Awesome Database Learning / Transaction / Concurrency Control

Concurrency Control Theory
Two-Phase Locking Concurrency Control
Timestamp Ordering Concurrency Control
Multi-Version Concurrency Control
Multi-Version Concurrency Control (Design Decisions)
Multi-Version Concurrency Control (Protocols)
Multi-Version Concurrency Control (Garbage Collection)
The Notions of Consistency and Predicate Locks in a Database System1976, , Communications of the ACM
Concurrency Control in Distributed Database Systems1981, , ACM Computing Surveys
On Optimistic Methods for Concurrency Control1981, , ACM Transactions on Database Systems
Multiversion Concurrency Control - Theory and Algorithms1983, , ACM Transactions on Database Systems
Serializable Snapshot Isolation in PostgreSQL2012, , VLDB
Calvin: Fast Distributed Transactions for Partitioned Database Systems2012, , SIGMOD
MaaT: effective and scalable coordination of distributed transactions in the cloud2014, , VLDB
Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores2014, , VLDB
An Evaluation of the Advantages and Disadvantages of Deterministic Database Systems2014, , VLDB
Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems2015, , SIGMOD
An Empirical Evaluation of In-Memory Multi-Version Concurrency Control2017, , VLDB
An Evaluation of Distributed Concurrency Control2017, , VLDB
Scalable Garbage Collection for In-Memory MVCC Systems2019, , VLDB

Awesome Database Learning / Network

Advanced Database Systems (15-721)CMU , thanks to

Awesome Database Learning / Network / Advanced Database Systems (15-721)

Networking Protocols

Awesome Database Learning / Network

The End of Slow Networks: It's Time for a Redesign2016, , VLDB
Accelerating Relational Databases by Leveraging Remote Memory and RDMA2016, , SIGMOD
Don't Hold My Data Hostage: A Case for Client Protocol Redesign2017, , VLDB

Awesome Database Learning / Storage / NoSQL Systems

Bigtable: A Distributed Storage System for Structured Data2006, , OSDI
Dynamo: Amazon’s Highly Available Key-value Store2007, , SOSP
PNUTS: Yahoo!’s Hosted Data Serving Platform2008, , VLDB
Cassandra - A Decentralized Structured Storage System2010, , SOSP
PNUTS to Sherpa: Lessons from Yahoo!’s Cloud Database2019, , VLDB

Awesome Database Learning / Storage / Buffer Management

Database Systems (15-445/645)CMU , thanks to

Awesome Database Learning / Storage / Buffer Management / Database Systems (15-445/645)

Buffer Pools

Awesome Database Learning / Storage / Buffer Management

The 5 Minute Rule for Trading Memory for Disc Accesses and the 5 Byte Rule for Trading Memory for CPU Time1987, , SIGMOD
The Five Minute Rule 20 Years Later and How Flash Memory Changes the Rules2008, , ACM Queue
Managing Non-Volatile Memory in Database Systems2018, , SIGMOD
LeanStore: In-Memory Data Management Beyond Main Memory2018, , ICDE
Umbra: A Disk-Based System with In-Memory Performance2020, , CIDR

Awesome Database Learning / Storage / Disk IO

On Disk IO, Part 1: Flavors of IO, thanks to
On Disk IO, Part 2: More Flavours of IO, thanks to
On Disk IO, Part 3: LSM Trees, thanks to
On Disk IO, Part 4: B-Trees and RUM Conjecture, thanks to
On Disk IO, Part 5: Access Patterns in LSM Trees, thanks to
Ensuring data reaches disk(LWN)
Read, write & space amplification - pick 2, thanks to
Design Tradeoffs of Data Access Methods2016, , SIGMOD
Designing Access Methods: The RUM Conjecture2016, , EDBT

Awesome Database Learning / Storage / B-Tree

B树、B+树索引算法原理(上)thanks to
B树、B+树索引算法原理(下)
Trees Indexes I
Trees Indexes II
OLTP Indexes (B+Tree Data Structures)
The Ubiquitous B-Tree1979,

Awesome Database Learning / Storage / LSM-Tree

The Log-Structured Merge-Tree (LSM-Tree)1996, ,
A Comparison of Fractal Trees to Log-Structured Merge (LSM) Trees2014,
WiscKey: Separating Keys from Values in SSD-conscious Storage2017, , TOS
LSM-based Storage Techniques: A Survey2019,

Awesome Database Learning / Storage / Learned Indexes Structures

The Case for Learned Index Structures2018,
Learning Multi-dimensional Indexes2019,
XIndex: A Scalable Learned Index for Multicore Data Storage2020,
RadixSpline: A Single-Pass Learned Index2020, , , aiDM@SIGMOD
The PGM-index: a fully-dynamic compressed learned index with provable worst-case bounds2020, , , VLDB
From WiscKey to Bourbon: A Learned Index for Log-Structured Merge Trees2020,

Awesome Database Learning / Serializing & RPC

Protocol Buffers Developer Guide
gRPC Documntation

Awesome Database Learning / Data Partitioning

TiDB Internal (I) - Data Storage
Partitioning Behavior of DynamoDB, thanks to
Dynamo: Amazon’s Highly Available Key-value Store2007, , SOSP

Awesome Database Learning / Replication & Consistency

Tick or Tock? Keeping Time and Order in Distributed Databases, thanks to
Consistency Tradeoffs in Modern Distributed Database System Design2012,
Strong and Efficient Consistency with Consistency-Aware Durability2020, , FAST 2020

Awesome Database Learning / Consensus

Distributed consensus revisedUniversity of Cambridge , a great paper about Consenssus especially Paxos and Paxos-Related algorithms, by Heidi Howard
Ark: A Real-World Consensus Implementation2014, , CoRR

Awesome Database Learning / Scheduling

Building a Large-scale Distributed Storage System Based on Raft, by Ed Huang
Automated Demand-driven Resource Scaling in Relational Database-as-a-Service2016, , SIGMOD
Autoscaling Tiered Cloud Storage in Anna2019, , VLDB
Adaptive HTAP through Elastic Resource Scheduling2020, , SIGMOD
MorphoSys: Automatic Physical Design Metamorphosis for Distributed Database Systems2020, , VLDB

Awesome Database Learning / Benchmark & Testing

Use go-ycsb to benchmark different databases (1), thanks to
Chaos Tools and Techniques for Testing the TiDB Distributed NewSQL Database, thanks to
Creating Custom Sysbench Scripts, thanks to
Benchmarking Cloud Serving Systems with YCSB2010, , SOCC

Awesome Database Learning / HTAP

TiDB: A Raft-based HTAP Database2020, , VLDB
F1 Lightning: HTAP as a Service2020, , VLDB

Awesome Database Learning / TLA+

The TLA+ Video Course

Backlinks from these awesome lists: