awesome-system-design

Overview of how Reddit system design scaled

Graduate level course on distributed systems from MIT (2020)

Undergraduate course on distributed systems from UCSC (2020)

Highly popular way to deploy, manage and automatically scale a cluster of containers on bare-metal or virtual servers

Dashboard that integrates most of hadoop related technologies for easy management and executions

Create workflows in xml to execute jobs (from other hadoop-ecosystem applications) in steps, allows for parallel execution as well

Query hadoop stored data in SQL

Scriping language that looks like SQL to query hadoop data

Solves a similar problem to Spark and MapReduce, it's more efficient than MapReduce because it calculates the most efficient way of doing it

MapReduce, as the name implies, maps data and reduce the results

Powerful data processing to not only process data like Tez (and MapReduce), it can process streams of data in real time, apply regression analysis algorithms in ML and much more

*Retired project, it's a YARN-native platform that unifies stream and batch processing

[3.6k ⭐] - Modeled after Google's Bigtable and written in Java. Developed as a part of Apache Hadoop project

'Yet Another Resource Negotiator', works like a kernel to manage computer resources across the clusters

Works like a Linux Kernel by managing CPU, memory, storage and other resources across the cluster