beam
Data processor
A unified programming model for batch and streaming data processing pipelines
Apache Beam is a unified programming model for Batch and Streaming data processing.
8k stars
257 watching
4k forks
Language: Java
last commit: about 14 hours ago
Linked from 4 awesome lists
batchbeambig-datagolangjavapythonsqlstreaming
ethicalml/awesome-production-machine-learning
manuzhang/awesome-streaming
rayokota/awesome-hbase
kelvins/awesome-dataopsRelated projects:
| Repository | Description | Stars |
|---|---|---|
| apache/flink | An open-source stream processing framework with powerful capabilities for handling high-throughput and low-latency data streams in various programming languages | 24,156 |
| apache/jmeter | A tool used to simulate heavy loads on servers and measure their performance under different conditions. | 8,421 |
| apache/datafusion | A query engine that supports various data formats and allows customization of its functionality. | 6,340 |
 redpanda-data/connect | Stream processor for connecting various data sources and sinks using Apache V2 or Enterprise builds. | 8,140 |
| apache/samza | A distributed stream processing framework for handling high-volume data streams with fault tolerance and durability guarantees | 819 |
| apache/rocketmq-flink | Provides integration for RocketMQ with Apache Flink, enabling data streaming and messaging using the RocketMQ source and sink. | 144 |
 gazette/core | Enables teams to build platforms mixing SQL, batch, and real-time streaming processing paradigms | 719 |
| apache/rocketmq-connect | A tool for streaming data between Apache RocketMQ and other systems | 122 |
| apache/tez | A system that enables flexible data processing pipelines using a low-level engine for higher-level frameworks | 480 |
 lovoo/goka | A distributed stream processing library for Apache Kafka written in Go, enabling scalable and fault-tolerant microservices. | 2,362 |
| apache/streampipes | A toolbox for industrial data analytics and stream processing | 607 |
| apache/dubbo | A framework for building enterprise-ready microservices with support for RPC, service discovery, traffic management, and observability. | 40,527 |
 arroyosystems/arroyo | A distributed stream processing engine designed to efficiently perform stateful computations on high-volume real-time data streams. | 3,806 |
| apache/rocketmq | A distributed messaging and streaming platform with low latency, high performance, and reliability. | 21,278 |
| apache/rocketmq-streams | Provides a lightweight stream processing framework | 172 |