beam
Data processor
A unified programming model for batch and streaming data processing pipelines
Apache Beam is a unified programming model for Batch and Streaming data processing.
8k stars
255 watching
4k forks
Language: Java
last commit: about 2 months ago
Linked from 4 awesome lists
batchbeambig-datagolangjavapythonsqlstreaming
ethicalml/awesome-production-machine-learning
manuzhang/awesome-streaming
rayokota/awesome-hbase
kelvins/awesome-dataopsRelated projects:
| Repository | Description | Stars |
|---|---|---|
| apache/flink | An open-source stream processing framework with powerful capabilities for handling high-throughput and low-latency data streams in various programming languages | 24,261 |
| apache/jmeter | A tool used to simulate heavy loads on servers and measure their performance under different conditions. | 8,467 |
| apache/datafusion | A query engine that supports various data formats and allows customization of its functionality. | 6,462 |
 redpanda-data/connect | Stream processor for connecting various data sources and sinks using Apache V2 or Enterprise builds. | 8,165 |
| apache/samza | A distributed stream processing framework for handling high-volume data streams with fault tolerance and durability guarantees | 817 |
| apache/rocketmq-flink | Provides integration for RocketMQ with Apache Flink, enabling data streaming and messaging using the RocketMQ source and sink. | 145 |
 gazette/core | Enables teams to build platforms mixing SQL, batch, and real-time streaming processing paradigms | 723 |
| apache/rocketmq-connect | A tool for streaming data between Apache RocketMQ and other systems | 122 |
| apache/tez | A system that enables flexible data processing pipelines using a low-level engine for higher-level frameworks | 482 |
 lovoo/goka | A distributed stream processing library for Apache Kafka written in Go, enabling scalable and fault-tolerant microservices. | 2,367 |
| apache/streampipes | A toolbox for industrial data analytics and stream processing | 614 |
| apache/dubbo | A framework for building enterprise-ready microservices with support for RPC, service discovery, traffic management, and observability. | 40,569 |
 arroyosystems/arroyo | A distributed stream processing engine designed to efficiently perform stateful computations on high-volume real-time data streams. | 3,840 |
| apache/rocketmq | A distributed messaging and streaming platform with low latency, high performance, and reliability. | 21,354 |
| apache/rocketmq-streams | Provides a lightweight stream processing framework | 172 |