awesome-kafka
Kafka toolkit
A comprehensive resource and collection of tools and libraries for Apache Kafka development
Everything about Apache Kafka
204 stars
15 watching
30 forks
last commit: over 1 year ago
Linked from 2 awesome lists
awesomeawsome-listkafka
awesome-kafka / Development / Client libraries | |||
| Go | 11,574 | 5 days ago | |
| Go | 130 | almost 8 years ago | |
| Go | 216 | about 5 years ago | |
| Go Kafka streaming | 2,362 | 2 months ago | |
| Node | |||
| Node | 297 | about 1 month ago | |
| Node | 290 | almost 2 years ago | |
| Node Kafka streams | 831 | over 1 year ago | |
| Node librdkafka | 0 | over 6 years ago | |
| Php | 1,442 | over 2 years ago | |
| Php | 2,089 | 19 days ago | |
| Perl | 41 | over 4 years ago | |
| Python (asyncio) | 1,167 | 7 days ago | |
| Python (kafka-python) | 5,624 | about 1 month ago | |
| Python (pykafka) | 1,119 | almost 4 years ago | |
| Ruby | 1,275 | over 1 year ago | |
| Phobos - Ruby App framework | 220 | about 1 year ago | |
| Karafka - Ruby and Rails framework | 2,091 | 8 days ago | |
| Dart | 56 | about 2 years ago | |
| JRuby | 71 | over 8 years ago | |
| Elixir | 598 | about 1 month ago | |
| Erlang | 662 | 15 days ago | |
| Rust | 1,253 | about 1 month ago | |
| LinkedIn wrapper library | 133 | over 1 year ago | |
| kasper - a lightweight library for processing Kafka topics | 439 | about 7 years ago | |
| others | and | ||
awesome-kafka / Development / Kafka Streams Libraries | |||
| Scala | 190 | over 1 year ago | |
| faust - Python Stream Processing | 6,746 | 4 months ago | |
| Southpaw - Streaming left joins for data change capture | 52 | 6 months ago | |
awesome-kafka / Development / KSQL | |||
| KSQL JDBC Driver | 88 | over 2 years ago | |
| KSQL Addon - User Defined Function (UDF) for Machine Learning | 11 | over 6 years ago | |
awesome-kafka / Development / Connectors | |||
| JDBC | 21 | 9 days ago | |
| Elasticsearch | 15 | 7 days ago | |
| HDFS | 12 | 7 days ago | |
| Beast - Kafka to BigQuery Sink | 148 | almost 3 years ago | |
| BigQuery | 17 | almost 2 years ago | |
| SQL | 38 | almost 2 years ago | |
| GCloud pubsub | 4 | almost 2 years ago | |
| BigTable | 3 | almost 2 years ago | |
| Google Cloud Storage (GCS) | 70 | 3 months ago | |
| HBase | 43 | about 4 years ago | |
| FileSystem | 110 | about 2 years ago | |
| Streaming files from a local filesystem | 324 | 9 days ago | |
| kafka-connect-protobuf-converter | 94 | over 1 year ago | |
| IBM MQ Source | 95 | about 2 months ago | |
| IBM MQ Sink | 35 | about 2 months ago | |
| Azure IoT Hub | 40 | over 4 years ago | |
| Pulsar | 24 | over 1 year ago | |
| HTTP | 30 | over 2 years ago | |
awesome-kafka / Development / Producers | |||
| Maxwell's daemon - a mysql-to-json kafka producer | 4,047 | about 1 month ago | |
awesome-kafka / Development / Consumers | |||
| Spark | 635 | over 2 years ago | |
| Storm | |||
| Mapreduce | |||
| Divolte | 15 | about 10 years ago | |
| Azure Data Explorer/Kusto | 47 | 17 days ago | |
| Firehose | 325 | 2 months ago | Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems |
| Raccoon | 199 | about 2 months ago | Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols and publish to Kafka |
awesome-kafka / Development / Transformations | |||
| Dagger | 267 | about 1 year ago | Dagger is an easy-to-use, SQL framework to process built on top of Apache Flink for stateful processing of Kafka data |
| Single Message Transformations (SMT) | 74 | 18 days ago | |
awesome-kafka / Development / Testing | |||
| Mocked Streams - Topology testing library for Kafka Streams (no broker required) | 187 | almost 4 years ago | |
| Unit-testing for embedded kafka services | 25 | almost 6 years ago | |
| Kafka JUnit | 170 | 4 months ago | |
| Kafka Unit | 174 | about 3 years ago | |
| Kafka for JUnit | 53 | 8 months ago | |
| Karate and Kafka | 26 | 3 months ago | |
| embedded-kafka - in-memory Kafka instance to run your tests against | 386 | 12 days ago | |
| Kafka Containers | |||
| Mockintosh - through asynchronous Kafka actors | |||
awesome-kafka / Operations / CI/CD | |||
| kafka-gitops | 321 | over 1 year ago | |
awesome-kafka / Operations / Operational Utilities | |||
| Cruise control | 2,765 | 11 days ago | |
| DoctorKafka | 634 | almost 3 years ago | and its successor |
| Brucke - Inter-cluster bridge of kafka topics | 44 | almost 2 years ago | |
| Kafka-Kit | 489 | 7 months ago | |
| Secor - service for persisting Kafka logs | 1,847 | 3 days ago | |
| kafkabalancer | 48 | about 7 years ago | |
| Yelp Kafka-Utils | 314 | about 1 year ago | |
| Cerner Common Kafka utilities | 36 | over 1 year ago | |
| connectctl - manage kafka connect connectors easily | 12 | over 2 years ago | |
awesome-kafka / Operations / Monitoring | |||
| Kafka Offset Exporter | 27 | about 1 year ago | |
| Fluent plugin | 303 | 4 months ago | |
| Burrow - Kafka Consumer Lag Checking | 3,762 | 4 months ago | |
| Remora | 196 | about 2 years ago | |
| Kafka Offset Monitoring | 281 | almost 3 years ago | |
| kafka-eagle | 3,084 | 4 months ago | |
| Kafka-prometheus | 135 | 8 months ago | |
| Consumer lag monitoring | 50 | 8 days ago | |
awesome-kafka / Operations / Performance tools | |||
| sangrenel | 205 | over 1 year ago | |
| KLoadGen - load generator plugin for jmeter | 209 | about 2 months ago | |
awesome-kafka / Operations / Security | |||
| Kafka Security Manager | 362 | 10 months ago | |
| Kafka security playbook | 181 | 10 months ago | |
awesome-kafka / Operations / Audit | |||
| chaperone - A Kafka audit system | 641 | almost 4 years ago | |
awesome-kafka / Operations / Mirroring | |||
| MirrorMaker | |||
| Message Handler For MirrorMaker to change topic names | 42 | almost 4 years ago | |
| uReplicator | 917 | 11 months ago | |
| MirrorTool for Kafka Connect | 70 | almost 4 years ago | |
| Mirus | 203 | 25 days ago | |
awesome-kafka / Operations / Backup | |||
| Backup and Restore topics & offsets | 166 | about 1 month ago | |
awesome-kafka / Operations / Tools | |||
| Karapace - Your Kafka essentials in one tool | 473 | about 20 hours ago | |
| REST-proxy | 46 | 8 days ago | |
| Kafka-Pixy - gRPC/REST Proxy | 774 | 7 months ago | |
| Kafka Proxy providing SASL/PLAIN authentication and SSL | 500 | 8 days ago | |
| Reactive/Akka API | 1,417 | 5 days ago | |
| kafkat | 504 | over 5 years ago | |
| kt | 948 | 8 months ago | |
| kafka-connect-tools | 118 | over 3 years ago | |
| Node Kafka connect | 128 | almost 2 years ago | |
| Kafka-monitor | 2,023 | about 1 year ago | |
| Kafka Auto completion | 34 | almost 7 years ago | |
| Kafka Cheet Sheet | 305 | almost 4 years ago | |
| Kafka Write Protocol encoder/decoder | 95 | 17 days ago | |
| Winton Kafka Streams | 313 | almost 6 years ago | |
| Spring-kafka | 2,192 | 7 days ago | |
| Maven quick start for Kafka Connect connectors | 145 | almost 4 years ago | |
| Trifecta | 215 | about 6 years ago | |
| Kafka Specs | 209 | 6 days ago | |
| Strimzi Kafka CLI | 80 | 16 days ago | |
awesome-kafka / Operations / Metadata Management | |||
| Go library for Kafka metadata in Zookeeper | 100 | over 5 years ago | |
| Meteor | 191 | 23 days ago | Meteor is a scalable, easy-to-use, extensible metadata collection framework from the different cloud providers and on-prem sources and publish to Kafka |
awesome-kafka / Operations / Schema Management | |||
| Stencil | 222 | 6 months ago | Stencil is a Protobuf schema registry for Kafka that enables you to create, maintain and consume protobuf messages and APIs dynamically, efficiently, and reliably |
| Schema registry | 2,225 | 8 days ago | |
| Node Schema registry | 24 | over 5 years ago | |
| Schema Registry Transfer SMT | 56 | about 1 year ago | |
| Expedia Stream Registry | 122 | about 2 months ago | |
awesome-kafka / Operations / UI | |||
| kafka-manager | 11,836 | over 1 year ago | |
| Kafka HQ | 3,411 | 11 days ago | |
| Kafdrop 3.x | 5,582 | 8 days ago | – web UI for Kafka. For versions older than 0.10.0 use |
| Kafka Webview | 395 | about 2 months ago | |
| Kafka Topics UI | 877 | almost 3 years ago | |
| Kafka Connect UI | 506 | 11 months ago | |
| Kafka Schema UI | 421 | 9 months ago | |
| Kafkawize - self service topic management portal | 21 | over 1 year ago | |
| Kadmin - Kafka Producer/Consumer UI | 69 | almost 5 years ago | |
| Kowl | 3,829 | 7 days ago | |
| Kafka Streams Topology Sketch Diagram Visualization | 411 | over 1 year ago | |
| Web based GUI | 9,809 | 4 months ago | |
awesome-kafka / Deployment / Kubernetes | |||
| Kafka k8s operator | 297 | about 6 years ago | |
| Strimzi | 4,858 | 6 days ago | Operator for deploying and running Apache Kafka on Kubernetes |
| konsumerator - Kafka Consumer Operator | 22 | over 1 year ago | |
awesome-kafka / Deployment / Docker | |||
| fast-data-dev | 2,022 | 19 days ago | |
awesome-kafka / Deployment / Integrations | |||
| Ecosystem | |||
| hermes | 819 | about 21 hours ago | |
| IBMStreams | 13 | over 2 years ago | |
| Brooklin | 920 | 6 months ago | |
awesome-kafka / Alternative Implementations | |||
| Go | 4,935 | about 1 year ago | |
| C/C++ | 284 | 6 days ago | |
awesome-kafka / Projects built on top of Kafka | |||
| In-Memory Cache Backed by Kafka | 245 | 10 days ago | |
| ransactional Metadata Store Backed by Kafka | 22 | 8 days ago | |
awesome-kafka / Unsorted | |||
| https://github.com/Landoop/kafka-connect-kcql-smt | 32 | over 4 years ago | |
| https://github.com/nodefluent/purpur | 7 | over 5 years ago | |
| Azkarra Streams - lightweight java framework which makes easy to create production-ready Kafka Streams applications | 186 | almost 2 years ago | |
semantalytics/awesome-kafka
monksy/awesome-kafka
fede1024/rust-rdkafka
doujiang24/lua-resty-kafka
deepfabric/elasticell
tikv/tikv
morganstanley/modern-cpp-kafka
karatelabs/karate