awesome-kafka
Kafka toolkit
A comprehensive resource and collection of tools and libraries for Apache Kafka development
Everything about Apache Kafka
206 stars
14 watching
30 forks
last commit: about 2 years ago
Linked from 2 awesome lists
awesomeawsome-listkafka
awesome-kafka / Development / Client libraries | |||
| Go | 11,629 | 10 months ago | |
| Go | 129 | over 8 years ago | |
| Go | 216 | almost 6 years ago | |
| Go Kafka streaming | 2,367 | about 1 year ago | |
| Node | |||
| Node | 297 | 12 months ago | |
| Node | 290 | almost 3 years ago | |
| Node Kafka streams | 831 | over 2 years ago | |
| Node librdkafka | 0 | over 7 years ago | |
| Php | 1,442 | over 3 years ago | |
| Php | 2,095 | 11 months ago | |
| Perl | 41 | about 5 years ago | |
| Python (asyncio) | 1,178 | 10 months ago | |
| Python (kafka-python) | 5,650 | 10 months ago | |
| Python (pykafka) | 1,121 | over 4 years ago | |
| Ruby | 1,275 | over 2 years ago | |
| Phobos - Ruby App framework | 220 | about 2 years ago | |
| Karafka - Ruby and Rails framework | 2,100 | 10 months ago | |
| Dart | 56 | almost 3 years ago | |
| JRuby | 71 | about 9 years ago | |
| Elixir | 599 | 10 months ago | |
| Erlang | 666 | 10 months ago | |
| Rust | 1,262 | 12 months ago | |
| LinkedIn wrapper library | 133 | over 2 years ago | |
| kasper - a lightweight library for processing Kafka topics | 439 | about 8 years ago | |
| others | and | ||
awesome-kafka / Development / Kafka Streams Libraries | |||
| Scala | 190 | over 2 years ago | |
| faust - Python Stream Processing | 6,751 | about 1 year ago | |
| Southpaw - Streaming left joins for data change capture | 52 | over 1 year ago | |
awesome-kafka / Development / KSQL | |||
| KSQL JDBC Driver | 87 | over 3 years ago | |
| KSQL Addon - User Defined Function (UDF) for Machine Learning | 11 | over 7 years ago | |
awesome-kafka / Development / Connectors | |||
| JDBC | 7 | 11 months ago | |
| Elasticsearch | 20 | 10 months ago | |
| HDFS | 15 | 10 months ago | |
| Beast - Kafka to BigQuery Sink | 148 | over 3 years ago | |
| BigQuery | 17 | almost 3 years ago | |
| SQL | 38 | almost 3 years ago | |
| GCloud pubsub | 4 | over 2 years ago | |
| BigTable | 3 | almost 3 years ago | |
| Google Cloud Storage (GCS) | 70 | about 1 year ago | |
| HBase | 43 | almost 5 years ago | |
| FileSystem | 110 | almost 3 years ago | |
| Streaming files from a local filesystem | 324 | 10 months ago | |
| kafka-connect-protobuf-converter | 94 | over 2 years ago | |
| IBM MQ Source | 95 | 10 months ago | |
| IBM MQ Sink | 35 | about 1 year ago | |
| Azure IoT Hub | 40 | over 5 years ago | |
| Pulsar | 24 | over 2 years ago | |
| HTTP | 30 | over 3 years ago | |
awesome-kafka / Development / Producers | |||
| Maxwell's daemon - a mysql-to-json kafka producer | 4,059 | 10 months ago | |
awesome-kafka / Development / Consumers | |||
| Spark | 635 | over 3 years ago | |
| Storm | |||
| Mapreduce | |||
| Divolte | 15 | almost 11 years ago | |
| Azure Data Explorer/Kusto | 48 | 10 months ago | |
| Firehose | 329 | about 1 year ago | Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems |
| Raccoon | 200 | about 1 year ago | Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols and publish to Kafka |
awesome-kafka / Development / Transformations | |||
| Dagger | 268 | about 2 years ago | Dagger is an easy-to-use, SQL framework to process built on top of Apache Flink for stateful processing of Kafka data |
| Single Message Transformations (SMT) | 75 | 10 months ago | |
awesome-kafka / Development / Testing | |||
| Mocked Streams - Topology testing library for Kafka Streams (no broker required) | 187 | over 4 years ago | |
| Unit-testing for embedded kafka services | 25 | almost 7 years ago | |
| Kafka JUnit | 170 | 11 months ago | |
| Kafka Unit | 174 | almost 4 years ago | |
| Kafka for JUnit | 53 | 11 months ago | |
| Karate and Kafka | 26 | about 1 year ago | |
| embedded-kafka - in-memory Kafka instance to run your tests against | 386 | 10 months ago | |
| Kafka Containers | |||
| Mockintosh - through asynchronous Kafka actors | |||
awesome-kafka / Operations / CI/CD | |||
| kafka-gitops | 323 | over 2 years ago | |
awesome-kafka / Operations / Operational Utilities | |||
| Cruise control | 2,777 | 10 months ago | |
| DoctorKafka | 633 | almost 4 years ago | and its successor |
| Brucke - Inter-cluster bridge of kafka topics | 44 | almost 3 years ago | |
| Kafka-Kit | 488 | over 1 year ago | |
| Secor - service for persisting Kafka logs | 1,846 | 10 months ago | |
| kafkabalancer | 48 | almost 8 years ago | |
| Yelp Kafka-Utils | 314 | about 2 years ago | |
| Cerner Common Kafka utilities | 36 | about 2 years ago | |
| connectctl - manage kafka connect connectors easily | 12 | over 3 years ago | |
awesome-kafka / Operations / Monitoring | |||
| Kafka Offset Exporter | 27 | almost 2 years ago | |
| Fluent plugin | 303 | about 1 year ago | |
| Burrow - Kafka Consumer Lag Checking | 3,776 | 10 months ago | |
| Remora | 196 | about 3 years ago | |
| Kafka Offset Monitoring | 281 | over 3 years ago | |
| kafka-eagle | 3,090 | about 1 year ago | |
| Kafka-prometheus | 136 | over 1 year ago | |
| Consumer lag monitoring | 50 | 11 months ago | |
awesome-kafka / Operations / Performance tools | |||
| sangrenel | 204 | over 2 years ago | |
| KLoadGen - load generator plugin for jmeter | 208 | 11 months ago | |
awesome-kafka / Operations / Security | |||
| Kafka Security Manager | 362 | over 1 year ago | |
| Kafka security playbook | 180 | over 1 year ago | |
awesome-kafka / Operations / Audit | |||
| chaperone - A Kafka audit system | 641 | over 4 years ago | |
awesome-kafka / Operations / Mirroring | |||
| MirrorMaker | |||
| Message Handler For MirrorMaker to change topic names | 42 | over 4 years ago | |
| uReplicator | 918 | almost 2 years ago | |
| MirrorTool for Kafka Connect | 70 | almost 5 years ago | |
| Mirus | 204 | 11 months ago | |
awesome-kafka / Operations / Backup | |||
| Backup and Restore topics & offsets | 164 | 10 months ago | |
awesome-kafka / Operations / Tools | |||
| Karapace - Your Kafka essentials in one tool | 478 | 10 months ago | |
| REST-proxy | 54 | 10 months ago | |
| Kafka-Pixy - gRPC/REST Proxy | 773 | over 1 year ago | |
| Kafka Proxy providing SASL/PLAIN authentication and SSL | 504 | 10 months ago | |
| Reactive/Akka API | 1,417 | 10 months ago | |
| kafkat | 504 | over 6 years ago | |
| kt | 947 | over 1 year ago | |
| kafka-connect-tools | 118 | over 4 years ago | |
| Node Kafka connect | 128 | almost 3 years ago | |
| Kafka-monitor | 2,023 | almost 2 years ago | |
| Kafka Auto completion | 34 | almost 8 years ago | |
| Kafka Cheet Sheet | 306 | almost 5 years ago | |
| Kafka Write Protocol encoder/decoder | 95 | 11 months ago | |
| Winton Kafka Streams | 311 | over 6 years ago | |
| Spring-kafka | 2,244 | 10 months ago | |
| Maven quick start for Kafka Connect connectors | 146 | over 4 years ago | |
| Trifecta | 215 | almost 7 years ago | |
| Kafka Specs | 216 | 10 months ago | |
| Strimzi Kafka CLI | 82 | 11 months ago | |
awesome-kafka / Operations / Metadata Management | |||
| Go library for Kafka metadata in Zookeeper | 100 | over 6 years ago | |
| Meteor | 193 | 11 months ago | Meteor is a scalable, easy-to-use, extensible metadata collection framework from the different cloud providers and on-prem sources and publish to Kafka |
awesome-kafka / Operations / Schema Management | |||
| Stencil | 222 | over 1 year ago | Stencil is a Protobuf schema registry for Kafka that enables you to create, maintain and consume protobuf messages and APIs dynamically, efficiently, and reliably |
| Schema registry | 2,233 | 10 months ago | |
| Node Schema registry | 24 | over 6 years ago | |
| Schema Registry Transfer SMT | 56 | about 2 years ago | |
| Expedia Stream Registry | 122 | 10 months ago | |
awesome-kafka / Operations / UI | |||
| kafka-manager | 11,853 | about 2 years ago | |
| Kafka HQ | 3,432 | 10 months ago | |
| Kafdrop 3.x | 5,620 | 10 months ago | – web UI for Kafka. For versions older than 0.10.0 use |
| Kafka Webview | 394 | 10 months ago | |
| Kafka Topics UI | 876 | over 3 years ago | |
| Kafka Connect UI | 508 | almost 2 years ago | |
| Kafka Schema UI | 421 | over 1 year ago | |
| Kafkawize - self service topic management portal | 21 | over 2 years ago | |
| Kadmin - Kafka Producer/Consumer UI | 69 | over 5 years ago | |
| Kowl | 3,860 | 10 months ago | |
| Kafka Streams Topology Sketch Diagram Visualization | 411 | over 2 years ago | |
| Web based GUI | 9,986 | about 1 year ago | |
awesome-kafka / Deployment / Kubernetes | |||
| Kafka k8s operator | 295 | almost 7 years ago | |
| Strimzi | 4,904 | 10 months ago | Operator for deploying and running Apache Kafka on Kubernetes |
| konsumerator - Kafka Consumer Operator | 22 | over 2 years ago | |
awesome-kafka / Deployment / Docker | |||
| fast-data-dev | 2,024 | 10 months ago | |
awesome-kafka / Deployment / Integrations | |||
| Ecosystem | |||
| hermes | 820 | 10 months ago | |
| IBMStreams | 13 | about 3 years ago | |
| Brooklin | 931 | over 1 year ago | |
awesome-kafka / Alternative Implementations | |||
| Go | 4,939 | almost 2 years ago | |
| C/C++ | 332 | 10 months ago | |
awesome-kafka / Projects built on top of Kafka | |||
| In-Memory Cache Backed by Kafka | 245 | 10 months ago | |
| ransactional Metadata Store Backed by Kafka | 22 | 11 months ago | |
awesome-kafka / Unsorted | |||
| https://github.com/Landoop/kafka-connect-kcql-smt | 32 | over 5 years ago | |
| https://github.com/nodefluent/purpur | 7 | over 6 years ago | |
| Azkarra Streams - lightweight java framework which makes easy to create production-ready Kafka Streams applications | 185 | almost 3 years ago | |
semantalytics/awesome-kafka
monksy/awesome-kafka
fede1024/rust-rdkafka
doujiang24/lua-resty-kafka
deepfabric/elasticell
tikv/tikv
morganstanley/modern-cpp-kafka
karatelabs/karate