awesome-kafka

Everything about Apache Kafka

GitHub
202 stars
15 watching
30 forks
last commit: about 1 year ago
Linked from 2 awesome lists

awesomeawsome-listkafka

awesome-kafka / Development / Client libraries
Go 11,412 11 days ago
Go 130 over 7 years ago
Go 216 almost 5 years ago
Go Kafka streaming 2,348 10 days ago
Node
Node 297 about 3 years ago
Node 291 almost 2 years ago
Node Kafka streams 831 over 1 year ago
Node librdkafka 0 over 6 years ago
Php 1,442 over 2 years ago
Php 2,073 10 days ago
Perl 40 about 4 years ago
Python (asyncio) 1,124 25 days ago
Python (kafka-python) 5,586 2 months ago
Python (pykafka) 1,119 over 3 years ago
Ruby 1,271 over 1 year ago
Phobos - Ruby App framework 220 about 1 year ago
Karafka - Ruby and Rails framework 2,064 5 days ago
Dart 55 almost 2 years ago
JRuby 71 about 8 years ago
Elixir 595 3 months ago
Erlang 658 14 days ago
Rust 1,215 2 months ago
LinkedIn wrapper library 131 about 1 year ago
kasper - a lightweight library for processing Kafka topics 439 about 7 years ago
others and

awesome-kafka / Development / Kafka Streams Libraries
Scala 190 over 1 year ago
faust - Python Stream Processing 6,722 2 months ago
Southpaw - Streaming left joins for data change capture 52 4 months ago

awesome-kafka / Development / KSQL
KSQL JDBC Driver 88 over 2 years ago
KSQL Addon - User Defined Function (UDF) for Machine Learning 12 over 6 years ago

awesome-kafka / Development / Connectors
JDBC 17 10 days ago
Elasticsearch 11 19 days ago
HDFS 7 19 days ago
Beast - Kafka to BigQuery Sink 148 over 2 years ago
BigQuery 17 almost 2 years ago
SQL 39 almost 2 years ago
GCloud pubsub 4 over 1 year ago
BigTable 3 almost 2 years ago
Google Cloud Storage (GCS) 70 about 1 month ago
HBase 43 almost 4 years ago
FileSystem 111 almost 2 years ago
Streaming files from a local filesystem 321 10 days ago
kafka-connect-protobuf-converter 94 about 1 year ago
IBM MQ Source 93 about 2 months ago
IBM MQ Sink 35 about 2 months ago
Azure IoT Hub 40 over 4 years ago
Pulsar 24 over 1 year ago
HTTP 30 about 2 years ago

awesome-kafka / Development / Producers
Maxwell's daemon - a mysql-to-json kafka producer 4,003 10 days ago

awesome-kafka / Development / Consumers
Spark 632 over 2 years ago
Storm
Mapreduce 76 over 8 years ago
Divolte 15 almost 10 years ago
Azure Data Explorer/Kusto 46 10 days ago
Firehose 321 16 days ago Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems
Raccoon 197 16 days ago Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols and publish to Kafka

awesome-kafka / Development / Transformations
Dagger 266 about 1 year ago Dagger is an easy-to-use, SQL framework to process built on top of Apache Flink for stateful processing of Kafka data
Single Message Transformations (SMT) 72 26 days ago

awesome-kafka / Development / Testing
Mocked Streams - Topology testing library for Kafka Streams (no broker required) 186 over 3 years ago
Unit-testing for embedded kafka services 25 almost 6 years ago
Kafka JUnit 168 2 months ago
Kafka Unit 174 almost 3 years ago
Kafka for JUnit 53 6 months ago
Karate and Kafka 24 about 1 month ago
embedded-kafka - in-memory Kafka instance to run your tests against 381 18 days ago
Kafka Containers
Mockintosh - through asynchronous Kafka actors

awesome-kafka / Operations / CI/CD
kafka-gitops 316 over 1 year ago

awesome-kafka / Operations / Operational Utilities
Cruise control 2,734 20 days ago
DoctorKafka 632 almost 3 years ago and its successor
Brucke - Inter-cluster bridge of kafka topics 44 over 1 year ago
Kafka-Kit 487 5 months ago
Secor - service for persisting Kafka logs 1,844 about 1 month ago
kafkabalancer 48 almost 7 years ago
Yelp Kafka-Utils 314 about 1 year ago
Cerner Common Kafka utilities 36 about 1 year ago
connectctl - manage kafka connect connectors easily 12 over 2 years ago

awesome-kafka / Operations / Monitoring
Kafka Offset Exporter 27 11 months ago
Fluent plugin 303 about 2 months ago
Burrow - Kafka Consumer Lag Checking 3,725 2 months ago
Remora 197 almost 2 years ago
Kafka Offset Monitoring 282 over 2 years ago
kafka-eagle 3,065 about 2 months ago
Kafka-prometheus 134 6 months ago
Consumer lag monitoring 50 6 months ago

awesome-kafka / Operations / Performance tools
sangrenel 206 over 1 year ago
KLoadGen - load generator plugin for jmeter 206 4 months ago

awesome-kafka / Operations / Security
Kafka Security Manager 359 8 months ago
Kafka security playbook 180 8 months ago

awesome-kafka / Operations / Audit
chaperone - A Kafka audit system 640 over 3 years ago

awesome-kafka / Operations / Mirroring
MirrorMaker
Message Handler For MirrorMaker to change topic names 42 over 3 years ago
uReplicator 910 10 months ago
MirrorTool for Kafka Connect 70 almost 4 years ago
Mirus 201 2 months ago

awesome-kafka / Operations / Backup
Backup and Restore topics & offsets 165 almost 3 years ago

awesome-kafka / Operations / Tools
Karapace - Your Kafka essentials in one tool 452 10 days ago
REST-proxy 33 10 days ago
Kafka-Pixy - gRPC/REST Proxy 768 5 months ago
Kafka Proxy providing SASL/PLAIN authentication and SSL 487 2 months ago
Reactive/Akka API 1,417 2 days ago
kafkat 503 over 5 years ago
kt 950 6 months ago
kafka-connect-tools 118 over 3 years ago
Node Kafka connect 128 almost 2 years ago
Kafka-monitor 2,015 12 months ago
Kafka Auto completion 34 almost 7 years ago
Kafka Cheet Sheet 305 almost 4 years ago
Kafka Write Protocol encoder/decoder 95 15 days ago
Winton Kafka Streams 314 over 5 years ago
Spring-kafka 2,159 12 days ago
Maven quick start for Kafka Connect connectors 145 over 3 years ago
Trifecta 215 almost 6 years ago
Kafka Specs 193 17 days ago
Strimzi Kafka CLI 78 27 days ago

awesome-kafka / Operations / Metadata Management
Go library for Kafka metadata in Zookeeper 100 over 5 years ago
Meteor 185 4 months ago Meteor is a scalable, easy-to-use, extensible metadata collection framework from the different cloud providers and on-prem sources and publish to Kafka

awesome-kafka / Operations / Schema Management
Stencil 220 4 months ago Stencil is a Protobuf schema registry for Kafka that enables you to create, maintain and consume protobuf messages and APIs dynamically, efficiently, and reliably
Schema registry 2,200 10 days ago
Node Schema registry 24 over 5 years ago
Schema Registry Transfer SMT 54 12 months ago
Expedia Stream Registry 122 3 months ago

awesome-kafka / Operations / UI
kafka-manager 11,808 about 1 year ago
Kafka HQ 3,356 11 days ago
Kafdrop 3.x 5,453 15 days ago – web UI for Kafka. For versions older than 0.10.0 use
Kafka Webview 395 about 2 months ago
Kafka Topics UI 877 over 2 years ago
Kafka Connect UI 502 10 months ago
Kafka Schema UI 420 8 months ago
Kafkawize - self service topic management portal 21 about 1 year ago
Kadmin - Kafka Producer/Consumer UI 69 over 4 years ago
Kowl 3,754 10 days ago
Kafka Streams Topology Sketch Diagram Visualization 411 over 1 year ago
Web based GUI 9,490 2 months ago

awesome-kafka / Deployment / Kubernetes
Kafka k8s operator 297 almost 6 years ago
Strimzi 4,775 4 days ago Operator for deploying and running Apache Kafka on Kubernetes
konsumerator - Kafka Consumer Operator 22 over 1 year ago

awesome-kafka / Deployment / Docker
fast-data-dev 2,011 about 1 month ago

awesome-kafka / Deployment / Integrations
Ecosystem
hermes 811 16 days ago
IBMStreams 13 about 2 years ago
Brooklin 913 4 months ago

awesome-kafka / Alternative Implementations
Go 4,923 11 months ago
C/C++ 199 12 days ago

awesome-kafka / Projects built on top of Kafka
In-Memory Cache Backed by Kafka 243 12 days ago
ransactional Metadata Store Backed by Kafka 21 12 days ago

awesome-kafka / Unsorted
https://github.com/Landoop/kafka-connect-kcql-smt 32 over 4 years ago
https://github.com/nodefluent/purpur 7 over 5 years ago
Azkarra Streams - lightweight java framework which makes easy to create production-ready Kafka Streams applications 186 over 1 year ago

Backlinks from these awesome lists: