awesome-kafka

Kafka toolkit

A curated collection of useful libraries and tools for working with Apache Kafka

A curated list of awesome things related to Apache Kafka

GitHub
247 stars
19 watching
43 forks
last commit: 10 months ago
Linked from 1 awesome list

awesomeawesome-listkafka

Awesome Kafka / Libraries / Kafka
topicctl 603 9 months ago A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more
sarama 11,629 9 months ago Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later)
kadeck Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
conduktor
mirus 204 10 months ago Mirus is a cross data-center data replication tool for Apache Kafka
kareldb 393 9 months ago A Relational Database Backed by Apache Kafka
kcache 245 9 months ago An In-Memory Cache Backed by Apache Kafka
kafka-lag-based-assignor 12 almost 5 years ago Kafka partition assignor that distributes lag evenly across a consumer group
kafka-pixy 773 over 1 year ago Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control
Burrow 3,776 9 months ago Kafka Consumer Lag Checking
kafka-tools 595 over 3 years ago A collection of tools for working with Apache Kafka
schema-registry-ui 421 over 1 year ago Web tool for Avro Schema Registry
kafkacat 5,468 about 1 year ago Generic command line non-JVM Apache Kafka producer and consumer
kafka-jackson 7 over 7 years ago Kafka Serializer, Deserializer, and Serde for Jackson JSON
kafkahq 3,432 9 months ago Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
secor 1,846 9 months ago Secor is a service implementing Kafka log persistence
reactive-kafka 1,417 9 months ago Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka
kafka-monitor 2,023 almost 2 years ago
chaperone 641 over 4 years ago A Kafka audit system
uReplicator 918 over 1 year ago Improvement of Apache Kafka Mirrormaker
kafka-websocket 347 over 1 year ago
kafka-unit 174 almost 4 years ago
cruise-control 2,777 9 months ago Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster
jocko 4,939 almost 2 years ago Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hermes 820 9 months ago Fast and reliable message broker built on top of Kafka
kt 947 over 1 year ago Kafka command line tool
kafka-eagle 3,090 about 1 year ago Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information
dockerkafka 633 over 3 years ago DoctorKafka is a service for Kafka cluster auto healing and workload balancing
kasper 439 almost 8 years ago Kasper is a lightweight library for processing Kafka topics
Yelp kafka-utils 314 about 2 years ago
kafka-spark-consumer 635 over 3 years ago
kafka-streams-cep 308 over 1 year ago Complex Event Processing on top of Kafka Streams
spring-kafka 2,244 9 months ago
ksql-machine-learning-udf 11 over 7 years ago
kafkabeat 6 almost 8 years ago Elasticsearch Beats for kafka
sangrenel 204 over 2 years ago
Strimzi 4,904 9 months ago Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
Kafka Manager 11,853 about 2 years ago Web-based tool for managing a Kafka cluster
Strimzi Kafka CLI 82 10 months ago A CLI for Strimzi Kafka Operator
Kafka Cluster Kraft Mode - DockerCompose 18 about 1 year ago Workable kafka cluster with kraft mode using docker-compose
Decaton 337 12 months ago High throughput asynchronous task processing framework

Awesome Kafka / Libraries / Kafka UI
KafkaTrail 6 about 1 year ago
kafka-topics-ui 876 over 3 years ago
kafka-connect-ui 508 over 1 year ago Web tool for Kafka Connect
Kafdrop 5,620 9 months ago Web UI for browsing Kafka topics and consumer groups
kafka-webview 394 9 months ago
kafka-view 219 almost 3 years ago
cruise-control-ui 350 about 1 year ago
real-time-ui-with-kafka-streams 31 almost 8 years ago
tsujun 53 about 7 years ago Yet another Web UI for KSQL
kowl 3,860 9 months ago
akhq

Awesome Kafka / Libraries / Kafka Streams
schema-registry-ui 421 over 1 year ago
kafka-streams-consumer-offsets-to-json 13 almost 6 years ago A Kafka Streams process to convert consumer_offsets to a JSON-readable topic
kafka-operator 295 almost 7 years ago
kafkabalancer 48 almost 8 years ago
mockedstreams 187 over 4 years ago
winton-kafka-streams 311 over 6 years ago A Python implementation of Apache Kafka Streams

Awesome Kafka / Libraries / Kafka Connect
Apache Camel Kafka Connect 340+ Apache Camel components as Kafka Connect connectors
kafka-connect-client 40 11 months ago A kafka-connect REST api client for java
Confluent Connector Hub
kafka-connect-protobuf-converter 94 about 2 years ago Protobuf converter plugin for Kafka Connect
Maxwell 4,059 9 months ago
kafka-connect-transformers 8 almost 7 years ago
kafka-connect-mq-source 95 9 months ago
kafka-connect-jenkins 30 almost 3 years ago
toketi-kafka-connect-iohub 40 over 5 years ago
connectctl 12 about 3 years ago Manage kafka connect connectors easily
kafka-connect-transform-common 156 about 1 year ago Common Transforms for Kafka Connect
aiven-kafka-connect-transforms 75 9 months ago A collection of Single Message Transformations (SMTs) for Kafka Connect
kafka-connect-file-pulse 324 9 months ago A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
kafka-connect-http 30 about 3 years ago
snowflake-kafka-connector 141 9 months ago

Awesome Kafka / Libraries / REST Proxy
kafka-rest 54 9 months ago Confluent REST Proxy
strimzi-http-bridge 293 9 months ago Strimzi Kafka Bridge (AMQP & HTTP)
Zilla 553 9 months ago An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol

Awesome Kafka / Libraries / Schema Registry
apicurio-registry 606 9 months ago Apicurio API/schema registry (includes UI)
schema-registry 2,233 9 months ago Confluent Schema registry for Kafka
ksql-jdbc-driver 87 over 3 years ago

Awesome Kafka / Libraries / Other Awesome Kafka
ApacheKafka 35 over 1 year ago
dharmeshkakadia/awesome-kafka 206 about 2 years ago
infoslack/awesome-kafka 578 over 1 year ago
awesome-kafka-playground 1 over 7 years ago

Awesome Kafka / Libraries / Kafkaesque
Pulsar 14,315 9 months ago
Flink
Spark
Heron 3,638 over 2 years ago
Beam
Arrow
Samza
Nats
Chronicle-Queue 3,350 9 months ago
CorfuDB 642 9 months ago
GenStage
faust 6,751 about 1 year ago Python Stream Processing
redpanda
Meteor 23 about 3 years ago Lightweight, broker-less alternative to Kafka for message streaming

Awesome Kafka / Resources / Learning
Passionate Developer: Kafka Streams DSL vs processor API
Mastering Kafka Streams
Spark Streaming + Kafka Integration Guide

Awesome Kafka / Resources / Blogs
CodingJunkie Random Thoughts on Coding by Bill Bejeck
Understanding Kafka with Factorio

Awesome Kafka / Resources / Books
Kafka Streams in Action
Kafka in Action
Kafka the Definitive Guide
Streaming Data Pipelines with Kafka

Backlinks from these awesome lists:

More related projects: