awesome-kafka

Kafka toolkit

A curated collection of useful libraries and tools for working with Apache Kafka

A curated list of awesome things related to Apache Kafka

GitHub
247 stars
19 watching
43 forks
last commit: 2 months ago
Linked from 1 awesome list

awesomeawesome-listkafka

Awesome Kafka / Libraries / Kafka
topicctl 603 about 1 month ago A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more
sarama 11,629 about 1 month ago Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later)
kadeck Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
conduktor
mirus 204 about 2 months ago Mirus is a cross data-center data replication tool for Apache Kafka
kareldb 393 about 1 month ago A Relational Database Backed by Apache Kafka
kcache 245 about 1 month ago An In-Memory Cache Backed by Apache Kafka
kafka-lag-based-assignor 12 over 4 years ago Kafka partition assignor that distributes lag evenly across a consumer group
kafka-pixy 773 9 months ago Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control
Burrow 3,776 about 1 month ago Kafka Consumer Lag Checking
kafka-tools 595 almost 3 years ago A collection of tools for working with Apache Kafka
schema-registry-ui 421 11 months ago Web tool for Avro Schema Registry
kafkacat 5,468 6 months ago Generic command line non-JVM Apache Kafka producer and consumer
kafka-jackson 7 almost 7 years ago Kafka Serializer, Deserializer, and Serde for Jackson JSON
kafkahq 3,432 about 1 month ago Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
secor 1,846 about 1 month ago Secor is a service implementing Kafka log persistence
reactive-kafka 1,417 about 1 month ago Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka
kafka-monitor 2,023 over 1 year ago
chaperone 641 almost 4 years ago A Kafka audit system
uReplicator 918 about 1 year ago Improvement of Apache Kafka Mirrormaker
kafka-websocket 347 about 1 year ago
kafka-unit 174 about 3 years ago
cruise-control 2,777 about 2 months ago Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster
jocko 4,939 about 1 year ago Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hermes 820 about 1 month ago Fast and reliable message broker built on top of Kafka
kt 947 9 months ago Kafka command line tool
kafka-eagle 3,090 5 months ago Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information
dockerkafka 633 about 3 years ago DoctorKafka is a service for Kafka cluster auto healing and workload balancing
kasper 439 over 7 years ago Kasper is a lightweight library for processing Kafka topics
Yelp kafka-utils 314 over 1 year ago
kafka-spark-consumer 635 almost 3 years ago
kafka-streams-cep 308 about 1 year ago Complex Event Processing on top of Kafka Streams
spring-kafka 2,244 about 1 month ago
ksql-machine-learning-udf 11 almost 7 years ago
kafkabeat 6 over 7 years ago Elasticsearch Beats for kafka
sangrenel 204 over 1 year ago
Strimzi 4,904 about 1 month ago Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
Kafka Manager 11,853 over 1 year ago Web-based tool for managing a Kafka cluster
Strimzi Kafka CLI 82 2 months ago A CLI for Strimzi Kafka Operator
Kafka Cluster Kraft Mode - DockerCompose 18 6 months ago Workable kafka cluster with kraft mode using docker-compose
Decaton 337 5 months ago High throughput asynchronous task processing framework

Awesome Kafka / Libraries / Kafka UI
KafkaTrail 6 8 months ago
kafka-topics-ui 876 almost 3 years ago
kafka-connect-ui 508 about 1 year ago Web tool for Kafka Connect
Kafdrop 5,620 about 1 month ago Web UI for browsing Kafka topics and consumer groups
kafka-webview 394 about 1 month ago
kafka-view 219 over 2 years ago
cruise-control-ui 350 5 months ago
real-time-ui-with-kafka-streams 31 over 7 years ago
tsujun 53 over 6 years ago Yet another Web UI for KSQL
kowl 3,860 about 1 month ago
akhq

Awesome Kafka / Libraries / Kafka Streams
schema-registry-ui 421 11 months ago
kafka-streams-consumer-offsets-to-json 13 about 5 years ago A Kafka Streams process to convert consumer_offsets to a JSON-readable topic
kafka-operator 295 about 6 years ago
kafkabalancer 48 about 7 years ago
mockedstreams 187 about 4 years ago
winton-kafka-streams 311 almost 6 years ago A Python implementation of Apache Kafka Streams

Awesome Kafka / Libraries / Kafka Connect
Apache Camel Kafka Connect 340+ Apache Camel components as Kafka Connect connectors
kafka-connect-client 40 3 months ago A kafka-connect REST api client for java
Confluent Connector Hub
kafka-connect-protobuf-converter 94 over 1 year ago Protobuf converter plugin for Kafka Connect
Maxwell 4,059 about 1 month ago
kafka-connect-transformers 8 about 6 years ago
kafka-connect-mq-source 95 about 1 month ago
kafka-connect-jenkins 30 about 2 years ago
toketi-kafka-connect-iohub 40 almost 5 years ago
connectctl 12 over 2 years ago Manage kafka connect connectors easily
kafka-connect-transform-common 156 6 months ago Common Transforms for Kafka Connect
aiven-kafka-connect-transforms 75 about 2 months ago A collection of Single Message Transformations (SMTs) for Kafka Connect
kafka-connect-file-pulse 324 about 2 months ago A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
kafka-connect-http 30 over 2 years ago
snowflake-kafka-connector 141 about 1 month ago

Awesome Kafka / Libraries / REST Proxy
kafka-rest 54 about 1 month ago Confluent REST Proxy
strimzi-http-bridge 293 about 1 month ago Strimzi Kafka Bridge (AMQP & HTTP)
Zilla 553 about 1 month ago An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol

Awesome Kafka / Libraries / Schema Registry
apicurio-registry 606 about 1 month ago Apicurio API/schema registry (includes UI)
schema-registry 2,233 about 1 month ago Confluent Schema registry for Kafka
ksql-jdbc-driver 87 almost 3 years ago

Awesome Kafka / Libraries / Other Awesome Kafka
ApacheKafka 35 about 1 year ago
dharmeshkakadia/awesome-kafka 206 over 1 year ago
infoslack/awesome-kafka 578 11 months ago
awesome-kafka-playground 1 over 6 years ago

Awesome Kafka / Libraries / Kafkaesque
Pulsar 14,315 about 1 month ago
Flink
Spark
Heron 3,638 almost 2 years ago
Beam
Arrow
Samza
Nats
Chronicle-Queue 3,350 about 1 month ago
CorfuDB 642 about 1 month ago
GenStage
faust 6,751 6 months ago Python Stream Processing
redpanda
Meteor 23 over 2 years ago Lightweight, broker-less alternative to Kafka for message streaming

Awesome Kafka / Resources / Learning
Passionate Developer: Kafka Streams DSL vs processor API
Mastering Kafka Streams
Spark Streaming + Kafka Integration Guide

Awesome Kafka / Resources / Blogs
CodingJunkie Random Thoughts on Coding by Bill Bejeck
Understanding Kafka with Factorio

Awesome Kafka / Resources / Books
Kafka Streams in Action
Kafka in Action
Kafka the Definitive Guide
Streaming Data Pipelines with Kafka

Backlinks from these awesome lists:

More related projects: