awesome-kafka

Kafka toolkit

A curated collection of useful libraries and tools for working with Apache Kafka

A curated list of awesome things related to Apache Kafka

GitHub
247 stars
19 watching
43 forks
last commit: 12 months ago
Linked from 1 awesome list

awesomeawesome-listkafka

Awesome Kafka / Libraries / Kafka
topicctl 603 11 months ago A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more
sarama 11,629 11 months ago Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later)
kadeck Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
conduktor
mirus 204 12 months ago Mirus is a cross data-center data replication tool for Apache Kafka
kareldb 393 11 months ago A Relational Database Backed by Apache Kafka
kcache 245 11 months ago An In-Memory Cache Backed by Apache Kafka
kafka-lag-based-assignor 12 about 5 years ago Kafka partition assignor that distributes lag evenly across a consumer group
kafka-pixy 773 over 1 year ago Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control
Burrow 3,776 11 months ago Kafka Consumer Lag Checking
kafka-tools 595 almost 4 years ago A collection of tools for working with Apache Kafka
schema-registry-ui 421 over 1 year ago Web tool for Avro Schema Registry
kafkacat 5,468 over 1 year ago Generic command line non-JVM Apache Kafka producer and consumer
kafka-jackson 7 over 7 years ago Kafka Serializer, Deserializer, and Serde for Jackson JSON
kafkahq 3,432 11 months ago Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
secor 1,846 11 months ago Secor is a service implementing Kafka log persistence
reactive-kafka 1,417 11 months ago Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka
kafka-monitor 2,023 about 2 years ago
chaperone 641 almost 5 years ago A Kafka audit system
uReplicator 918 almost 2 years ago Improvement of Apache Kafka Mirrormaker
kafka-websocket 347 almost 2 years ago
kafka-unit 174 about 4 years ago
cruise-control 2,777 11 months ago Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster
jocko 4,939 almost 2 years ago Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hermes 820 11 months ago Fast and reliable message broker built on top of Kafka
kt 947 over 1 year ago Kafka command line tool
kafka-eagle 3,090 about 1 year ago Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information
dockerkafka 633 almost 4 years ago DoctorKafka is a service for Kafka cluster auto healing and workload balancing
kasper 439 about 8 years ago Kasper is a lightweight library for processing Kafka topics
Yelp kafka-utils 314 about 2 years ago
kafka-spark-consumer 635 over 3 years ago
kafka-streams-cep 308 almost 2 years ago Complex Event Processing on top of Kafka Streams
spring-kafka 2,244 11 months ago
ksql-machine-learning-udf 11 over 7 years ago
kafkabeat 6 about 8 years ago Elasticsearch Beats for kafka
sangrenel 204 over 2 years ago
Strimzi 4,904 11 months ago Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
Kafka Manager 11,853 over 2 years ago Web-based tool for managing a Kafka cluster
Strimzi Kafka CLI 82 12 months ago A CLI for Strimzi Kafka Operator
Kafka Cluster Kraft Mode - DockerCompose 18 over 1 year ago Workable kafka cluster with kraft mode using docker-compose
Decaton 337 about 1 year ago High throughput asynchronous task processing framework

Awesome Kafka / Libraries / Kafka UI
KafkaTrail 6 over 1 year ago
kafka-topics-ui 876 over 3 years ago
kafka-connect-ui 508 almost 2 years ago Web tool for Kafka Connect
Kafdrop 5,620 11 months ago Web UI for browsing Kafka topics and consumer groups
kafka-webview 394 11 months ago
kafka-view 219 about 3 years ago
cruise-control-ui 350 about 1 year ago
real-time-ui-with-kafka-streams 31 about 8 years ago
tsujun 53 over 7 years ago Yet another Web UI for KSQL
kowl 3,860 11 months ago
akhq

Awesome Kafka / Libraries / Kafka Streams
schema-registry-ui 421 over 1 year ago
kafka-streams-consumer-offsets-to-json 13 about 6 years ago A Kafka Streams process to convert consumer_offsets to a JSON-readable topic
kafka-operator 295 about 7 years ago
kafkabalancer 48 almost 8 years ago
mockedstreams 187 almost 5 years ago
winton-kafka-streams 311 almost 7 years ago A Python implementation of Apache Kafka Streams

Awesome Kafka / Libraries / Kafka Connect
Apache Camel Kafka Connect 340+ Apache Camel components as Kafka Connect connectors
kafka-connect-client 40 about 1 year ago A kafka-connect REST api client for java
Confluent Connector Hub
kafka-connect-protobuf-converter 94 over 2 years ago Protobuf converter plugin for Kafka Connect
Maxwell 4,059 11 months ago
kafka-connect-transformers 8 almost 7 years ago
kafka-connect-mq-source 95 11 months ago
kafka-connect-jenkins 30 almost 3 years ago
toketi-kafka-connect-iohub 40 over 5 years ago
connectctl 12 over 3 years ago Manage kafka connect connectors easily
kafka-connect-transform-common 156 over 1 year ago Common Transforms for Kafka Connect
aiven-kafka-connect-transforms 75 11 months ago A collection of Single Message Transformations (SMTs) for Kafka Connect
kafka-connect-file-pulse 324 11 months ago A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
kafka-connect-http 30 over 3 years ago
snowflake-kafka-connector 141 11 months ago

Awesome Kafka / Libraries / REST Proxy
kafka-rest 54 11 months ago Confluent REST Proxy
strimzi-http-bridge 293 11 months ago Strimzi Kafka Bridge (AMQP & HTTP)
Zilla 553 11 months ago An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol

Awesome Kafka / Libraries / Schema Registry
apicurio-registry 606 11 months ago Apicurio API/schema registry (includes UI)
schema-registry 2,233 11 months ago Confluent Schema registry for Kafka
ksql-jdbc-driver 87 over 3 years ago

Awesome Kafka / Libraries / Other Awesome Kafka
ApacheKafka 35 almost 2 years ago
dharmeshkakadia/awesome-kafka 206 over 2 years ago
infoslack/awesome-kafka 578 over 1 year ago
awesome-kafka-playground 1 over 7 years ago

Awesome Kafka / Libraries / Kafkaesque
Pulsar 14,315 11 months ago
Flink
Spark
Heron 3,638 over 2 years ago
Beam
Arrow
Samza
Nats
Chronicle-Queue 3,350 11 months ago
CorfuDB 642 11 months ago
GenStage
faust 6,751 over 1 year ago Python Stream Processing
redpanda
Meteor 23 over 3 years ago Lightweight, broker-less alternative to Kafka for message streaming

Awesome Kafka / Resources / Learning
Passionate Developer: Kafka Streams DSL vs processor API
Mastering Kafka Streams
Spark Streaming + Kafka Integration Guide

Awesome Kafka / Resources / Blogs
CodingJunkie Random Thoughts on Coding by Bill Bejeck
Understanding Kafka with Factorio

Awesome Kafka / Resources / Books
Kafka Streams in Action
Kafka in Action
Kafka the Definitive Guide
Streaming Data Pipelines with Kafka

Backlinks from these awesome lists:

More related projects: