awesome-kafka

Kafka toolkit

A curated collection of useful libraries and tools for working with Apache Kafka

A curated list of awesome things related to Apache Kafka

GitHub
247 stars
19 watching
43 forks
last commit: 4 months ago
Linked from 1 awesome list

awesomeawesome-listkafka

Awesome Kafka / Libraries / Kafka
topicctl 603 2 months ago A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more
sarama 11,629 2 months ago Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later)
kadeck Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
conduktor
mirus 204 3 months ago Mirus is a cross data-center data replication tool for Apache Kafka
kareldb 393 2 months ago A Relational Database Backed by Apache Kafka
kcache 245 2 months ago An In-Memory Cache Backed by Apache Kafka
kafka-lag-based-assignor 12 over 4 years ago Kafka partition assignor that distributes lag evenly across a consumer group
kafka-pixy 773 10 months ago Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control
Burrow 3,776 2 months ago Kafka Consumer Lag Checking
kafka-tools 595 about 3 years ago A collection of tools for working with Apache Kafka
schema-registry-ui 421 about 1 year ago Web tool for Avro Schema Registry
kafkacat 5,468 8 months ago Generic command line non-JVM Apache Kafka producer and consumer
kafka-jackson 7 almost 7 years ago Kafka Serializer, Deserializer, and Serde for Jackson JSON
kafkahq 3,432 2 months ago Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
secor 1,846 2 months ago Secor is a service implementing Kafka log persistence
reactive-kafka 1,417 2 months ago Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka
kafka-monitor 2,023 over 1 year ago
chaperone 641 about 4 years ago A Kafka audit system
uReplicator 918 about 1 year ago Improvement of Apache Kafka Mirrormaker
kafka-websocket 347 about 1 year ago
kafka-unit 174 over 3 years ago
cruise-control 2,777 3 months ago Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster
jocko 4,939 over 1 year ago Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hermes 820 2 months ago Fast and reliable message broker built on top of Kafka
kt 947 11 months ago Kafka command line tool
kafka-eagle 3,090 7 months ago Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information
dockerkafka 633 about 3 years ago DoctorKafka is a service for Kafka cluster auto healing and workload balancing
kasper 439 over 7 years ago Kasper is a lightweight library for processing Kafka topics
Yelp kafka-utils 314 over 1 year ago
kafka-spark-consumer 635 almost 3 years ago
kafka-streams-cep 308 about 1 year ago Complex Event Processing on top of Kafka Streams
spring-kafka 2,244 2 months ago
ksql-machine-learning-udf 11 almost 7 years ago
kafkabeat 6 over 7 years ago Elasticsearch Beats for kafka
sangrenel 204 almost 2 years ago
Strimzi 4,904 2 months ago Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
Kafka Manager 11,853 over 1 year ago Web-based tool for managing a Kafka cluster
Strimzi Kafka CLI 82 4 months ago A CLI for Strimzi Kafka Operator
Kafka Cluster Kraft Mode - DockerCompose 18 7 months ago Workable kafka cluster with kraft mode using docker-compose
Decaton 337 6 months ago High throughput asynchronous task processing framework

Awesome Kafka / Libraries / Kafka UI
KafkaTrail 6 9 months ago
kafka-topics-ui 876 about 3 years ago
kafka-connect-ui 508 about 1 year ago Web tool for Kafka Connect
Kafdrop 5,620 2 months ago Web UI for browsing Kafka topics and consumer groups
kafka-webview 394 2 months ago
kafka-view 219 over 2 years ago
cruise-control-ui 350 6 months ago
real-time-ui-with-kafka-streams 31 over 7 years ago
tsujun 53 over 6 years ago Yet another Web UI for KSQL
kowl 3,860 2 months ago
akhq

Awesome Kafka / Libraries / Kafka Streams
schema-registry-ui 421 about 1 year ago
kafka-streams-consumer-offsets-to-json 13 over 5 years ago A Kafka Streams process to convert consumer_offsets to a JSON-readable topic
kafka-operator 295 over 6 years ago
kafkabalancer 48 over 7 years ago
mockedstreams 187 about 4 years ago
winton-kafka-streams 311 about 6 years ago A Python implementation of Apache Kafka Streams

Awesome Kafka / Libraries / Kafka Connect
Apache Camel Kafka Connect 340+ Apache Camel components as Kafka Connect connectors
kafka-connect-client 40 4 months ago A kafka-connect REST api client for java
Confluent Connector Hub
kafka-connect-protobuf-converter 94 over 1 year ago Protobuf converter plugin for Kafka Connect
Maxwell 4,059 2 months ago
kafka-connect-transformers 8 over 6 years ago
kafka-connect-mq-source 95 2 months ago
kafka-connect-jenkins 30 over 2 years ago
toketi-kafka-connect-iohub 40 almost 5 years ago
connectctl 12 over 2 years ago Manage kafka connect connectors easily
kafka-connect-transform-common 156 7 months ago Common Transforms for Kafka Connect
aiven-kafka-connect-transforms 75 3 months ago A collection of Single Message Transformations (SMTs) for Kafka Connect
kafka-connect-file-pulse 324 3 months ago A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
kafka-connect-http 30 over 2 years ago
snowflake-kafka-connector 141 2 months ago

Awesome Kafka / Libraries / REST Proxy
kafka-rest 54 2 months ago Confluent REST Proxy
strimzi-http-bridge 293 2 months ago Strimzi Kafka Bridge (AMQP & HTTP)
Zilla 553 2 months ago An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol

Awesome Kafka / Libraries / Schema Registry
apicurio-registry 606 2 months ago Apicurio API/schema registry (includes UI)
schema-registry 2,233 2 months ago Confluent Schema registry for Kafka
ksql-jdbc-driver 87 almost 3 years ago

Awesome Kafka / Libraries / Other Awesome Kafka
ApacheKafka 35 about 1 year ago
dharmeshkakadia/awesome-kafka 206 over 1 year ago
infoslack/awesome-kafka 578 about 1 year ago
awesome-kafka-playground 1 over 6 years ago

Awesome Kafka / Libraries / Kafkaesque
Pulsar 14,315 2 months ago
Flink
Spark
Heron 3,638 almost 2 years ago
Beam
Arrow
Samza
Nats
Chronicle-Queue 3,350 2 months ago
CorfuDB 642 2 months ago
GenStage
faust 6,751 7 months ago Python Stream Processing
redpanda
Meteor 23 over 2 years ago Lightweight, broker-less alternative to Kafka for message streaming

Awesome Kafka / Resources / Learning
Passionate Developer: Kafka Streams DSL vs processor API
Mastering Kafka Streams
Spark Streaming + Kafka Integration Guide

Awesome Kafka / Resources / Blogs
CodingJunkie Random Thoughts on Coding by Bill Bejeck
Understanding Kafka with Factorio

Awesome Kafka / Resources / Books
Kafka Streams in Action
Kafka in Action
Kafka the Definitive Guide
Streaming Data Pipelines with Kafka

Backlinks from these awesome lists:

More related projects: