awesome-kafka

Kafka toolkit

A curated collection of useful libraries and tools for working with Apache Kafka

A curated list of awesome things related to Apache Kafka

GitHub
244 stars
19 watching
42 forks
last commit: 15 days ago
Linked from 1 awesome list

awesomeawesome-listkafka

Awesome Kafka / Libraries / Kafka
topicctl 598 15 days ago A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more
sarama 11,560 10 days ago Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later)
kadeck Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
conduktor
mirus 203 23 days ago Mirus is a cross data-center data replication tool for Apache Kafka
kareldb 393 17 days ago A Relational Database Backed by Apache Kafka
kcache 245 8 days ago An In-Memory Cache Backed by Apache Kafka
kafka-lag-based-assignor 12 about 4 years ago Kafka partition assignor that distributes lag evenly across a consumer group
kafka-pixy 774 7 months ago Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control
Burrow 3,762 4 months ago Kafka Consumer Lag Checking
kafka-tools 595 almost 3 years ago A collection of tools for working with Apache Kafka
schema-registry-ui 421 9 months ago Web tool for Avro Schema Registry
kafkacat 5,447 5 months ago Generic command line non-JVM Apache Kafka producer and consumer
kafka-jackson 7 over 6 years ago Kafka Serializer, Deserializer, and Serde for Jackson JSON
kafkahq 3,411 10 days ago Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
secor 1,846 about 1 month ago Secor is a service implementing Kafka log persistence
reactive-kafka 1,417 3 days ago Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka
kafka-monitor 2,023 about 1 year ago
chaperone 641 almost 4 years ago A Kafka audit system
uReplicator 917 11 months ago Improvement of Apache Kafka Mirrormaker
kafka-websocket 347 11 months ago
kafka-unit 174 about 3 years ago
cruise-control 2,765 9 days ago Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster
jocko 4,935 about 1 year ago Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hermes 819 7 days ago Fast and reliable message broker built on top of Kafka
kt 948 8 months ago Kafka command line tool
kafka-eagle 3,084 4 months ago Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information
dockerkafka 634 almost 3 years ago DoctorKafka is a service for Kafka cluster auto healing and workload balancing
kasper 439 about 7 years ago Kasper is a lightweight library for processing Kafka topics
Yelp kafka-utils 314 about 1 year ago
kafka-spark-consumer 635 over 2 years ago
kafka-streams-cep 308 11 months ago Complex Event Processing on top of Kafka Streams
spring-kafka 2,192 5 days ago
ksql-machine-learning-udf 11 over 6 years ago
kafkabeat 6 about 7 years ago Elasticsearch Beats for kafka
sangrenel 205 over 1 year ago
Strimzi 4,858 5 days ago Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
Kafka Manager 11,836 over 1 year ago Web-based tool for managing a Kafka cluster
Strimzi Kafka CLI 80 15 days ago A CLI for Strimzi Kafka Operator
Kafka Cluster Kraft Mode - DockerCompose 17 4 months ago Workable kafka cluster with kraft mode using docker-compose
Decaton 335 3 months ago High throughput asynchronous task processing framework

Awesome Kafka / Libraries / Kafka UI
KafkaTrail 4 6 months ago
kafka-topics-ui 877 almost 3 years ago
kafka-connect-ui 506 11 months ago Web tool for Kafka Connect
Kafdrop 5,582 6 days ago Web UI for browsing Kafka topics and consumer groups
kafka-webview 395 about 2 months ago
kafka-view 219 about 2 years ago
cruise-control-ui 346 3 months ago
real-time-ui-with-kafka-streams 31 about 7 years ago
tsujun 53 over 6 years ago Yet another Web UI for KSQL
kowl 3,829 6 days ago
akhq

Awesome Kafka / Libraries / Kafka Streams
schema-registry-ui 421 9 months ago
kafka-streams-consumer-offsets-to-json 13 about 5 years ago A Kafka Streams process to convert consumer_offsets to a JSON-readable topic
kafka-operator 297 about 6 years ago
kafkabalancer 48 about 7 years ago
mockedstreams 187 almost 4 years ago
winton-kafka-streams 313 almost 6 years ago A Python implementation of Apache Kafka Streams

Awesome Kafka / Libraries / Kafka Connect
Apache Camel Kafka Connect 340+ Apache Camel components as Kafka Connect connectors
kafka-connect-client 37 about 1 month ago A kafka-connect REST api client for java
Confluent Connector Hub
kafka-connect-protobuf-converter 94 over 1 year ago Protobuf converter plugin for Kafka Connect
Maxwell 4,047 about 1 month ago
kafka-connect-transformers 8 about 6 years ago
kafka-connect-mq-source 95 about 2 months ago
kafka-connect-jenkins 30 about 2 years ago
toketi-kafka-connect-iohub 40 over 4 years ago
connectctl 12 over 2 years ago Manage kafka connect connectors easily
kafka-connect-transform-common 154 4 months ago Common Transforms for Kafka Connect
aiven-kafka-connect-transforms 74 16 days ago A collection of Single Message Transformations (SMTs) for Kafka Connect
kafka-connect-file-pulse 324 7 days ago A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
kafka-connect-http 30 over 2 years ago
snowflake-kafka-connector 139 3 days ago

Awesome Kafka / Libraries / REST Proxy
kafka-rest 46 6 days ago Confluent REST Proxy
strimzi-http-bridge 286 6 days ago Strimzi Kafka Bridge (AMQP & HTTP)
Zilla 543 8 days ago An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol

Awesome Kafka / Libraries / Schema Registry
apicurio-registry 609 6 days ago Apicurio API/schema registry (includes UI)
schema-registry 2,225 6 days ago Confluent Schema registry for Kafka
ksql-jdbc-driver 88 over 2 years ago

Awesome Kafka / Libraries / Other Awesome Kafka
ApacheKafka 34 12 months ago
dharmeshkakadia/awesome-kafka 204 over 1 year ago
infoslack/awesome-kafka 579 10 months ago
awesome-kafka-playground 1 over 6 years ago

Awesome Kafka / Libraries / Kafkaesque
Pulsar 14,252 7 days ago
Flink
Spark
Heron 3,643 over 1 year ago
Beam
Arrow
Samza
Nats
Chronicle-Queue 3,307 6 days ago
CorfuDB 641 17 days ago
GenStage
faust 6,746 4 months ago Python Stream Processing
redpanda
Meteor 23 over 2 years ago Lightweight, broker-less alternative to Kafka for message streaming

Awesome Kafka / Resources / Learning
Passionate Developer: Kafka Streams DSL vs processor API
Mastering Kafka Streams
Spark Streaming + Kafka Integration Guide

Awesome Kafka / Resources / Blogs
CodingJunkie Random Thoughts on Coding by Bill Bejeck
Understanding Kafka with Factorio

Awesome Kafka / Resources / Books
Kafka Streams in Action
Kafka in Action
Kafka the Definitive Guide
Streaming Data Pipelines with Kafka

Backlinks from these awesome lists:

More related projects: