kafka-spark-consumer
Kafka Consumer Tool
A tool for consuming messages from Kafka topics using Spark Streaming while maintaining reliability and handling failures
High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.
635 stars
70 watching
318 forks
Language: Java
last commit: over 4 years agoLinked from 2 awesome lists
semantalytics/awesome-kafka
dharmeshkakadia/awesome-kafkaRelated projects:
| Repository | Description | Stars |
|---|---|---|
 ibmstreams/streamsx.kafka | A toolkit for integrating Apache Kafka with Stream Processor SPL applications | 13 |
 streamthoughts/kafka-connect-file-pulse | A tool that enables streaming and processing of various file formats into Apache Kafka | 324 |
 yannael/kafka-sparkstreaming-cassandra | An environment for experimenting with real-time data processing using Kafka, Spark streaming, and Cassandra | 97 |
 instaclustr/sample-kafkasparkcassandra | An introductory Scala app using Apache Spark Streaming to process data from Kafka and write summaries to Cassandra. | 23 |
 sderosiaux/kafka-streams-consumer-offsets-to-json | Converts Kafka __consumer_offsets to JSON | 13 |
 b/kafka-websocket | Provides a WebSocket interface to interact with Kafka's distributed message broker | 347 |
| streamthoughts/azkarra-streams | A lightweight Java framework to develop and manage cloud-native streaming microservices based on Apache Kafka Streams. | 185 |
 snowflakedb/snowflake-kafka-connector | A plugin that enables data ingestion from Kafka topics to Snowflake tables | 141 |
 fgeller/kt | A command-line tool for interacting with Apache Kafka clusters | 947 |
 turbolytics/sql-flow | An open-source tool that enables writing Kafka stream processing logic using standard SQL syntax. | 73 |
 raystack/firehose | Delivers real-time streaming data to various destinations | 329 |
 dart-kafka/kafka | Provides a Dart client library for interacting with the Apache Kafka message broker | 56 |
 mre/kafka-influxdb | A Kafka consumer for InfluxDB that buffers metric data during high loads and enables reliable sending of metrics from remote locations. | 216 |
 nodefluent/kafka-streams | A Node.js library implementing Kafka Streams functionality for stream state processing and table representation. | 831 |
 perfectlysoft/perfect-kafka | An Express Swift wrapper of librdkafka to provide a unified interface for Apache Kafka client operations | 24 |