Sparkling
Internet Archive's Sparkling Data Processing Library
AI summary
Web data processor
A data processing library built on top of Apache Spark to handle temporal web data
- stars
- 11
- forks
- 2
- watching
- 20
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
apache/spark40.2K
Data processor
An analytics engine designed to handle large-scale data processing and analysis
Web Crawler
A high-performance web crawler built on Apache Spark that fetches and analyzes web resources in real-time.
Archive processor
A distributed compute analysis system for web archive collections
Data processor
A framework for efficient data processing and extraction from archival collections, enabling the transformation of raw data into more accessible formats.
Spark wrapper
A Clojure API for interacting with Apache Spark
Web app manager
A Sparkle update project for web application management and automation.
CSV parser
A library for parsing and querying CSV data with Apache Spark
Dataset processor
A lightweight Python implementation of Spark's RDD and DStream interfaces for improved performance on small datasets
ML integration
Integrates H2O's machine learning capabilities with Apache Spark for big data processing and analytics
Data analyzer
Enables distributed data analysis using PySpark and Pandas APIs
Spark Data Processor
An introductory Scala app using Apache Spark Streaming to process data from Kafka and write summaries to Cassandra.
Analytics engine
A tool for creating resilient, scalable analytics applications with Haskell on top of Apache Spark
Text processor
Provides tools and data to convert text into tidy data formats for natural language processing tasks
Spark interface
An R interface to Apache Spark for distributed data analysis and machine learning
Sparql processor
Provides a robust, incremental processing of streaming results from SPARQL servers.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions Sparkling.
