joblib-spark
by joblib
Joblib Apache Spark Backend
AI summary
Task parallelizer
Enables parallelization of machine learning tasks on a distributed Spark cluster using the joblib library.
- stars
- 243
- forks
- 26
- watching
- 9
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Worker pool manager
Simplifies creating multiple worker threads to execute tasks in parallel
Deep learning framework
A PyTorch implementation on Apache Spark for distributed deep learning model training and inference.
Task programming library
A library providing a simple and expressive way to write parallel programs with complex task dependencies.
EMR automation tool
Automates deployment of an AWS EMR cluster and execution of Spark jobs
Neural net framework
Distributed neural network framework for Apache Spark
apache/spark40.2K
Data processor
An analytics engine designed to handle large-scale data processing and analysis
Clustering algorithms library
Implementations of clustering algorithms using Spark in Scala
Machine Learning Library
A Python library that integrates PySpark and scikit-learn for distributed machine learning
Spark job server demo
Demonstrates using Spark Jobserver to run Apache Spark analytics with Cassandra
DBMS library
A library that brings useful functions from various modern database management systems to Apache Spark
Process parallelizer
An Elixir module that executes multiple processes in parallel to speed up slow computations
Spark wrapper
Provides a reusable set of Nextflow subworkflows and processes for creating transient Apache Spark clusters on any infrastructure.
Dataset processor
A lightweight Python implementation of Spark's RDD and DStream interfaces for improved performance on small datasets
Spark wrapper
Provides a C# API for interacting with Apache Spark
Spark API
Provides compatibility and extensions between Kotlin and Apache Spark for big data processing
Featured in 1 awesome list
Each link jumps to the spot where the list mentions joblib-spark.
