dask
Parallelizer
A parallel computing library for analytics and scientific computing.
Parallel computing with task scheduling
13k stars
213 watching
2k forks
Language: Python
last commit: about 1 month ago
Linked from 7 awesome lists
dasknumpypandaspydatapythonscikit-learnscipy
vinta/awesome-python
jobbole/awesome-python-cn
ethicalml/awesome-production-machine-learning
pditommaso/awesome-pipeline
ly0n/awesome-robotic-tooling
pawl/awesome-etl
krzjoa/awesome-python-data-scienceRelated projects:
| Repository | Description | Stars |
|---|---|---|
| dask/distributed | A library for managing and orchestrating parallel computing tasks across multiple machines | 1,582 |
 utdemir/distributed-dataset | A Haskell-based framework for processing and distributing large datasets across multiple nodes in parallel. | 116 |
 dougbinks/enkits | An API and scheduling library for parallel programming using multicore CPUs | 1,763 |
 harisekhon/devops-python-tools | Tools for managing and automating DevOps tasks, data processing, and cloud infrastructure using Python. | 783 |
 wakatime/wakaq | A background task queue for Python applications backed by Redis and Celery. | 576 |
 joblib/joblib-spark | Enables parallelization of machine learning tasks on a distributed Spark cluster using the joblib library. | 243 |
 p-ranav/task_system | A task scheduling system built with C++14 primitives to manage concurrent tasks and queues. | 41 |
 spdk/spdk | A toolset for building high-performance, scalable storage applications without kernel involvement. | 3,129 |
 seantanly/elixir-paratize | An Elixir library providing parallel processing facilities with customizable worker size and timeout options. | 28 |
 dpdk/dpdk | A set of libraries and drivers for fast packet processing in multiple processor architectures. | 3,439 |
 dano/aioprocessing | A Python library that integrates asyncio with multiprocessing for concurrent task execution | 653 |
| harisekhon/devops-perl-tools | An extensive collection of DevOps and Big Data CLI tools written in Perl. | 93 |
 daskos/mentor | An extensible framework for building Python applications on Apache Mesos clusters | 33 |
 danengelbrecht/bikeshed | A high-performance, lock-free task scheduler for managing hierarchical tasks with dependencies. | 111 |
 line/decaton | A framework for high-throughput, concurrent task processing on Apache Kafka | 337 |