dask
Parallelizer
A parallel computing library for analytics and scientific computing.
Parallel computing with task scheduling
13k stars
212 watching
2k forks
Language: Python
last commit: 8 days ago
Linked from 7 awesome lists
dasknumpypandaspydatapythonscikit-learnscipy
vinta/awesome-python
jobbole/awesome-python-cn
ethicalml/awesome-production-machine-learning
pditommaso/awesome-pipeline
pawl/awesome-etl
ly0n/awesome-robotic-tooling
krzjoa/awesome-python-data-scienceRelated projects:
| Repository | Description | Stars |
|---|---|---|
| dask/distributed | A library for managing and orchestrating parallel computing tasks across multiple machines | 1,579 |
 utdemir/distributed-dataset | A Haskell-based framework for processing and distributing large datasets across multiple nodes in parallel. | 116 |
 dougbinks/enkits | An API and scheduling library for parallel programming using multicore CPUs | 1,753 |
 harisekhon/devops-python-tools | A collection of 80+ CLI tools for DevOps, Cloud, Big Data, and Python development | 773 |
 wakatime/wakaq | A minimal background task queue system with Redis and Celery integration | 574 |
 joblib/joblib-spark | Enables parallelization of machine learning tasks on a distributed Spark cluster using the joblib library. | 242 |
 p-ranav/task_system | A task scheduling system built with C++14 primitives to manage concurrent tasks and queues. | 40 |
 spdk/spdk | A toolset for building high-performance, scalable storage applications without kernel involvement. | 3,093 |
 seantanly/elixir-paratize | An Elixir library providing parallel processing facilities with customizable worker size and timeout options. | 28 |
 dpdk/dpdk | A set of libraries and drivers for fast packet processing on various architectures. | 3,387 |
 dano/aioprocessing | A Python library that integrates asyncio with multiprocessing for concurrent task execution | 655 |
| harisekhon/devops-perl-tools | An extensive collection of DevOps and Big Data CLI tools written in Perl. | 91 |
 daskos/mentor | An extensible framework for building Python applications on Apache Mesos clusters | 33 |
 danengelbrecht/bikeshed | A high-performance, lock-free task scheduler for managing hierarchical tasks with dependencies. | 108 |
 line/decaton | A framework for high-throughput, concurrent task processing on Apache Kafka | 335 |