mario
Data pipeline library
A library that enables the definition of complex data pipelines in a functional, typesafe, and efficient way using a declarative syntax
Functional, Typesafe, Declarative Data Pipelines
139 stars
90 watching
16 forks
Language: Scala
last commit: over 8 years agoLinked from 1 awesome list
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 joboccara/pipes | A header-only C++14 library for building expressive data pipelines using a chainable interface. | 808 |
 darky/rocket-pipes | A TypeScript library that enables the creation of modular, composable, and reusable data processing pipelines | 25 |
 giacbrd/smartpipeline | A framework for designing and executing concurrent data pipelines with a focus on simplicity and efficiency | 25 |
 galaxyproject/galaxy | A platform for data-intensive scientific analysis and workflow management | 1,431 |
 kevin-hanselman/dud | A lightweight tool for managing and versioning large data alongside source code in data pipelines | 184 |
 optics-dev/monocle | A Scala library providing a functional programming style interface to manipulate and compose data structures using lenses and other combinatorial techniques. | 1,663 |
 nessos/streams | A lightweight library for building efficient data pipelines using functional programming concepts | 383 |
 log2timeline/dftimewolf | A framework for orchestrating data collection, processing, and export | 299 |
 scalalandio/chimney | A Scala library for automating data transformations without boilerplate code. | 1,181 |
 whitaker-io/machine | A library for creating data workflows that can be simple or complex, with features like recursion and memoization. | 159 |
 renkun-ken/piper | Provides functions and methods to chain operations in R, enhancing readability and maintainability of data pipelines. | 169 |
 huggingface/datatrove | A platform-agnostic data processing framework for large-scale text data pipelines | 2,103 |
 silascutler/malpipe | An ingestion and processing framework for malware and indicator data from various feeds. | 104 |
 databiosphere/toil | A workflow management system designed to efficiently run pipelines in various environments. | 901 |
 druths/xp | A tool for creating flexible and self-documenting data science pipelines | 56 |