petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

GitHub
2k stars
40 watching
285 forks
Language: Python
last commit: 10 months ago
Linked from 1 awesome list

deep-learningmachine-learningparquetparquet-filespyarrowpysparkpytorchsysmltensorflow

Backlinks from these awesome lists: