optimum-benchmark
Performance comparer
A tool for comparing and optimizing the performance of various machine learning frameworks and models on different hardware platforms.
🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
251 stars
5 watching
48 forks
Language: Python
last commit: 19 days ago
Linked from 1 awesome list
benchmarkneural-compressoronnxruntimeopenvinopytorchtensorrt-llmtext-generation-inference
ethicalml/awesome-production-machine-learningRelated projects:
| Repository | Description | Stars |
|---|---|---|
| huggingface/optimum-quanto | A PyTorch quantization backend for models. | 822 |
 mazhar-ansari-ardeh/benchmarkfcns | Provides benchmarking functions for mathematical optimization algorithms | 66 |
| huggingface/evaluate | An evaluation framework for machine learning models and datasets, providing standardized metrics and tools for comparing model performance. | 2,034 |
 benchflow/benchflow | Automated performance testing and analysis tool for distributed systems | 25 |
 performanceestimation/performance-estimation-toolbox | An automated toolbox for performing worst-case analyses of first-order optimization methods in convex and nonconvex optimization. | 52 |
 automl/hpobench | A collection of benchmark problems for hyperparameter optimization | 139 |
 lucfra/far-ho | A package for optimizing hyperparameters and meta-learning using gradient-based methods in TensorFlow. | 187 |
 catboost/benchmarks | Comparative benchmarks of various machine learning algorithms | 169 |
 guopengf/auto-fedrl | A reinforcement learning-based framework for optimizing hyperparameters in distributed machine learning environments. | 15 |
 aifeg/benchlmm | An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models | 82 |
 szilard/benchm-ml | A benchmark for evaluating machine learning algorithms' performance on large datasets | 1,869 |
 petabridge/nbench | A performance benchmarking and testing framework for .NET applications. | 533 |
 python/pyperformance | An authoritative source of real-world benchmarks for Python implementations. | 869 |
 datacanvasio/hypergbm | An AutoML toolkit designed to automate the entire machine learning process pipeline for tabular data | 337 |
 ivankoster/nimbench | A tool for comparing the performance of different programming languages or code snippets in a controlled and measurable way | 18 |