llmperf
LLM benchmarking tool
A tool for evaluating the performance of large language model APIs
LLMPerf is a library for validating and benchmarking LLMs
678 stars
9 watching
115 forks
Language: Python
last commit: about 1 month ago
Linked from 1 awesome list
ethicalml/awesome-production-machine-learningRelated projects:
| Repository | Description | Stars |
|---|---|---|
 qcri/llmebench | A benchmarking framework for large language models | 81 |
 damo-nlp-sg/m3exam | A benchmark for evaluating large language models in multiple languages and formats | 93 |
 wgryc/phasellm | A framework for managing and testing large language models to evaluate their performance and optimize user experiences. | 451 |
 ajndkr/lanarky | A Python web framework specifically designed to build LLM microservices with built-in support for FastAPI and streaming capabilities. | 978 |
 relari-ai/continuous-eval | Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics | 455 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,557 |
 luogen1996/lavin | An open-source implementation of a vision-language instructed large language model | 513 |
 r2d4/openlm | Library that provides a unified API to interact with various Large Language Models (LLMs) | 367 |
 mlcommons/inference | Measures the performance of deep learning models in various deployment scenarios. | 1,256 |
| ray-project/ray | A unified framework for scaling AI and Python applications by providing a distributed runtime and a set of libraries for machine learning and other compute tasks. | 34,412 |
 aifeg/benchlmm | An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models | 84 |
 dreadnode/rigging | A framework for leveraging language models in production code | 216 |
 multimodal-art-projection/omnibench | Evaluates and benchmarks multimodal language models' ability to process visual, acoustic, and textual inputs simultaneously. | 15 |
 aiplanethub/beyondllm | An open-source toolkit for building and evaluating large language models | 267 |
 internlm/lagent | A lightweight framework for building agent-based applications using LLMs and transformer architectures | 1,924 |