llmperf
LLM benchmarking tool
A tool for evaluating the performance of large language model APIs
LLMPerf is a library for validating and benchmarking LLMs
641 stars
9 watching
107 forks
Language: Python
last commit: 3 months ago
Linked from 1 awesome list
ethicalml/awesome-production-machine-learningRelated projects:
| Repository | Description | Stars |
|---|---|---|
 qcri/llmebench | A benchmarking framework for large language models | 80 |
 damo-nlp-sg/m3exam | A benchmark for evaluating large language models in multiple languages and formats | 92 |
 wgryc/phasellm | A framework for managing and testing large language models to evaluate their performance and optimize user experiences. | 448 |
 ajndkr/lanarky | A Python web framework specifically designed to build LLM microservices with built-in support for FastAPI and streaming capabilities. | 976 |
 relari-ai/continuous-eval | Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics | 446 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,529 |
 luogen1996/lavin | An open-source implementation of a vision-language instructed large language model | 508 |
 r2d4/openlm | Library that provides a unified API to interact with various Large Language Models (LLMs) | 366 |
 mlcommons/inference | Measures the performance of deep learning models in various deployment scenarios. | 1,236 |
| ray-project/ray | A unified framework for scaling AI and Python applications by providing a distributed runtime and a set of libraries for machine learning and other compute tasks. | 33,994 |
 aifeg/benchlmm | An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models | 83 |
 dreadnode/rigging | An LLM framework that simplifies interacting with language models in production code | 209 |
 multimodal-art-projection/omnibench | Evaluates and benchmarks multimodal language models' ability to process visual, acoustic, and textual inputs simultaneously. | 14 |
 aiplanethub/beyondllm | An open-source toolkit for building and evaluating large language models | 261 |
 internlm/lagent | A lightweight framework for building agent-based applications using LLMs and transformer architectures | 1,865 |