lm-evaluation-harness
by EleutherAI
A framework for few-shot evaluation of language models.
AI summary
Evaluation framework
Provides a unified framework to test generative language models on various evaluation tasks.
- stars
- 7.2K
- forks
- 1.9K
- watching
- 38
- awesome lists
- 4
Similar projects
Found by comparing what the projects do, not just their names.
LLM toolkit
An open-source toolkit for pretraining and fine-tuning large language models
LLM evaluation toolkit
A toolkit for evaluating and optimizing Large Language Model applications with objective metrics, test data generation, and seamless integrations.
Model evaluator
A unified framework for evaluating large language models' performance and robustness in various scenarios.
microsoft/lmops3.7K
LLM booster
A research initiative focused on developing fundamental technology to improve the performance and efficiency of large language models.
LLM toolkit
A toolkit for fine-tuning and inferring large machine learning models
AI toolkit
An all-encompassing tool providing a web interface to access and utilize various AI models for tasks such as text generation, image analysis, music generation, and more.
LLM evaluator
A framework for evaluating large language models
LLM toolkit
An integrated development platform for large language models (LLMs) that provides observability, analytics, and management tools.
haotian-liu/llava20.7K
Visual Instruction System
A system that uses large language and vision models to generate and process visual instructions
Prompt engineering guide
Guides software developers on how to effectively use and build systems around Large Language Models like GPT-4.
Prompt testing env
An environment for battle-testing prompts to Large Language Models (LLMs) to evaluate response quality and performance.
LLM guide
A curated list of resources to help developers navigate the landscape of large language models and their applications in NLP
Tool learning platform
A platform for training, serving, and evaluating large language models to enable tool use capability
openai/evals15.2K
Benchmarking framework
A framework for evaluating large language models and systems, providing a registry of benchmarks.
LLM framework
A framework for training and serving large language models using JAX/Flax
Featured in 4 awesome lists
Each link jumps to the spot where the list mentions lm-evaluation-harness.
