evals
by openai
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
AI summary
Benchmarking framework
A framework for evaluating large language models and systems, providing a registry of benchmarks.
- stars
- 15.2K
- forks
- 2.6K
- watching
- 264
- awesome lists
- 4
Similar projects
Found by comparing what the projects do, not just their names.
Model evaluation framework
A framework for evaluating OpenAI models and an open-source registry of benchmarks.
Model Evaluator
Evaluates language models using standardized benchmarks and prompting techniques.
LLM evaluation toolkit
A toolkit for evaluating and optimizing Large Language Model applications with objective metrics, test data generation, and seamless integrations.
Evaluation framework
Provides a unified framework to test generative language models on various evaluation tasks.
LLM evaluator
A framework for evaluating large language models
Prompt testing env
An environment for battle-testing prompts to Large Language Models (LLMs) to evaluate response quality and performance.
Evaluation pipeline
A framework for evaluating language models on NLP tasks
Evaluation tool
An open-source framework that enables language model evaluation using Prometheus and GPT4
RL frameworks
High-quality implementations of reinforcement learning algorithms for research and development purposes
Model monitor
An observability framework for evaluating and monitoring the performance of machine learning models and data pipelines
LLM evaluation framework
Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics
Prompt tester
A set of tools for testing and evaluating natural language processing models and vector databases.
Evaluation framework
An evaluation toolkit for large vision-language models
LLM evaluation resource
A repository of papers and resources for evaluating large language models.
AI auditor
Automates the detection of performance, bias, and security issues in AI applications
Featured in 4 awesome lists
Each link jumps to the spot where the list mentions evals.
