MixEval

huggingface/lighteval879

LLM evaluator

An all-in-one toolkit for evaluating Large Language Models (LLMs) across multiple backends.

mlgroupjlu/llm-eval-survey1.5K

LLM evaluation resource

A repository of papers and resources for evaluating large language models.

relari-ai/continuous-eval455

LLM evaluation framework

Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics

wgryc/phasellm451

LLM framework

A framework for managing and testing large language models to evaluate their performance and optimize user experiences.

prometheus-eval/prometheus-eval820

Evaluation tool

An open-source framework that enables language model evaluation using Prometheus and GPT4

evolvinglmms-lab/lmms-eval2.2K

Model evaluation toolkit

Tools and evaluation framework for accelerating the development of large multimodal models by providing an efficient way to assess their performance

freedomintelligence/mllm-bench56

Model evaluator

Evaluates and compares the performance of multimodal large language models on various tasks

ailab-cvc/seed-bench322

Multimodal LLM test suite

A benchmark for evaluating large language models' ability to process multimodal input

mlabonne/llm-autoeval566

Model evaluator

A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters.

qcri/llmebench81

LLM benchmarker

A benchmarking framework for large language models

chenllliang/mmevalpro22

Model Evaluator

A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline.

flageval-baai/flageval307

Model evaluation framework

An evaluation toolkit and platform for assessing large models in various domains

victordibia/llmx79

LLM API

An API that provides a unified interface to multiple large language models for chat fine-tuning

h2oai/h2o-llm-eval50

LLM evaluator

An evaluation framework for large language models with Elo rating system and A/B testing capabilities

hkust-nlp/ceval1.7K

Evaluation suite

An evaluation suite providing multiple-choice questions for foundation models in various disciplines, with tools for assessing model performance.