LLM-Evals-Catalogue
This repository stems from our paper, “Cataloguing LLM Evaluations”, and serves as a living, collaborative catalogue of LLM evaluation frameworks, benchmarks and papers.
AI summary
LLM evaluator collection
A collaborative catalogue of LLM evaluation frameworks and papers
- stars
- 13
- forks
- 2
- watching
- 1
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
LLM evaluation resource
A repository of papers and resources for evaluating large language models.
LLM evaluation framework
Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics
LLM evaluator
An evaluation framework for large language models with Elo rating system and A/B testing capabilities
LLM evaluator
An all-in-one toolkit for evaluating Large Language Models (LLMs) across multiple backends.
Model evaluation toolkit
Tools and evaluation framework for accelerating the development of large multimodal models by providing an efficient way to assess their performance
Model evaluator
A framework for efficiently evaluating and benchmarking large models
Model evaluation framework
A framework for evaluating OpenAI models and an open-source registry of benchmarks.
Model evaluator
Evaluates and compares the performance of multimodal large language models on various tasks
Visual Model Benchmark
An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models
Model evaluator
A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters.
Evaluation pipeline
A framework for evaluating language models on NLP tasks
Model evaluator
An evaluation framework for large language models trained with instruction tuning methods
Model Evaluator
A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline.
LLM evaluator
An evaluation suite and dynamic data release platform for large language models
LLM trainer
A flexible RL training framework designed for large language models
Featured in 1 awesome list
Each link jumps to the spot where the list mentions LLM-Evals-Catalogue.
