llm-autoeval
Model evaluator
A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters.
Automatically evaluate your LLMs in Google Colab
558 stars
7 watching
91 forks
Language: Python
last commit: 7 months ago
Linked from 1 awesome list
ethicalml/awesome-production-machine-learningRelated projects:
| Repository | Description | Stars |
|---|---|---|
 allenai/olmo-eval | An evaluation framework for large language models. | 310 |
 chenllliang/mmevalpro | A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline. | 22 |
 evolvinglmms-lab/lmms-eval | Tools and evaluation suite for large multimodal models | 2,058 |
 huggingface/lighteval | A toolkit for evaluating Large Language Models across multiple backends | 804 |
 maluuba/nlg-eval | A toolset for evaluating and comparing natural language generation models | 1,347 |
 mlgroupjlu/llm-eval-survey | A repository of papers and resources for evaluating large language models. | 1,433 |
 freedomintelligence/mllm-bench | Evaluates and compares the performance of multimodal large language models on various tasks | 55 |
 declare-lab/instruct-eval | An evaluation framework for large language models trained with instruction tuning methods | 528 |
 open-compass/lawbench | Evaluates the legal knowledge of large language models using a custom benchmarking framework. | 267 |
 edublancas/sklearn-evaluation | A tool for evaluating and visualizing machine learning model performance | 3 |
 h2oai/h2o-llm-eval | An evaluation framework for large language models with Elo rating system and A/B testing capabilities | 50 |
| huggingface/evaluate | An evaluation framework for machine learning models and datasets, providing standardized metrics and tools for comparing model performance. | 2,034 |
 langchain-ai/auto-evaluator | Automated evaluation of language models for question answering tasks | 744 |
 rlancemartin/auto-evaluator | An evaluation tool for question-answering systems using large language models and natural language processing techniques | 1,063 |
 xverse-ai/xverse-7b | A multilingual large language model developed by XVERSE Technology Inc. | 50 |