MLLM-Bench
Model evaluator
Evaluates and compares the performance of multimodal large language models on various tasks
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
56 stars
10 watching
3 forks
Language: Python
last commit: almost 2 years agoRelated projects:
| Repository | Description | Stars |
|---|---|---|
 evolvinglmms-lab/lmms-eval | Tools and evaluation framework for accelerating the development of large multimodal models by providing an efficient way to assess their performance | 2,164 |
 chenllliang/mmevalpro | A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline. | 22 |
 mshukor/evalign-icl | Evaluating and improving large multimodal models through in-context learning | 21 |
 mlgroupjlu/llm-eval-survey | A repository of papers and resources for evaluating large language models. | 1,450 |
 ailab-cvc/seed-bench | A benchmark for evaluating large language models' ability to process multimodal input | 322 |
 mlabonne/llm-autoeval | A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters. | 566 |
 open-compass/vlmevalkit | An evaluation toolkit for large vision-language models | 1,514 |
 junyangwang0410/amber | An LLM-free benchmark suite for evaluating MLLMs' hallucination capabilities in various tasks and dimensions | 98 |
 felixgithub2017/mmcu | Measures the understanding of massive multitask Chinese datasets using large language models | 87 |
 yuweihao/mm-vet | Evaluates the capabilities of large multimodal models using a set of diverse tasks and metrics | 274 |
 aifeg/benchlmm | An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models | 84 |
 maluuba/nlg-eval | A toolset for evaluating and comparing natural language generation models | 1,350 |
 benhamner/metrics | Provides implementations of various supervised machine learning evaluation metrics in multiple programming languages. | 1,632 |
 declare-lab/instruct-eval | An evaluation framework for large language models trained with instruction tuning methods | 535 |
 i-gallegos/fair-llm-benchmark | Compiles bias evaluation datasets and provides access to original data sources for large language models | 115 |