MLLM-Bench

Model evaluator

Evaluates and compares the performance of multimodal large language models on various tasks

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

GitHub

56 stars
10 watching
3 forks
Language: Python
last commit: 3 months ago

Related projects:

Repository Description Stars
evolvinglmms-lab/lmms-eval Tools and evaluation framework for accelerating the development of large multimodal models by providing an efficient way to assess their performance 2,164
chenllliang/mmevalpro A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline. 22
mshukor/evalign-icl Evaluating and improving large multimodal models through in-context learning 21
mlgroupjlu/llm-eval-survey A repository of papers and resources for evaluating large language models. 1,450
ailab-cvc/seed-bench A benchmark for evaluating large language models' ability to process multimodal input 322
mlabonne/llm-autoeval A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters. 566
open-compass/vlmevalkit An evaluation toolkit for large vision-language models 1,514
junyangwang0410/amber An LLM-free benchmark suite for evaluating MLLMs' hallucination capabilities in various tasks and dimensions 98
felixgithub2017/mmcu Measures the understanding of massive multitask Chinese datasets using large language models 87
yuweihao/mm-vet Evaluates the capabilities of large multimodal models using a set of diverse tasks and metrics 274
aifeg/benchlmm An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models 84
maluuba/nlg-eval A toolset for evaluating and comparing natural language generation models 1,350
benhamner/metrics Provides implementations of various supervised machine learning evaluation metrics in multiple programming languages. 1,632
declare-lab/instruct-eval An evaluation framework for large language models trained with instruction tuning methods 535
i-gallegos/fair-llm-benchmark Compiles bias evaluation datasets and provides access to original data sources for large language models 115