helm
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).
AI summary
ModelEvaluator
A framework to evaluate and compare language models by analyzing their performance on various tasks
- stars
- 2K
- forks
- 260
- watching
- 36
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Model Evaluator
An evaluation framework for machine learning models and datasets, providing standardized metrics and tools for comparing model performance.
Model Evaluator
A benchmarking framework for evaluating Large Multimodal Models by providing rigorous metrics and an efficient evaluation pipeline.
Language model trainer
A framework for training large language models that prioritizes legibility, scalability, and reproducibility
Model evaluation framework
An evaluation toolkit and platform for assessing large models in various domains
Model evaluator
A framework for efficiently evaluating and benchmarking large models
Model evaluator
An evaluation framework for large language models trained with instruction tuning methods
Evaluation pipeline
A framework for evaluating language models on NLP tasks
Multimodal model evaluator
Evaluating and improving large multimodal models through in-context learning
Model Evaluator
Evaluates language models using standardized benchmarks and prompting techniques.
Model evaluator
A community-developed tool for evaluating climate models and providing diagnostic metrics.
Model evaluator
Evaluates the capabilities of large multimodal models using a set of diverse tasks and metrics
Model explainer
A tool to help understand and explain the behavior of complex machine learning models
ConvCRF model
An implementation of a convolutional Conditional Random Field model for semantic segmentation tasks.
LLM evaluator
An all-in-one toolkit for evaluating Large Language Models (LLMs) across multiple backends.
Model evaluator
A tool for evaluating and visualizing machine learning model performance
Featured in 1 awesome list
Each link jumps to the spot where the list mentions helm.
