bigcode-evaluation-harness
A framework for the evaluation of autoregressive code generation language models.
AI summary
Code evaluation framework
A framework for evaluating autoregressive code generation language models in terms of their accuracy and robustness.
- stars
- 846
- forks
- 225
- watching
- 12
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Code generator
Trains models to generate code in multiple programming languages
Model evaluator
A framework for efficiently evaluating and benchmarking large models
Model evaluation framework
An evaluation toolkit and platform for assessing large models in various domains
Code environment framework
An interactive code environment framework for evaluating language agents through execution feedback.
Code generator
An AI model designed to generate and execute code automatically
DevOps benchmark
An evaluation suite for assessing foundation models in the DevOps field.
LLM evaluation framework
Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics
Model evaluation framework
A framework for evaluating OpenAI models and an open-source registry of benchmarks.
Code analyzer
A code analysis and automation platform
Model inspector
A framework for evaluating large language models
Evaluation pipeline
A framework for evaluating language models on NLP tasks
Coding flaws
A collection of common Python coding mistakes and poor practices
Model Evaluator
An evaluation framework for machine learning models and datasets, providing standardized metrics and tools for comparing model performance.
Code generator
An evaluation harness for generating Verilog code from natural language prompts
Code generator
A state-of-the-art open-source code generation model with human evaluability score comparable to GPT-4 and Google's proprietary models.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions bigcode-evaluation-harness.
