codefuse-devops-eval
by codefuse-ai
Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain.
AI summary
DevOps benchmark
An evaluation suite for assessing foundation models in the DevOps field.
- stars
- 690
- forks
- 44
- watching
- 9
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
DevOps Q&A model
An industrial-first language model for answering questions in the DevOps domain
DevOps assistant
An AI-powered tool designed to simplify and optimize various stages of the software development lifecycle
Testing assistant
A tool that empowers software testing with large language models.
Task-based fine-tuner
A framework for fine-tuning large language models with multiple tasks to improve their accuracy and efficiency
DevOps examples
A collection of publicly available resources on DevOps practices from companies around the world
Code environment framework
An interactive code environment framework for evaluating language agents through execution feedback.
Model evaluation framework
A framework for evaluating OpenAI models and an open-source registry of benchmarks.
cloud-cv/evalai1.8K
Benchmarking tool
A platform for comparing and evaluating AI and machine learning algorithms at scale
hkust-nlp/ceval1.7K
Evaluation suite
An evaluation suite providing multiple-choice questions for foundation models in various disciplines, with tools for assessing model performance.
Test generator
Generates sample tests and input/output files for competitive programming contests
Benchmarker
Tools for comparing and benchmarking small code snippets
Code intelligence tool
A benchmark dataset and open challenge to improve AI models' ability to understand and generate code
Model Evaluator
Evaluates language models using standardized benchmarks and prompting techniques.
Test runner
An action for running Codeception tests in GitHub workflows
Game environment benchmark
A benchmark for evaluating reinforcement learning agent performance on procedurally generated game-like environments.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions codefuse-devops-eval.
