promptfoo
by promptfoo
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
AI summary
LLM tester
A tool for testing and evaluating large language models (LLMs) to ensure they are reliable and secure
- stars
- 5K
- forks
- 405
- watching
- 21
- awesome lists
- 2
Similar projects
Found by comparing what the projects do, not just their names.
Prompt tester
A set of tools for testing and evaluating natural language processing models and vector databases.
Prompt testing env
An environment for battle-testing prompts to Large Language Models (LLMs) to evaluate response quality and performance.
Text parser
A tool that uses large language models to extract structured information from unstructured text
Prompt generator
A utility library for crafting prompts to help Large Language Models generate specific outputs
AI app templates
Provides pre-built AI application templates to integrate Large Language Models (LLMs) with various data sources for scalable RAG and enterprise search.
LLM evaluation toolkit
A toolkit for evaluating and optimizing Large Language Model applications with objective metrics, test data generation, and seamless integrations.
Model evaluator
A unified framework for evaluating large language models' performance and robustness in various scenarios.
LLM evaluator
A framework for evaluating large language models
Prompt Server
An ExpressJS middleware that allows users to execute LLM prompts stored in a git repository and retrieve results from a chosen model.
Prompt analysis tool
A framework for analyzing the robustness of large language models to adversarial prompt attacks
Evaluation framework
Provides a unified framework to test generative language models on various evaluation tasks.
langgenius/dify54.9K
AI application builder
An open-source LLM app development platform that enables users to build and deploy AI-powered applications quickly and efficiently.
LLM app builder
A framework for building enterprise LLM-based applications using small, specialized models
Agent framework
A Python framework to build LLM-powered applications by setting up agents with optional components and having them collaboratively solve problems through message exchange
LLM Tester
An extension of Vitest for testing LLM applications using local language models
Featured in 2 awesome lists
Each link jumps to the spot where the list mentions promptfoo.
