prompttools
Prompt tester
A set of tools for testing and evaluating natural language processing models and vector databases.
Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
3k stars
32 watching
234 forks
Language: Python
last commit: 7 months ago
Linked from 1 awesome list
deep-learningdeveloper-toolsembeddingslarge-language-modelsllmsmachine-learningprompt-engineeringpythonvector-search
hannibal046/awesome-llmRelated projects:
| Repository | Description | Stars |
|---|---|---|
 promptfoo/promptfoo | A tool for testing and evaluating large language models (LLMs) to ensure they are reliable and secure | 4,976 |
 ianarawjo/chainforge | An environment for battle-testing prompts to Large Language Models (LLMs) to evaluate response quality and performance. | 2,413 |
 promptslab/promptify | A tool that uses large language models to extract structured information from unstructured text | 3,327 |
 microsoft/prompt-engine | A utility library for crafting prompts to help Large Language Models generate specific outputs | 2,602 |
| microsoft/promptbench | A unified framework for evaluating large language models' performance and robustness in various scenarios. | 2,487 |
 brexhq/prompt-engineering | Guides software developers on how to effectively use and build systems around Large Language Models like GPT-4. | 8,487 |
 mshumer/gpt-prompt-engineer | A tool for automating the process of generating and ranking effective prompts for AI models like GPT-4, GPT-3.5-Turbo, or Claude 3 Opus. | 9,411 |
 bigscience-workshop/promptsource | A toolkit for creating and using natural language prompts to enable large language models to generalize to new tasks. | 2,718 |
 eleutherai/lm-evaluation-harness | Provides a unified framework to test generative language models on various evaluation tasks. | 7,200 |
 openvinotoolkit/open_model_zoo | A collection of pre-trained deep learning models and demo applications for accelerating inference tasks | 4,118 |
 openai/evals | A framework for evaluating large language models and systems, providing a registry of benchmarks. | 15,168 |
 thunlp/openprompt | A flexible framework for adapting pre-trained language models to downstream NLP tasks using textual templates | 4,398 |
| openvinotoolkit/openvino | A toolkit for optimizing and deploying artificial intelligence models in various applications | 7,439 |
 jackmort/chatgpt.nvim | A plugin for Neovim that integrates with the ChatGPT API to generate natural language responses and assist with coding tasks. | 3,825 |
 bin-huang/chatbox | A desktop application for interacting with AI models and language learning platforms | 23,791 |