PromptInject
Prompt analysis tool
A framework for analyzing the robustness of large language models to adversarial prompt attacks
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022
318 stars
11 watching
32 forks
Language: Python
last commit: 10 months ago adversarial-attacksagiagi-alignmentai-alignmentai-safetychain-of-thoughtgpt-3language-modelslarge-language-modelsmachine-learningml-safetyprompt-engineering
Related projects:
| Repository | Description | Stars |
|---|---|---|
 protectai/rebuff | Protects AI applications from prompt injection attacks through multiple layers of defense | 1,144 |
 jthack/pipe | A guide to help developers understand and mitigate the security risks of prompt injection in AI-powered applications and features. | 376 |
 miesnerjacob/learn-prompting | A comprehensive resource for learning prompt engineering techniques for interacting with large language models. | 36 |
 krrishdholakia/betterprompt | An API for evaluating the quality of text prompts used in Large Language Models (LLMs) based on perplexity estimation | 43 |
 vaibkumr/prompt-optimizer | A tool to reduce the complexity of text prompts to minimize API costs and model computations. | 246 |
 mitre/advmlthreatmatrix | A framework to help security analysts understand and prepare for adversarial machine learning attacks on AI systems | 1,056 |
 microsoft/promptbench | A unified framework for evaluating large language models' performance and robustness in various scenarios. | 2,487 |
 instadeepai/mava | A research-friendly codebase for experimenting with multi-agent reinforcement learning in JAX | 749 |
 prompt-security/ps-fuzz | An interactive tool that tests and hardens the security of system prompts used in GenAI applications against various attacks. | 419 |
 rafalzawadzki/spellbook-forge | An ExpressJS middleware that allows users to execute LLM prompts stored in a git repository and retrieve results from a chosen model. | 74 |
 ga642381/speechprompt | An approach to leveraging pre-trained models for efficient speech processing tasks by using prompt tuning | 97 |
 deadbits/vigil-llm | A security scanner for Large Language Model prompts to detect potential threats and vulnerabilities | 326 |
 demisto/cops | Standardized framework for creating and sharing incident response processes in a shared language | 151 |
 ncwilson78/system-prompt-library | A comprehensive collection of customizable prompts for Generative Pre-trained Transformers (GPTs) designed specifically for educational use. | 77 |
 xcambar/purs | A Rust implementation of a minimal, fast, and aesthetically pleasing prompt system | 252 |