awesome-prompt-injection

Vulnerability assessment

Provides resources and information on a type of vulnerability targeting machine learning models

Learn about a type of vulnerability that specifically targets machine learning models

GitHub
196 stars
6 watching
33 forks
last commit: 6 months ago
Linked from 1 awesome list

awesomeawesome-list

Awesome Prompt Injection / Articles and Blog posts
Prompt injection: What's the worst that can happen? General overview of Prompt Injection attacks, part of a series
ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery This post shows how a malicious website can take control of a ChatGPT chat session and exfiltrate the history of the conversation
Data exfiltration via Indirect Prompt Injection in ChatGPT This post explores two prompt injections in OpenAI's browsing plugin for ChatGPT. These techniques exploit the input-dependent nature of AI conversational models, allowing an attacker to exfiltrate data through several prompt injection methods, posing significant privacy and security risks
Prompt Injection Cheat Sheet: How To Manipulate AI Language Models A prompt injection cheat sheet for AI bot integrations
Prompt injection explained Video, slides, and a transcript of an introduction to prompt injection and why it's important
Adversarial Prompting A guide on the various types of adversarial prompting and ways to mitigate them
Don't you (forget NLP): Prompt injection with control characters in ChatGPT A look into how to achieve prompt injection from control characters from Dropbox
Testing the Limits of Prompt Injection Defence A practical discussion about the unique complexities of securing LLMs from prompt injection attacks

Awesome Prompt Injection / Tutorials
Prompt Injection Prompt Injection tutorial from Learn Prompting
AI Read Teaming from Google Google's red team walkthrough of hacking AI systems

Awesome Prompt Injection / Tools
Token Turbulenz 15 over 1 year ago A fuzzer to automate looking for possible Prompt Injections
Garak 3,043 14 days ago Automate looking for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses in LLM's

Awesome Prompt Injection / CTF
Promptalanche As well as traditional challenges, this CTF also introduce scenarios that mimic agents in real-world applications
Gandalf Your goal is to make Gandalf reveal the secret password for each level. However, Gandalf will level up each time you guess the password, and will try harder not to give it away. Can you beat level 7? (There is a bonus level 8)
ChatGPT with Browsing is drunk! There is more to it than you might expect at first glance This riddle requires you to have ChatGPT Plus access and enable the Browsing mode in Settings->Beta Features

Awesome Prompt Injection / Community
Learn Prompting Discord server from Learn Prompting

Backlinks from these awesome lists:

More related projects: