awesome-prompt-injection

General overview of Prompt Injection attacks, part of a series

This post shows how a malicious website can take control of a ChatGPT chat session and exfiltrate the history of the conversation

This post explores two prompt injections in OpenAI's browsing plugin for ChatGPT. These techniques exploit the input-dependent nature of AI conversational models, allowing an attacker to exfiltrate data through several prompt injection methods, posing significant privacy and security risks

A prompt injection cheat sheet for AI bot integrations

Video, slides, and a transcript of an introduction to prompt injection and why it's important

A guide on the various types of adversarial prompting and ways to mitigate them

A look into how to achieve prompt injection from control characters from Dropbox

A practical discussion about the unique complexities of securing LLMs from prompt injection attacks

Prompt Injection tutorial from Learn Prompting

Google's red team walkthrough of hacking AI systems

Token Turbulenz

A fuzzer to automate looking for possible Prompt Injections

Garak

Automate looking for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses in LLM's

As well as traditional challenges, this CTF also introduce scenarios that mimic agents in real-world applications

Your goal is to make Gandalf reveal the secret password for each level. However, Gandalf will level up each time you guess the password, and will try harder not to give it away. Can you beat level 7? (There is a bonus level 8)

This riddle requires you to have ChatGPT Plus access and enable the Browsing mode in Settings->Beta Features

Discord server from Learn Prompting