LLM-security-and-privacy

LLM threat analysis

A curated collection of research papers and tools examining the security and privacy risks associated with large language models.

LLM security and privacy

GitHub
41 stars
2 watching
7 forks
Language: TeX
last commit: about 1 month ago
Linked from 1 awesome list

awesomeawesome-listawesome-llm-security-and-privacygenerative-aillmllm-frameworkllm-privacyllm-securityllm-threatsllm-vulnerabilitiesprivacysecurity

LLM Security and Privacy / Papers
InjectAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
LLM Agents can Autonomously Hack Websites
An Overview of Catastrophic AI Risks
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities
LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?
Beyond the Safeguards: Exploring the Security Risks of ChatGPT
Prompt Injection attack against LLM-integrated Applications
Identifying and Mitigating the Security Risks of Generative AI
PassGPT: Password Modeling and (Guided) Generation with Large Language Models
ESORICS
javirandor/passgpt 49 about 1 year ago
Harnessing GPT-4 for generation of cybersecurity GRC policies: A focus on ransomware attack mitigation
Computers & Security
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
greshake/llm-security 1,828 over 1 year ago
Examining Zero-Shot Vulnerability Repair with Large Language Models
IEEE S&P
LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins
Chain-of-Verification Reduces Hallucination in Large Language Models
Pop Quiz! Can a Large Language Model Help With Reverse Engineering?
Extracting Training Data from Large Language Models
Usenix Security
ftramer/LM_Memorization 175 almost 2 years ago
Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications
StavC/ComPromptMized 195 9 months ago
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
EMNLP
ShuyangCao/cliff_summ 45 almost 3 years ago

LLM Security and Privacy / Frameworks & Taxonomies
OWASP Top 10 for Large Language Model Applications
MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems)
NIST AI 100-2 E2023: Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

LLM Security and Privacy / Tools
Arsenal - CALDERA Plugin 85 over 1 year ago
Vigil - Prompt injection detection 309 10 months ago

LLM Security and Privacy / News Articles, Blog Posts, and Talks
Is Generative AI Dangerous?
Adversarial examples in the age of ChatGPT
LLMs in Security: Demos vs Deployment?
Free AI Programs Prone to Security Risks, Researchers Say
Why 'Good AI' Is Likely The Antidote To The New Era Of AI Cybercrime
Meet PassGPT, the AI Trained on Millions of Leaked Passwords

Backlinks from these awesome lists:

0