vllm-safety-benchmark
by UCSC-VLAA
[ECCV 2024] Official PyTorch Implementation of "How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs"
AI summary
Vision model safety test
A benchmark for evaluating the safety and robustness of vision language models against adversarial attacks.
- stars
- 72
- forks
- 3
- watching
- 4
Similar projects
Found by comparing what the projects do, not just their names.
Visual Model Benchmark
An open-source benchmarking framework for evaluating cross-style visual capability of large multimodal models
Multimodal model trainer
An implementation of a multimodal LLM training paradigm to enhance truthfulness and ethics in language models
Safety fine-tuner
Improves safety and helpfulness of large language models by fine-tuning them using safety-critical tasks
LLM vulnerability detector
A toolkit to detect and protect against vulnerabilities in Large Language Models.
Model risk assessment toolkit
A set of tools and guidelines for assessing the security vulnerabilities of language models in AI applications
Safe RL Benchmark
A unified benchmark for safe reinforcement learning algorithms and environments.
Trust Assessment Toolkit
A toolkit for assessing trustworthiness in large language models
Model robustness testing
Evaluates and benchmarks the robustness of deep learning models to various corruptions and perturbations in computer vision tasks.
Visual decoder
A large language model designed to process and generate visual information
Vision Language Model
Develops a PyTorch implementation of an enhanced vision language model
VQA model
A multimodal LLM designed to handle text-rich visual questions
Vision-Language Model
A PyTorch implementation of an encoder-free vision-language model that can be fine-tuned for various tasks and modalities
Image segmentation tool
A system that uses large language models to generate segmentation masks for images based on complex queries and world knowledge.
Computer Vision Toolkit
A PyTorch toolbox for supporting research and development of domain adaptation, generalization, and semi-supervised learning methods in computer vision.
Multimodal LLM test suite
A benchmark for evaluating large language models' ability to process multimodal input
