Osprey
by CircleRadon
[CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"
AI summary
Visual guidance
This project presents a new approach to fine-grained visual understanding using pixel-wise mask regions in language instructions
- stars
- 781
- forks
- 42
- watching
- 14
Similar projects
Found by comparing what the projects do, not just their names.
Instruction generator
Creating synthetic visual reasoning instructions to improve the performance of large language models on image-related tasks
Visual Instruction Tuning
An open-source project that enhances visual instruction tuning for text-rich image understanding by integrating GPT-4 models with multimodal datasets.
roboflow/maestro1.4K
fine-tuner
A tool to streamline fine-tuning of multimodal models for vision-language tasks
Safety fine-tuner
Improves safety and helpfulness of large language models by fine-tuning them using safety-critical tasks
Region-of-Interest Training
Training and deploying large language models on computer vision tasks using region-of-interest inputs
Visual Instruction Toolkit
A method and toolkit for fine-tuning large language models to perform visual instruction tasks in multiple languages.
aidc-ai/ovis575
Multimodal aligner
An MLLM architecture designed to align visual and textual embeddings through structural alignment
Visual search framework
PyTorch implementation of guided visual search mechanism for multimodal LLMs
Word embeddings
Multi-sense word embeddings learned from visual cooccurrences
Navigation library
A Ruby gem for creating hierarchical navigation structures in web applications
Visual Instruction Tuning
A dataset and model designed to scale visual instruction tuning using language-only GPT-4 models.
Vision Language Integrator
Improves performance of vision language tasks by integrating computer vision capabilities into large language models
Visual matcher
An open-source PyTorch implementation of a visual semantic reasoning model for image-text matching
Visual Text Understanding
This project enables multi-modal language models to understand and generate text about visual content using referential comprehension.
OCR library
A Ruby library providing an interface to the Tesseract OCR system.
