LLaVAR
by SALT-NLP
Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"
AI summary
Visual Instruction Tuning
An open-source project that enhances visual instruction tuning for text-rich image understanding by integrating GPT-4 models with multimodal datasets.
- stars
- 259
- forks
- 12
- watching
- 5
Similar projects
Found by comparing what the projects do, not just their names.
Visual Instruction Tuning Tool
A tool for generating and evaluating multimodal Large Language Models with visual instruction tuning capabilities
Chatbot fine-tuning
Fine-tuning the LLaMA 2 chat model using DeepSpeed and Lora for improved performance on a large dataset.
Visual Instruction Tuning
A dataset and model designed to scale visual instruction tuning using language-only GPT-4 models.
LLM tuning
A shared task for fine-tuning large language models to answer questions and generate responses in Ukrainian.
Image processor
An all-in-one demo for interactive image processing and generation
Visual encoder
A vision-language model that uses a query transformer to encode images as visual tokens and allows flexible choice of the number of visual tokens.
Video image processor
An image-based language model that uses large language models to generate visual and text features from videos
Visual guidance
This project presents a new approach to fine-grained visual understanding using pixel-wise mask regions in language instructions
Instruction generator
Creating synthetic visual reasoning instructions to improve the performance of large language models on image-related tasks
Visual Instruction Toolkit
A method and toolkit for fine-tuning large language models to perform visual instruction tasks in multiple languages.
aidc-ai/ovis575
Multimodal aligner
An MLLM architecture designed to align visual and textual embeddings through structural alignment
Visual Prompt Model
A system designed to enable large multimodal models to understand arbitrary visual prompts
haotian-liu/llava20.7K
Visual Instruction System
A system that uses large language and vision models to generate and process visual instructions
Multimodal model trainer
An implementation of a multimodal LLM training paradigm to enhance truthfulness and ethics in language models
Model optimizer
This project presents an optimization technique for large-scale image models to reduce computational requirements while maintaining performance.
