Chat-UniVi
[CVPR 2024 Highlightš„] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
AI summary
Visual unification framework
A framework for unified visual representation in image and video understanding models, enabling efficient training of large language models on multimodal data.
- stars
- 895
- forks
- 43
- watching
- 7
Similar projects
Found by comparing what the projects do, not just their names.
Multimodal alignment model
Extending pretraining models to handle multiple modalities by aligning language and video representations
Video benchmarking toolkit
Evaluates and benchmarks large language models' video understanding capabilities
Vision Language Integrator
Improves performance of vision language tasks by integrating computer vision capabilities into large language models
jy0205/lavit544
Visual understanding and generation framework
A unified framework for training large language models to understand and generate visual content
nvlabs/prismer1.3K
Vision-Language Model
A deep learning framework for training multi-modal models with vision and language capabilities.
Visual Prompt Model
A system designed to enable large multimodal models to understand arbitrary visual prompts
Visual representation improvement
A project aimed at improving visual representations in vision-language models by developing an object detection model for richer visual object and concept representations.
Multimodal evaluation framework
Develops a multimodal task and dataset to assess vision-language models' ability to handle interleaved image-text inputs.
Mixture of Experts Model
A large vision-language model using a mixture-of-experts architecture to improve performance on multi-modal learning tasks
Vision Reasoning Framework
A deep learning framework for iteratively decomposing vision and language reasoning via large language models.
Vision-Language Model Framework
Implementing a unified modal learning framework for generative vision-language models
Visual Knowledge Model
This project integrates visual knowledge into large language models to improve their capabilities and reduce hallucinations.
Image captioner
An end-to-end image captioning system that uses large multi-modal models and provides tools for training, inference, and demo usage.
Visual search framework
PyTorch implementation of guided visual search mechanism for multimodal LLMs
Image translator
An unsupervised deep learning framework for translating images between different modalities
