UnIVAL
by mshukor
[TMLR23] Official implementation of UnIVAL: Unified Model for Image, Video, Audio and Language Tasks.
AI summary
Multitask model
A unified model for image, video, audio, and language tasks that can be fine-tuned for various downstream applications.
- stars
- 224
- forks
- 22
- watching
- 5
Similar projects
Found by comparing what the projects do, not just their names.
Multimodal model evaluator
Evaluating and improving large multimodal models through in-context learning
Multi-task learning framework
Provides a comprehensive framework for multi-task learning via structural regularization in MATLAB.
Multimodal learner
Develops a multimodal vision-language model to enable machines to understand complex relationships between instructions and images in various tasks.
Multimodal model developer
Develops large multimodal models for high-resolution understanding and analysis of text, images, and other data types.
Multimodal Model
A unified multimodal language model capable of interpreting and reasoning about various modalities without paired data.
Image captioner
An end-to-end image captioning system that uses large multi-modal models and provides tools for training, inference, and demo usage.
Retrieval model trainer
Trains and evaluates a universal multimodal retrieval model to perform various information retrieval tasks.
Multimodal LLM
A multi-modal large language model that integrates natural language and visual capabilities with fine-tuning for various tasks
Multimodal model
A large multimodal model for visual question answering, trained on a dataset of 2.1B image-text pairs and 8.2M instruction sequences.
Multi-modal ML framework
An implementation of a unified architecture for multi-modal multi-task learning using PyTorch.
Multimodal benchmarking
Evaluates and benchmarks multimodal language models' ability to process visual, acoustic, and textual inputs simultaneously.
ML Model Benchmarker
Provides a benchmarking framework and dataset for evaluating the performance of large language models in text-to-image tasks
Multimodal model framework
A framework for grounding language models to images and handling multimodal inputs and outputs
openbmb/viscpm1.1K
Multimodal Models
A family of large multimodal models supporting multimodal conversational capabilities and text-to-image generation in multiple languages
Multimodal alignment model
Extending pretraining models to handle multiple modalities by aligning language and video representations
