LLaVA-MoD

This project provides a novel framework for training small-scale Multimodal Language Models by distilling knowledge from larger models using a sparse Mixture of Experts architecture.

Making LLaVA Tiny via MoE-Knowledge Distillation

GitHub

last commit: 3 months ago