OmniNet
Multi-modal ML framework
An implementation of a unified architecture for multi-modal multi-task learning using PyTorch.
Official Pytorch implementation of "OmniNet: A unified architecture for multi-modal multi-task learning" | Authors: Subhojeet Pramanik, Priyanka Agrawal, Aman Hussain
515 stars
19 watching
58 forks
Language: Python
last commit: over 4 years ago artificial-intelligencedeep-learningimage-captioningmachine-learningmultimodal-learningmultitask-learningneural-networknlptransformervideo-recognition
Related projects:
| Repository | Description | Stars |
|---|---|---|
 multimodal-art-projection/omnibench | Evaluates and benchmarks multimodal language models' ability to process visual, acoustic, and textual inputs simultaneously. | 15 |
 open-mmlab/mmengine | Provides a flexible and configurable framework for training deep learning models with PyTorch. | 1,196 |
 minqi/learning-to-communicate-pytorch | This project implements a PyTorch-based framework for learning discrete communication protocols in multi-agent reinforcement learning environments. | 349 |
 oml-team/open-metric-learning | A PyTorch-based framework for training and validating models that produce high-quality embeddings for computer vision and other tasks. | 897 |
 txwang/mogonet | A framework for multi-omics data integration and classification using graph convolutional networks | 144 |
| open-mmlab/multimodal-gpt | Trains a multimodal chatbot that combines visual and language instructions to generate responses | 1,478 |
 namisan/mt-dnn | A PyTorch package implementing multi-task deep neural networks for natural language understanding | 2,238 |
 pmerienne/trident-ml | A real-time online machine learning library built on top of Storm and Trident. | 381 |
 unslothai/hyperlearn | An optimized machine learning framework for improved performance and reduced memory usage on various hardware configurations. | 1,871 |
 pku-yuangroup/moe-llava | A large vision-language model using a mixture-of-experts architecture to improve performance on multi-modal learning tasks | 2,023 |
 molcik/python-neuron | A Python library for implementing and training various neural network architectures | 40 |
 okerew/okrolearn | A Python machine learning library providing efficient array operations and neural network functionality | 3 |
 elbayadm/attn2d | A PyTorch implementation of 2D convolutional neural networks for sequence-to-sequence prediction in machine translation | 502 |
 vlgiitr/dmn-plus | A PyTorch implementation of an improved question answering architecture with dynamic memory networks and attention mechanisms | 64 |
 tristandeleu/pytorch-maml-rl | Replication of Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks in PyTorch for reinforcement learning tasks | 830 |