Skywork-MoE
by SkyworkAI
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
AI summary
MoE model
A high-performance mixture-of-experts model with innovative training techniques for language processing tasks
- stars
- 126
- forks
- 7
- watching
- 7
Similar projects
Found by comparing what the projects do, not just their names.
Multilingual model
A pre-trained language model developed on 3.2TB of high-quality multilingual and code data for various applications including chatbots, text generation, and math calculations.
Multi-modal language model
An empirical study aiming to develop a large language model capable of effectively integrating multiple input modalities
Mixture of Experts Model
A large vision-language model using a mixture-of-experts architecture to improve performance on multi-modal learning tasks
Language Model
A high-performance language model designed to excel in tasks like natural language understanding, mathematical computation, and code generation
Mixture-of-Experts Model
Developed by XVERSE Technology Inc. as a multilingual large language model with a unique mixture-of-experts architecture and fine-tuned for various tasks such as conversation, question answering, and natural language understanding.
Multilingual Model
Develops and publishes large multilingual language models with advanced mixing-of-experts architecture.
Multilingual models
Large language models designed to perform well in multiple languages and address performance issues with current multilingual models.
Language Models
A repository of pre-trained language models for various tasks and domains.
Language models
A collection of lightweight state-of-the-art language models designed to support multilinguality, coding, and reasoning tasks on constrained resources.
Language model trainer
A toolkit for training neural network language models
Efficient LLM
A large language model with improved efficiency and performance compared to similar models
Language Model
An implementation of a neural network model for character-level language modeling.
01-ai/yi7.7K
NLP model
A series of large language models trained from scratch to excel in multiple NLP tasks
Chinese language model fine-tuning tool
Improves pre-trained Chinese language models by incorporating a correction task to alleviate inconsistency issues with downstream tasks
Language model trainer
This project provides code and model for improving language understanding through generative pre-training using a transformer-based architecture.
