MNBVC
by esbatmop
MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。
AI summary
Chinese text corpus
A massive corpus of Chinese text data covering various forms and styles
- stars
- 3.6K
- forks
- 249
- watching
- 66
Similar projects
Found by comparing what the projects do, not just their names.
LLM app dev guide
A comprehensive guide to building applications with Large Language Models (LLMs) for developers
AI automation tool
An experimental application showcasing GPT-4's capabilities through automation and AI-driven workflows.
Novel Generator
Generates Chinese novel text using a pre-trained language model
Chinese language models
Develops and publishes pre-trained Chinese language models using Whole Word Masking technology.
fudannlp/fnlp2.6K
NLP toolkit
A toolkit for Chinese natural language processing tasks
Deep Learning Book
Translation of a popular deep learning book into Chinese, aiming to improve accuracy and accessibility.
thudm/glm-45.5K
Multimodal chat model
A large language model designed for multilingual and multimodal chat applications with advanced features such as long-text reasoning and high-performance inference.
Dialog model
A multimodal dialog language model that generates responses based on images and text
Medical LLM
Develops and deploys a large language model for Chinese traditional medicine applications
LLM toolkit
Guides users to build applications using LangChain's framework and integrate it with Large Language Models (LLMs) for tasks like text generation, summarization, and search.
Chatbot
An open-source chatbot project built on Solid.js and OpenAI's GPT technology, with features like PWA support and customizable prompts.
tencent/kbone4.8K
Web frontend adapter
An adapter layer for running web frontend code in WeChat Mini Programs
Async thread pool manager
A framework for managing asynchronous threads with dynamic configuration and real-time monitoring
GPT2 Trainer
A repository providing a Chinese version of the GPT2 training code, utilizing BERT tokenizer.
Chinese NLP toolkit
A comprehensive, user-centered ecosystem of pre-trained NLP models for the Chinese language
