TaCL
by yxuansu
[NAACL'22] TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
AI summary
Token representation refinement
Improves pre-trained language models by encouraging an isotropic and discriminative distribution of token representations.
- stars
- 92
- forks
- 6
- watching
- 5
Similar projects
Found by comparing what the projects do, not just their names.
ymcui/pert356
Semantic learning model
Develops a pre-trained language model to learn semantic knowledge from permuted text without mask labels
Chinese NLP Models
Provides pre-trained models for Chinese natural language processing tasks using the XLNet architecture
Chinese language model fine-tuning tool
Improves pre-trained Chinese language models by incorporating a correction task to alleviate inconsistency issues with downstream tasks
Chinese language model
Trains a large Chinese language model on massive data and provides a pre-trained model for downstream tasks
Vision-Language Learning Model
Develops and trains models for vision-language learning with decoupled language pre-training
Chinese language models
Provides pre-trained models for Chinese language tasks with improved performance and smaller model sizes compared to existing models.
NLP model reproducer
Reproduces the results of an ACL 2018 paper on simple word-embedding-based models for natural language processing tasks.
Multimodal alignment model
Extending pretraining models to handle multiple modalities by aligning language and video representations
Linguistic Model
A pre-trained language model designed to leverage linguistic features and outperform comparable baselines on Chinese natural language understanding tasks.
Benchmark
An image-context reasoning benchmark designed to challenge large vision-language models and help improve their accuracy
Education NLP model
A pre-trained language model designed to improve natural language processing tasks in education
Tokeniser library
A Python binding to an advanced, extensible tokeniser written in C++
Textual entailment system
Reproducible code and pre-trained model for an ACL2018 paper on textual entailment via deep explorations of inter-sentence interactions.
Chinese text encoder
A pre-trained BERT-based Chinese text encoder with enhanced N-gram representations
Reinforcement Learning framework
Replication of Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks in PyTorch for reinforcement learning tasks
