awesome-sentence-embedding

awesome listarchivedPythonpushed over 5 years ago

A curated list of pretrained sentence and word embedding models

stars: 2.2K
forks: 261
watching: 78
awesome list: 1
entries: 402

What's in the list

402 links in 9 sections, with live GitHub stats.activeno commit in 2y

Word Embeddings

WebVectors: A Toolkit for Building Web Interfaces for Vector Semantic Models
RusVectōrēs
Efficient Estimation of Word Representations in Vector Space
C
1.5Kover 3 years
Word2Vec
Word Representations via Gaussian Embedding
Cython
190over 8 years
A Probabilistic Model for Learning Multi-Prototype Word Embeddings
DMTK
116over 10 years
Dependency-Based Word Embeddings
C++
word2vecf
GloVe: Global Vectors for Word Representation
C
6.9Kalmost 2 years
GloVe
6.9Kalmost 2 years
Sparse Overcomplete Word Vector Representations
C++
549 years
From Paraphrase Database to Compositional Paraphrase Model and Back
Theano
30over 10 years
PARAGRAM
Non-distributional Word Vector Representations
Python
629 years
WordFeat
629 years
Joint Learning of Character and Word Embeddings
C
2996 years
SensEmbed: Learning Sense Embeddings for Word and Relational Similarity
SensEmbed
Topical Word Embeddings
Cython
314over 8 years
Swivel: Improving Embeddings by Noticing What's Missing
TF
77.3Kalmost 2 years
Counter-fitting Word Vectors to Linguistic Constraints
Python
145over 6 years
counter-fitting
(broken)
Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec
Chainer
3.2Kalmost 5 years
Siamese CBOW: Optimizing Word Embeddings for Sentence Representations
Theano
Siamese CBOW
Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations
Go
803almost 6 years
lexvec
803almost 6 years
Enriching Word Vectors with Subword Information
C++
26Kover 2 years
fastText
Morphological Priors for Probabilistic Neural Word Embeddings
Theano
52almost 10 years
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
C++
23almost 3 years
charNgram2vec
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Python
1.3K4 years
Numberbatch
1.3K4 years
Learning Word Meta-Embeddings
Meta-Emb
(broken)
Offline bilingual word vectors, orthogonal transformations and the inverted softmax
Python
1.2Kover 3 years
Multimodal Word Distributions
TF
283over 7 years
word2gm
283over 7 years
Poincaré Embeddings for Learning Hierarchical Representations
Pytorch
1.7K2 years
Context encoders as a simple but powerful extension of word2vec
Python
20over 6 years
Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
TF
649 years
Attract-Repel
649 years
Learning Chinese Word Representations From Glyphs Of Characters
C
30over 8 years
Making Sense of Word Embeddings
Python
212over 5 years
sensegram
Hash Embeddings for Efficient Word Representations
Keras
42almost 9 years
BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages
Gensim
1.2K2 years
BPEmb
1.2K2 years
SPINE: SParse Interpretable Neural Embeddings
Pytorch
52over 6 years
SPINE
AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP
Gensim
395over 5 years
AraVec
395over 5 years
Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics
C
8487 years
Dict2vec : Learning Word Embeddings using Lexical Dictionaries
C++
115almost 6 years
Dict2vec
115almost 6 years
Joint Embeddings of Chinese Words, Characters, and Fine-grained Subcharacter Components
C
99over 7 years
Representation Tradeoffs for Hyperbolic Embeddings
Pytorch
377over 3 years
h-MDS
377over 3 years
Dynamic Meta-Embeddings for Improved Sentence Representations
Pytorch
3326 years
DME/CDME
3326 years
Analogical Reasoning on Chinese Morphological and Semantic Relations
ChineseWordVectors
11.9Kalmost 3 years
Probabilistic FastText for Multi-Sense Word Embeddings
C++
148over 8 years
Probabilistic FastText
148over 8 years
Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks
TF
291over 3 years
SynGCN
FRAGE: Frequency-Agnostic Word Representation
Pytorch
118over 7 years
Wikipedia2Vec: An Optimized Tool for LearningEmbeddings of Words and Entities from Wikipedia
Cython
946over 2 years
Wikipedia2Vec
Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings
ChineseEmbedding
cw2vec: Learning Chinese Word Embeddings with Stroke n-gram Information
C++
274over 3 years
VCWE: Visual Character-Enhanced Word Embeddings
Pytorch
15over 7 years
VCWE
15over 7 years
Learning Cross-lingual Embeddings from Twitter via Distant Supervision
Text
14over 6 years
An Unsupervised Character-Aware Neural Approach to Word and Context Representation Learning
TF
0almost 8 years
ViCo: Word Embeddings from Visual Co-occurrences
Pytorch
257 years
ViCo
257 years
Spherical Text Embedding
C
175almost 3 years
Unsupervised word embeddings capture latent knowledge from materials science literature
Gensim
624over 3 years

OOV Handling

ALaCarte
:
104almost 8 years
Mimick
:
153almost 7 years
CompactReconstruction
:
9over 3 years

Contextualized Word Embeddings

Language Models are Unsupervised Multitask Learners
TF
22.6K2 years
117M
GPT-2( , , , , , )
22.6K2 years
Learned in Translation: Contextualized Word Vectors
Pytorch
473over 4 years
CoVe
473over 4 years
Universal Language Model Fine-tuning for Text Classification
Pytorch
26.4Kalmost 2 years
English
ULMFit( , )
Deep contextualized word representations
Pytorch
11.8Kalmost 4 years
AllenNLP
ELMO( , )
Efficient Contextualized Representation:Language Model Pruning for Sequence Labeling
Pytorch
147over 6 years
LD-Net
147over 6 years
Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation
Pytorch
1.5Kover 5 years
ELMo
1.5Kover 5 years
Direct Output Connection for a High-Rank Language Model
Pytorch
12over 7 years
DOC
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TF
38.4K2 years
BERT
BERT( , , )
38.4K2 years
Contextual String Embeddings for Sequence Labeling
Pytorch
14Kalmost 2 years
Flair
14Kalmost 2 years
Improving Language Understanding by Generative Pre-Training
TF
2.2Kover 7 years
GPT
2.2Kover 7 years
Multi-Task Deep Neural Networks for Natural Language Understanding
Pytorch
2.2Kover 2 years
MT-DNN
2.2Kover 2 years
BioBERT: pre-trained biomedical language representation model for biomedical text mining
TF
2K3 years
BioBERT
672over 6 years
Cross-lingual Language Model Pretraining
Pytorch
2.9Kover 3 years
XLM
2.9Kover 3 years
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
TF
3.6K4 years
Transformer-XL
3.6K4 years
Efficient Contextual Representation Learning Without Softmax Layer
Pytorch
4over 6 years
SciBERT: Pretrained Contextualized Embeddings for Scientific Text
Pytorch, TF
1.5Kover 4 years
SciBERT
1.5Kover 4 years
Publicly Available Clinical BERT Embeddings
Text
6806 years
clinicalBERT
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Pytorch
386almost 4 years
ClinicalBERT
ERNIE: Enhanced Language Representation with Informative Entities
Pytorch
1.4Kover 2 years
ERNIE
Unified Language Model Pre-training for Natural Language Understanding and Generation
Pytorch
20.4Kalmost 2 years
unilm1-large-cased
UniLMv1( , )
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
Pre-Training with Whole Word Masking for Chinese BERT
Pytorch, TF
9.7K3 years
BERT-wwm
9.7K3 years
XLNet: Generalized Autoregressive Pretraining for Language Understanding
TF
6.2Kover 3 years
XLNet
6.2Kover 3 years
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
PaddlePaddle
6.3K2 years
ERNIE 2.0
6.3K2 years
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Pytorch
8933 years
SpanBERT
8933 years
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Pytorch
30.7Kalmost 2 years
RoBERTa
30.7Kalmost 2 years
Subword ELMo
Pytorch
12over 6 years
Knowledge Enhanced Contextual Word Representations
TinyBERT: Distilling BERT for Natural Language Understanding
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Pytorch
10.8Kalmost 2 years
BERT-345M
Megatron-LM( , )
MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
Pytorch
284over 6 years
Extreme Language Model Compression with Optimal Subwords and Shared Projections
MULE: Multimodal Universal Language Embedding
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
K-BERT: Enabling Language Representation with Knowledge Graph
UNITER: Learning UNiversal Image-TExt Representations
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
TF
3.9Kalmost 4 years
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Pytorch
30.7Kalmost 2 years
bart.base
BART( , , , , )
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Pytorch, TF2.0
136.4Kalmost 2 years
DistilBERT
136.4Kalmost 2 years
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
TF
6.2K2 years
T5
6.2K2 years
CamemBERT: a Tasty French Language Model
CamemBERT
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
Pytorch
6454 years
Unsupervised Cross-lingual Representation Learning at Scale
Pytorch
2.9Kover 3 years
xlmr.large
XLM-R (XLM-RoBERTa)( , )
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
Pytorch
6942 years
ProphetNet-large-16GB
ProphetNet( , )
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Pytorch
2.3K3 years
CodeBERT
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Pytorch
20.4Kalmost 2 years
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
TF
2.3Kover 2 years
ELECTRA-Small
ELECTRA( , , )
MPNet: Masked and Permuted Pre-training for Language Understanding
Pytorch
2885 years
MPNet
ParsBERT: Transformer-based Model for Persian Language Understanding
Pytorch
341over 3 years
ParsBERT
Language Models are Few-Shot Learners
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Pytorch
20.4Kalmost 2 years

Pooling Methods

SIF
:
1.1K7 years
TF-IDF
:
9over 3 years
P-norm
:
186almost 6 years
DisC
:
54over 6 years
GEM
:
19almost 8 years
SWEM
:
284almost 4 years
VLAWE
:
10over 7 years
Efficient Sentence Embedding using Discrete Cosine Transform
fse: Gensim add-on for fast sentence embeddings. Supports Mean, Max, SIF, uSIF
618over 3 years
Efficient Sentence Embedding via Semantic Subspace Analysis

Encoders

Incremental Domain Adaptation for Neural Machine Translation in Low-Resource Settings
Python
57 years
Distributed Representations of Sentences and Documents
Pytorch
413almost 4 years
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Theano
427over 9 years
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
Theano
2.1Kover 6 years
Order-Embeddings of Images and Language
Theano
186almost 10 years
Towards Universal Paraphrastic Sentence Embeddings
Theano
193over 10 years
From Word Embeddings to Document Distances
C, Python
538over 2 years
Learning Distributed Representations of Sentences from Unlabelled Data
Python
124over 9 years
Charagram: Embedding Words and Sentences via Character n-grams
Theano
12510 years
Learning Generic Sentence Representations Using Convolutional Neural Networks
Theano
349 years
Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
C++
1.2K4 years
Learning to Generate Reviews and Discovering Sentiment
TF
1.5Kover 3 years
Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings
Theano
33over 9 years
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Pytorch
2.3K5 years
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
Pytorch
492almost 5 years
Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm
Keras
1.5K2 years
StarSpace: Embed All The Things!
C++
3.9Kalmost 4 years
DisSent: Learning Sentence Representations from Explicit Discourse Relations
Pytorch
33over 6 years
Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations
Theano
102almost 3 years
Dual-Path Convolutional Image-Text Embedding with Instance Loss
Matlab
287over 3 years
An efficient framework for learning sentence representations
TF
2057 years
Universal Sentence Encoder
TF-Hub
End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions
Theano
16over 8 years
Learning general purpose distributed sentence representations via large scale multi-task learning
Pytorch
3116 years
Embedding Text in Hyperbolic Spaces
TF
89 years
Representation Learning with Contrastive Predictive Coding
Keras
527over 7 years
Context Mover’s Distance & Barycenters: Optimal transport of contexts for building representations
Python
21over 5 years
Learning Universal Sentence Representations with Mean-Max Attention Autoencoder
TF
16almost 8 years
Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
TF-Hub
Improving Sentence Representations with Consensus Maximisation
BioSentVec: creating sentence embeddings for biomedical texts
Python
5783 years
Word Mover's Embedding: From Word2Vec to Document Embedding
C, Python
81almost 8 years
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks
Pytorch
1.2K3 years
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Pytorch
3.6Kover 2 years
Convolutional Neural Network for Universal Sentence Embeddings
Theano
2over 8 years
No Training Required: Exploring Random Encoders for Sentence Classification
Pytorch
184over 6 years
CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model
Pytorch
21over 7 years
GLOSS: Generative Latent Optimization of Sentence Representations
Multilingual Universal Sentence Encoder
TF-Hub
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Pytorch
15.6Kalmost 2 years
SBERT-WK: A Sentence Embedding Method By Dissecting BERT-based Word Models
Pytorch
178over 5 years
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
Pytorch
380over 3 years
Language-agnostic BERT Sentence Embedding
TF-Hub
On the Sentence Embeddings from Pre-trained Language Models
TF
530over 5 years

Evaluation

decaNLP
:
2.3Kover 2 years
SentEval
:
2.1Kover 2 years
GLUE
:
7795 years
Exploring Semantic Properties of Sentence Embeddings
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Word Embeddings Benchmarks
:
437almost 6 years
MLDoc
:
152over 4 years
LexNET
:
77.3Kalmost 2 years
wordvectors.net
:
120over 5 years
jiant
:
1.7Kover 3 years
jiant
:
1.7Kover 3 years
Evaluation of sentence embeddings in downstream and linguistic probing tasks
QVEC
:
75over 8 years
Grammatical Analysis of Pretrained Sentence Encoders with Acceptability Judgments
EQUATE : A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference
Evaluating Word Embedding Models: Methods andExperimental Results
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions
Linguistic Knowledge and Transferability of Contextual Representations
:
LINSPECTOR
:
24almost 7 years
Pitfalls in the Evaluation of Sentence Embeddings
Probing Multilingual Sentence Representations With X-Probe
:

Misc

Vector Mapping

Cross-lingual Word Vectors Projection Using CCA
:
568 years
vecmap
:
648over 3 years
MUSE
:
3.2K4 years
CrossLingualELMo
:
99over 6 years

Articles

Featured in 1 awesome list

Each link jumps to the spot where the list mentions awesome-sentence-embedding.

0ex/more-awesome