awesome-nlp-polish

Hate speech classification -distinguish between normal/non-harmful tweets (class: 0) and tweets that contain any kind of harmful information (class: 1) [ ] [ ]

The dataset for compositional distributional semantics. Polish CDSCorpus consists of 10K Polish sentence pairs which are human-annotated for semantic relatedness and entailment

corpus of Polish reviews annotated with sentiment at the level of the whole text ( ) and at the level of sentences ( ) for the following domains: hotels, medicine, products and university (reviews*)

Ermlab Opineo dataset

opineo reviews -

HateSpeech corpus contains over 2000 posts crawled from public Polish web

example: "Ateny Grecja Bagdad Irak" - useful for word embeddings evaluation

National Corpus of Polish. It contains classic literature, daily newspapers, specialist periodicals and journals, transcripts of conversations, and a variety of short-lived and internet texts. Only a small sub-corpus is available for (GNU GLP v.3). Direct contact and maybe necessary to get the full corpus

Polish Music Dataset is the largest dataset with information about artists, songs and lyrics in Poland (now only Hip Hop artists)

sentences 45.9M, polish tokens 287.1M ,collection of translated movie subtitles from

sentences 6.4M, polish tokens 157.1M

Polish Roberta Model

model was trained on a corpus consisting of Polish Wikipedia dump, Polish books and articles, Polish Parliamentary Corpus

PoLitBert

Polish RoBERTA model trained on Polish Wikipedia, Polish literature and Oscar. Major assumption is that quality text will give good model

PolBert

Polish BERT model. Model was trained with code provided in Google BERT's github repository. Merge with

Allegro HerBERT

Polish BERT model trained on Polish Corpora using only MLM objective with dynamic masking of whole words

SlavicBert - multilingual BERT model

-BERT, Slavic Cased: 4 languages(Bulgarian,Czech, Polish, Russian), 12-layer, 768-hidden, 12-heads, 110M parameters, 600Mb. There is also another SlavicBert model but I have problems to convert it to pytorch

A model of ELMo embeddings for Polish language trained on large textual corpora (KGR10)

Zalando Flair polish models

Contextual string embeddings that capture latent syntactic-semantic information that goes beyond standard word embeddings. There are two models "pl-forward and pl-backward"

Distributional language models for Polish trained on different corpora (KGR10, NKJP, Wikipedia)

Common Crawl

FastText polish model FB - train on: ,

sentence embeddings, it covers 16 languages (including Polish)

easy to use with

this collection contains ULMFiT recurrent language models trained on Wikipedia dumps for English and Polish. The models themselves were trained using FastAI and then exported to a TensorFlow-usable format. Code is available on

Single-word lemmatization and morphological analysis, Multi-word lemmatization,Disambiguated POS tagging, Dependency parsing, Shallow parsing, Named entity recognition, Summarization etc

https://github.com/sdadas/polish-nlp-resources

Github Repo with list of polish: word embeddings and language models (Word2vec, fasttext, Glove, Elmo) -

Polish Word Embeddings Review

Evaluation of polish word embeddings: word2vec, fastext etc. prepared by various research groups. Evaluation is done by words analogy task

Polish Sentence Evaluation

contains evaluation of eight sentence representation methods (Word2Vec, GloVe, FastText, ELMo, Flair, BERT, LASER, USE) on five polish linguistic tasks

complete user guide for trainning Roberta model with use of Huggingface/Transformers for polish