awesome-ukrainian-nlp

— 113GB of text, compilation of UberText 2.0, OSCAR, Ukrainian News

Brown-UK

— carefully curated corpus of modern Ukrainian language with dismabiguated tokens, 1 million words

— over 5 GB of news, Wikipedia, social, fiction, and legal texts

— shuffled sentences extracted from and classified with a language detection model. Ukrainian portion of it is 28GB deduplicated

— documents extracted from , automatically classified and filtered. Ukrainian part is 200M sentences or 10GB of deduplicated text

mC4

— filtered CommonCrawl again, 196GB of Ukrainian text

Ukrainian Twitter corpus

Ukrainian Twitter corpus for toxic text detection

Ukrainian forums

— 250k sentences scraped from forums

— 5.2M news headlines

— 5M sentence edits extracted from the Ukrainian Wikipedia revision history

— ~4000 questions and answers from Ukrainian External independent testing (ЗНО/ZNO)

UA-GEC

— grammatical error correction (GEC) and fluency corpus

NER-uk

— Brown-UK labeled for named entities

— book reviews, ratings and descriptions

Universal Dependencies

— dependency trees corpus

ua-news

— 150k news article in 5 categories

UA-SQuAD

— Ukrainian version of Stanford Question Answering Dataset

Ukrainian Winograd schema challenge (WSC) Dataset

— manually translated

Ukrainian OntoNotes Dataset

— scripts to build large silver dataset for coreference resolution

ВЕСУМ

— POS tag dictionary. Can generate a list of all word forms valid for spelling

a list of positive/negative words

obscene-ukr

— profanity dictionary

Word stress dictionary

— word stress for 2.7M word forms. See

Heteronyms

— words that share the same spelling but have different meaning/pronunciation

Abbreviations

— map abbreviation to expansion

— crowd-sourced prompts and reference outputs. Ukrainian part is ~500 prompts

tree_stem

— stemmer

pymorphy2

+ — POS tagger and lemmatizer

— grammar, style and spell checker

— Python package for tokenization, multi-word-tokenization, lemmatization, POS, dependency parsing, NER

nlp-uk

— Tools for cleaning and normalizing texts, tokenization, lemmatization, POS, disambiguation

NLP-Cube

Python package for tokenization, sentence splitting, multi-word-tokenization, lemmatization, part-of-speech tagging and dependency parsing

— massively multilingual LM, 13B parameters

— mT5 finetuned on wiki and oasst1 for chats in Ukrainian

UAlpaca

— Llama fine-tuned for instruction following on the machine-translated Alpaca dataset

XGLM

— multilingual autoregressive LM, the 4.5B checkpoint includes Ukrainian

uk4b

and - GPT-2 small, medium and large-style models trained on UberText 2.0 wikipedia, news and books

— truncated version of XLM-RoBERTa with only Ukrainian and English embeddings left

Helsinki-NLP / OPUS-MT models

— Ukrainian to/from 25 langaguages

M2M-100

— Ukrainian to/from 100 languages

Uk-En folktale corpus

— small sentence-aligned corpus of fairy tales

— 157 languages, including Ukrainian

Older official fastText trained on Wiki

— 294 languages, including Ukrainian

fastText_multilingual

— 78 languages, aligned to the same vector space

and — trained on UberText 2.0

easy to use with

Flair

— added in 2022

— punctuation and case restoration model based on XLM-RoBERTa-Uk

— another punctation and case restoration model based on bert-base-multilingual-cased

ukrainian-word-stress

— adds word stress

Ukrainian monolingual text, Ukrainian-English parallel text, partially annotated for named entities

Helsinki-NLP/ UkrainianLT

— another collection of links to Ukrainian language tools

egorsmkv / speech-recognition-uk

— speech recognition and text-to-speech models and datasets

UNLP 2024 shared task

— shared task (competition) on fine-tuning large language models (LLMs) for Ukrainian