awesome-hungarian-nlp

huntoken

👌🚀💯 Hungarian word and sentence splitter

quntoken

👌🚀💯 New Hungarian tokenizer based on quex, huntoken

emMorph (Humor)

💯 Hungarian morphological analyzer based on Humor

emMorphPy

👌💯A wrapper, a lemmatizer and REST API implemented in Python for emMorph (Humor) Hungarian morphological analyzer

🚀💯 is an open source tool and programming library for spell-checking, stemming and morphological analysing of agglutinative, german and other languages

hunmorph-foma

🚀💯 Hungarian morpholical analyzer and generator based on hunmorph

👌🚀💯 is an open-source spell-checker, stemmer and morphological analyzer

lara-hungarian-nlp

👌🚀💯 LARA is a lightweight Python NLP library for ChatBots in Hungarian

👌🚀💯 project aims at providing standardized open source multilingual platform for lemmatisation. ( | )

Simplemma

👌🚀💯 is a simple multilingual lemmatizer for Python

👌🚀💯 Hunpos is an open source reimplementation of TnT, the well known part-of-speech tagger by Thorsten Brants

PurePos

👌🚀 Open source morphological tagger based on HunPos

purepos.py

👌🚀 Python wrapper for PurePos

HunTag

👌🚀 A sequential tagger for NLP using Maximum Entropy Learning and Hidden Markov Models

HunTag3

👌🚀 Improved version of the original HunTag

👌🚀💯 Named Entity Recognition tool for Hungarian and English

DBpedia Spotlight

👌🚀💯 DBpedia Spotlight is a tool for automatically annotating mentions of DBpedia resources in text

emBERT

👌🚀💯 is an emtsv module for pre-trained Transfomer-based models. It provides tagging models based on Huggingface's transformers package

👌💯 A toolkit for the basic linguistic processing of Hungarian

magyarlanc_spark

👌💯 Spark wrapper for magyarlanc

eszterland

👌💯 Clojurized access to magyarlanc

HuSpaCy

👌🚀💯 Industrial-strength Hungarian Natural Language Processing

huNLP

👌💯 An experimental unified Java and REST API for magyarlanc and szegedNER

hunlp-GATE

💯 GATE plugin containing Hungarian NLP tools as GATE processing resources

Trendminer Hungarian Processing Pipeline

🚀 Hungarian NLP pipeline for social media text analysis (TrendMiner project)

🚀💯 Neural Models of Syntax

👌🚀💯 is a trainable pipeline for tokenization, tagging, lemmatization and dependency parsing of CoNLL-U files

👌🚀💯 is a natural language pipeline that supports massive multilingual applications

emtsv

👌💯 is a text processing system with inter-module communication via tsv + REST API

Stanza

👌🚀💯 is a Python NLP Library for Many Human Languages

spaCy StanfordNLP

👌🚀💯 wraps the StanfordNLP library, so you can use Stanford's models as a spaCy pipeline

trankit

👌🚀💯 A Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

🚀💯 A rule based Hungarian syntactical analyzer

HunParse

🚀💯 An NLTK-based parser using KR-style morphological annotation

Anagramma Parser

A parser based on psycholinguistics principles

benepar

👌🚀💯 A high-accuracy parser with models for 11 languages, implemented in Python. Based on Constituency Parsing with a Self-Attentive Encoder from ACL 2018

SentimentAnalysisHUN

👌🚀💯 is an open-source sentiment analysis tool for Hungarian language, written in Python

hun-date-parser

👌🚀💯 A tool for extracting datetime intervals from Hungarian sentences and turning datetime objects into Hungarian text

SZTAKI HunSum-1 models 👌🚀💯 , , ,

emLam

👌🚀💯 Preprocessing scripts for Hungarian Language Modeling

pywnxml

👌🚀💯 Python3 API for WordNet XML (Hungarian WordNet / BalkaNet / VisDic format)

Hun-appointment-chatbot

👌🚀💯 A simple Hungarian chatbot for booking an appointment using the Rasa framework

neural-punctuator

👌🚀💯 Automatic punctuation restoration with BERT models for English and Hungarian

hunaccent

👌🚀💯 Small Footprint Diacritic Restoration for Hungarian

Diacritics_restoration

🚀💯 Lightweight Diacritics Restoration with Dilated Convolutional Neural Networks

NYTK MT

👌🚀💯 NYTK Machine translation models

syntax-augmentation-nmt

🚀💯 Syntax-based data augmentation for Hungarian-English machine translation

anonymizer_hu

🚀💯 The Hungarian anonymization tool for CURLICAT

pre-trained word vectors for 90 languages, trained on Wikipedia using fastText

pre-trained word vectors for 157 languages, trained on Wikipedia and the Common Crawl using fastText's CBOW model

FastText_multilingual

Multilingual word vectors in 78 languages

polgyglot embeddings on Wikipedia

wordvectors

Pre-trained word2vec and fasttext word vectors on wikipedia of 30+ languages

A word2vec word embedding trained on the concatenation of the Hungarian Webcorpus and the Hungarian National Corpus in 600 dimensions with a cut-off of 10 words

Word embeddings (word2vec & fasttext) for Hungarian trained on 4.3 billion tokens

Hungarian analogical questions following Mikolov et al

Conceptnet Numberbatch

Conceptnet numbermatch multi- and cross-lingual semantic word embeddings

pretrained Subword Embeddings, downloadable in many formats

300d Floret embeddings trained on the Hungarian Webcorpus 2.0

100d Floret embeddings trained on the Hungarian Webcorpus 2.0

ELMo Representations

Deep contextualized word representation trained for many languages

Hungarian BERT base models trained on Webcorpus 2.0 and the Hungarian Wikipedia

Pretrained transformer models provided by HILANCO

is a Hungarian BERT large model based on MegatronBERT

is a Hungarian GPT-2 model

is a Hungarian GPT-NeoX model (6.7 billion parameter)

With over 1.48 billion words unfiltered (589 million words fully filtered), this is by far the largest Hungarian language corpus, and unlike the Hungarian National Corpus (125 million words), it is available in its entirety under a permissive Open Content license

The new version of the Hungarian Webcorpus was built from Common Crawl and includes a little over 9 billion words

is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture. (2339 million unique words)

A Language Modeling Benchmark Corpus for Hungarian, similar to the One Billion Word corpus (Chelba, 2014) for English

contains randomly selected sentences in the language of the corpus and are available in sizes from 10,000 sentences up to 1 million sentences. The sources are either newspaper texts or texts randomly collected from the web

Automatically created multilingual web corpus

Monolingual Datasets from Web Crawl Data

Automatic segmentation, tokenization and morphological and syntactic annotations of raw texts in 45 languages, generated by , together with word embeddings of dimension 100 computed from lowercased texts by

OpinHuBank is a human-annotated corpus to aid the research of opinion mining and sentiment analysis in Hungarian

HunEmPoli

corpus was built using pre-agenda speeches of the Hungarian National Assembly (2014-2018) and consists 764008 tokens/36475 sentences. Aspect level emotion annotation, with 39840 identified emotions, in addition, marked the keywords that evoked the emotion

This database is the first one dedicated to Opinion Mining in Hungarian. The data for further processing were gathered from the posts of the forum topic of the Hungarian government portal dealing with the referendum about dual citizenship

is a deeply annotated Hungarian sentiment corpus. It is composed of Hungarian opinion texts written about different types of products, published on the homepage [

The Szeged Treebank is the largest fully manually annotated treebank of the Hungarian language

The Szeged Dependency Treebank is a dependency-tree format version of the Szeged Treebank

The Named Entity Corpus for Hungarian is a subcorpus of the Szeged Treebank, which contains full syntactic annotations done manually by linguist experts

KorKor Pilotcorpus

is a gold standard corpus consisting of multiple layers such as dependency parse and coreference annotations

NerKor

is a gold standard named entity annotated corpus containing 1 million tokens

NerKor 1.41e

A 1M+-token Hungarian named entity dataset with ~30 entity types derived from NYTK-NerKor

a silver standard corpus for Hungarian Named Entity Recognition

contains 28M sentences from the MNSZ1 corpus annotated with shallow syntactic analysis

PrevCons

is a database of 21K hapaxes of verbs with verbal prefixes

containing 39 suitable word form samples for the purpose of word sense disambiguation

is a learners' corpus of Hungarian containing written data from 35 students majoring in Hungarian studies at the University of Zagreb, Croatia. Texts were morphologically and syntactically analyzed by the magyarlanc tool

HuLU

Hungarian Language Understanding Benchmark Kit

HuCOLA

Hungarian Corpus of Linguistic Acceptability

HuCoPA

Hungarian Choice of Plausible Alternatives Corpus

HuSST

Hungarian version of the Sentiment Treebank

HuWNLI

Anaphora resolution datasets for Hungarian as an inference task

HuWS

is the Hungarian set of the Winograd schemas

Hungarian Corpus for Reading Comprehension with Commonsense Reasoning

ELTE Poetry Corpus

is a database of complete poems of 50 Hungarian canonical poets together with the sound devices of the poems and the grammatical features of words in XML format

ELTE Novel Corpus

is a database of 400 Hungarian novels (with the annotation of structural units and the grammatical features of words in TEI XML format)

ELTE Drama Corpus

is a database of 58 dramas (with the annotation of structural units and the grammatical features of words in TEI XML format)

is a dataset containing over 1.1M unique news articles with lead and other metadata

HAPP

is the Hungarian translation of the

The Hunglish Corpus is a free sentence-aligned Hungarian-English parallel corpus of about 120 million words in 4 million sentence pairs

The English-Hungarian parallel corpus contains texts selected on the basis of grammatical and translational criteria

A Hungarian-Russian Parallel corpus comprises approximately 800 thousand words

Automatic segmentation, tokenization and morphological and syntactic annotations of raw texts from the Common Crawl

CSS10

A Collection of Single Speaker Speech Datasets for 10 Languages including Hungarian

sentence aligned TED talks including Hungarian

is a paraphrase corpus for 73 languages, including Hungarian, extracted from the Tatoeba database

is a dataset of comprehensive accepted translations from English to 5 different languages, including Hungarian

is an automatically extracted database containing millions of paraphrases in 16 different languages, including Hungarian

contains movie subtitles and alignments for 62 languages, including Hungarian

[OPUS Corpus][ is a growing collection of translated texts from the web

MASSIVE dataset

is a parallel dataset of > 1M utterances across 51 languages with annotations for the Natural Language Understanding tasks of intent prediction and slot annotation

PWS

is a parallel collection of the Winograd schemas in seven languages (including Hungarian)

is an open source morphological database of Hungarian, consisting of a lexicon and morphological grammar that are based on well-founded theoretical decisions

huwn

Hungarian Wordnet

The dictionaries were manually created on the basis of Wordnet-Affect lexicons

poltextLAB's sentiment lexicons

Highly accurate sentiment lexicons for analysing news data

4lang

Concept dictionary using Eilenberg machines

lists 500K verb frames extracted from the Mazsola database

Manocska

merges verb frames existing databases

PrevLex

List of phrasel verbs

panmorph

Tagsets and description of Hungarian morphological analysers

hun_ner_checklist

CHECKLIST diagnostic test cases for Hungarian Named Entity Recognition

huwn.rdf

Hungarian WordNet in RDF format for the Linked Open Data cloud

An open, multilingual knowledge graph (with partial Hungarian support)

In the keys, the

Natural-earth-vector

( imported from wikidata labels)

is a gazetteer of places (with )

Conference on Hungarian Computational Linguistics (since 2003)

Official blog of Precognox Inc

EENLP

The broad index of NLP resources for Eastern European languages