Skip to the main content

Professional paper

https://doi.org/10.31724/rihjj.46.2.8

Word Embedding Based on Large-Scale Web Corpora as a Powerful Lexicographic Tool

Radovan Garabík ; Ľ. Štúr Institute of Linguistics, Slovak Academy of Sciences


Full text: english pdf 1.013 Kb

page 603-618

downloads: 712

cite


Abstract

The Aranea Project offers a set of comparable corpora for two dozens of (mostly European) languages providing a convenient dataset for nLP applications that require training on large amounts of data. The article presents word embedding models trained on the Aranea corpora and an online interface to query the models and visualize the results. The implementation is aimed towards lexicographic use but can be also useful in other fields of linguistic study since the vector space is a plausible model of semantic space of word meanings. Three different models are available – one for a combination of part of speech and lemma, one for raw word forms, and one based on fastText algorithm uses subword vectors and is not limited to whole or known words in finding their semantic relations. The article is describing the interface and major modes of its functionality; it does not try to perform detailed linguistic analysis of presented examples.

Keywords

corpus; word embedding; vector similarity; semantic similarity; web corpora; visualization

Hrčak ID:

245458

URI

https://hrcak.srce.hr/245458

Publication date:

30.10.2020.

Article data in other languages: croatian

Visits: 2.269 *