Professional paper
https://doi.org/10.31724/rihjj.46.2.8
Word Embedding Based on Large-Scale Web Corpora as a Powerful Lexicographic Tool
Radovan Garabík
; Ľ. Štúr Institute of Linguistics, Slovak Academy of Sciences
Abstract
The Aranea Project offers a set of comparable corpora for two dozens of (mostly European) languages providing a convenient dataset for nLP applications that require training on large amounts of data. The article presents word embedding models trained on the Aranea corpora and an online interface to query the models and visualize the results. The implementation is aimed towards lexicographic use but can be also useful in other fields of linguistic study since the vector space is a plausible model of semantic space of word meanings. Three different models are available – one for a combination of part of speech and lemma, one for raw word forms, and one based on fastText algorithm uses subword vectors and is not limited to whole or known words in finding their semantic relations. The article is describing the interface and major modes of its functionality; it does not try to perform detailed linguistic analysis of presented examples.
Keywords
corpus; word embedding; vector similarity; semantic similarity; web corpora; visualization
Hrčak ID:
245458
URI
Publication date:
30.10.2020.
Visits: 1.551 *