Skip to the main content

Original scientific paper

https://doi.org/10.31341/jios.44.2.2

A Comparison of Approaches for Measuring the Semantic Similarity of Short Texts Based on Word Embeddings

Karlo Babić orcid id orcid.org/0000-0001-6343-0938 ; Department of Informatics, Center for Artificial Intelligence and Cybersecurity, University of Rijeka, Rijeka, Croatia
Francesco Guerra ; Department of Engineering Enzo Ferrari, University of Modena and Reggio Emilia, Modena, Italy
Sanda Martinčić-Ipšić orcid id orcid.org/0000-0002-1900-5333 ; Department of Informatics, Center for Artificial Intelligence and Cybersecurity, University of Rijeka, Rijeka, Croatia
Ana Meštrović ; Department of Informatics, Center for Artificial Intelligence and Cybersecurity, University of Rijeka, Rijeka, Croatia


Full text: english pdf 951 Kb

page 231-246

downloads: 749

cite


Abstract

Measuring the semantic similarity of texts has a vital role in various tasks from the field of natural language processing. In this paper, we describe a set of experiments we carried out to evaluate and compare the performance of different approaches for measuring the semantic similarity of short texts. We perform a comparison of four models based on word embeddings: two variants of Word2Vec (one based on Word2Vec trained on a specific dataset and the second extending it with embeddings of word senses), FastText, and TF-IDF. Since these models provide word vectors, we experiment with various methods that calculate the semantic similarity of short texts based on word vectors. More precisely, for each of these models, we test five methods for aggregating word embeddings into text embedding. We introduced three methods by making variations of two commonly used similarity measures. One method is an extension of the cosine similarity based on centroids, and the other two methods are variations of the Okapi BM25 function. We evaluate all approaches on the two publicly available datasets: SICK and Lee in terms of the Pearson and Spearman correlation. The results indicate that extended methods perform better from the original in most of the cases.

Keywords

semantic similarity; short texts similarity; word embedding; Word2Vec; FastText; TF-IDF

Hrčak ID:

247489

URI

https://hrcak.srce.hr/247489

Publication date:

9.12.2020.

Visits: 2.062 *