Original scientific paper

https://doi.org/10.17559/TV-20231214001207

Re-Clustering Documents to Enhance Search Accuracy with Imbalanced Abbreviation Data

Woon-Kyo Lee ; Seoul National University of Science & Technology Graduate school of Public Policy and Information Technology, 232 Gongneung-ro, Nowon-gu, Seoul, Korea
Ja-Hee Kim ; Seoul National University of Science & Technology Graduate school of Public Policy and Information Technology, 232 Gongneung-ro, Nowon-gu, Seoul, Korea *

* Corresponding author.

Abstract

Abbreviation ambiguity poses significant challenges when searching academic literature. This study evaluated the accuracy of clustering algorithms on imbalanced datasets with varying ratios of target groups. A corpus consisting of 1052 papers focused on the study of abbreviations. The "MSA" dataset was clustered using TF-IDF, cosine similarity, and k-means. Clustering performance declined as the ratios in the target group deviated from balanced thresholds. A re-clustering method was introduced, involving the selective exclusion of non-target clusters. Re-clustering improved accuracy and F1 scores in most scenarios, demonstrating particular stability with higher cluster counts. The re-clustering performance of comparisons was stronger when compared to k-means and self-adaptive methods. The study highlights issues stemming from data imbalance and presents an effective strategy for enhancing abbreviation search efficiency.

Keywords

imbalanced data, K-means algorithm, Re-clustering, word sense disambiguation

Hrčak ID:

321905

URI

https://hrcak.srce.hr/321905

Publication date:

31.10.2024.

Visits: 7 *