Original scientific paper

MISSING DATA PROBLEMS IN NON-GAUSSIAN PROBABILITY DISTRIBUTIONS

Lovorka Gotal Dmitrović ; University North, University Centre Varazdin, Croatia
Vesna Dušak ; Faculty of Organization and Informatics, University of Zagreb, Varazdin, Croatia
Jasminka Dobša ; Faculty of Organization and Informatics, University of Zagreb, Varazdin, Croatia

Abstract

Abstract
Ecology as a scientific discipline has been developing rapidly and becoming the interdisciplinary science based on Information and Communication Technologies (ICT). Discovering, integrating and analyzing a huge amount of heterogeneous data is crucial in exploring complex ecological issues. Ecoinformatics offers tools and approaches for the management of environmental data which it transforms further into information and knowledge. The development of Information Technologies with the special emphasis on the research methods of gathering and analyzing data, their storage and data access, has significantly enhanced the laboratory methods and their reports. The above, influences the data quality, as well as the research itself. Moreover, it provides a stable base for the development and the replacement of missing data. The improper missing data handling can lead to invalid conclusions. Therefore, it is important to use the adequate methods for handling the missing data. This paper compares The Deleting Rows Method (Listwise Deletion Method) and six single imputation methods, namely: Last Observation Carried Forward (LOCF), Hot-deck Imputation, Group Mean Imputation, Estimated Mean Value Imputation (Regression), Mode Imputation and Median Imputation. For the purposes of this study, the actual, empirical data was collected and used from the non- Gaussian probability distribution of the observed technical system. Mostly, these are asymmetric probability distributions with a tail. Data sets with missing data were created by deleting values with a random number generator. The experiment was repeated three times for each 100%, 95% and 75% sets of the collected data. Experiments have shown that the best imputation data results were provided by Hot-Deck Method, especially when there was a larger number of missing data, which has been confirmed by the Tests of Goodness. The same results, regardless of the set size, were provided by Listwise Deletion Method, which is simpler.

Keywords

missing data; imputation methods; probability distribution; ecoinformatics

Hrčak ID:

173840

URI

https://hrcak.srce.hr/173840

Publication date:

30.12.2016.

Article data in other languages: croatian

Visits: 1.861 *