A research project ranking method based on independent reviews by using the principle of the distance to the perfectly assessed project
AbstractThe paper discusses the problem of ranking research projects based on the assessment obtained from $n\geq 1$ independent blinded reviewers. Each reviewer assesses several project features, and the total score is defined as the weighted arithmetic mean, where the weights of features are determined according to the well-known AHP method. In this way, it is possible to identify each project by a point in $n$-dimensional space.
The ranking is performed on the basis of the distance of each project to the perfectly assessed project. Thereby the application of different metric functions is analyzed. We believe it is inappropriate to use a larger number of decimal places if two projects are almost equidistant (according to some distance function) to the perfectly assessed project. In that case, it would be more appropriate to give priority to the project that has received more uniform ratings. This can be achieved by combining different distance functions.
The method is illustrated by several simple examples and applied by ranking internal research projects at Josip Juraj Strossmayer University of Osijek, Croatia.
CRORR Journal Regular Issue
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).