Original scientific paper
Quality Control of Epidemiological Lectures Online: Scientific Evaluation of Peer Review
Faina Linkov
Mita Lovalekar
Ronald LaPorte
Abstract
Aim: To examine the feasibility of using peer review for the quality control of online materials.
Methods: We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at κ statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers.
Results: Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (κ) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4).
Conclusions: To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.
Keywords
peer review; inter-rater agreement; Internet; quality control; epidemiology
Hrčak ID:
12937
URI
Publication date:
15.4.2007.
Visits: 1.551 *