Introduction
Pre- and postanalytical errors represent most errors in the total testing cycle leading to increased risk of misdiagnosis and treatment of patients (1-3). Identification, phlebotomy, collection tube and sample preservation and transport related errors are common sources of error, particularly when the specimen collection staff are not laboratory-controlled (4). Postanalytical errors mainly involve delayed or incomplete delivery of results to referring doctors. Despite this, few benchmarking EQA programs are available for medical laboratories to assess these error rates objectively (5,6). Indicators of the extra-analytical phases of the Total Testing Process (TTP) have been developed in several countries, such as Australia and New Zealand, The US Brazil, and Spain/Catalonia, and other surveys and programs have been promoted in the UK, China and Croatia (7-14). In 2008, the IFCC launched a Working Group named “Laboratory Errors and Patient Safety” (WG: LEPS) to identify Quality Indicators (QIs) and related quality specifications that (i) produce benchmarks from comparing laboratories, (ii) promote error reduction and (iii) increase patient safety. The IFCC has developed QIs, which laboratories in several countries have evaluated, and the WG: LEPS has reported preliminary results (15-17).
To reduce pre- and post-analytical error, laboratories need to focus on these phases of the total testing cycle with relevant QIs and benchmarks from external sources (18-20). As with any form of benchmarking, laboratories must collect and track their QIs and then compare them against the relevant peer group. Initially, this ensures they collect the same data as their peers, then compare their performance. If there are significant differences after data collection error has been eliminated, then a review of the process is warranted. To ensure there is confidence in this comparison, a sufficiently large peer group collecting the same QIs needs to be available, and the data collection needs to be regular enough to detect trends (21).
But collecting QI data in pathology can be tedious. Benchmarking processes are far more difficult than standard analytical EQA because measurement may be complex and not standardised, data collection may not be easy, and targets are unavailable. As with many benchmarking exercises, there have been problems with the definitions and robustness of the QIs. Key Incident Monitoring and Management EQA Program has tried to ensure that participants collect and submit the same data for each of the QIs. While progress has been made in this direction, the data is still not always homogeneous. One of the aims of the KIMMS program is to produce a list of key incidents that should be tracked by an LIS. Also, changing a process is more difficult than identifying an error in an assay system.
The Royal College of Pathologists of Australasia Quality Assurance Programs (RCPAQAP) Key Incident Monitoring and Management EQA Program (KIMMS) started in 2011 and aims to monitor the pre- and post-analytical phase of the laboratory quality system (7). Key Incident Monitoring and Management EQA Program measures risks from incidents recorded by participant laboratories and compares these to those of peer organisations. Incidents and risks are related but not equal (22). The program assesses risk based on the consequences to the patient, the probability that the event will occur, and how easy it is to detect the incident. A reduction in probability or an increase in detectability can reduce the risk. A two-phase risk calculation is used in KIMMS, where a harm factor equals consequences multiplied by likelihood (21). The risk score equals harm times detectability (23).
As an example of this risk score process, we present this case where the overall performance summary highlights any “incident” that recorded a rate greater than 80% of all participants. This is presented in a table, which includes columns for the consequence, probability and detectability rating (Tables 1-3{ label needed for table-wrap[@id='t2'] }{ label needed for table-wrap[@id='t3'] }). The KIMMS Advisory Committee has pre-determined the consequence rating, so this field is pre-populated.
{ label needed for table-wrap[@id='t2'] }
{ label needed for table-wrap[@id='t3'] }
| Scale | Name | Example Definition |
|---|---|---|
| 1 | Rare | < 1/year |
| 2 | Unlikely | 1 per year |
| 3 | Occasional | 1 per month |
| 4 | Likely | 1 per week |
| 5 | Frequent | 1 per day or more |
Results review
It is recommended that participants review their results by applying the probability and detectability rating to determine the risk score (Tables 2-4{ label needed for table-wrap[@id='t3'] }{ label needed for table-wrap[@id='t4'] }). The risk score is calculated by multiplying the three ratings (consequence rating x probability rating x detectability rating). The risk score can be used as a guide to organisational risk; from that, a decision is made as to the next step(s).
{ label needed for table-wrap[@id='t4'] }
| Scale | Name | Definition |
|---|---|---|
| 1 | Detected | > 95% |
| 2 | Most detected | 75-95% |
| 3 | Half detected | 25-75% |
| 4 | Most not detected | < 5-25% |
| 5 | Not detected | < 5% |
In the above example, the consequence, probability and detectability rating give a risk score of 10. This would be considered low risk. Apart from flagging that the risk is higher than other participants’ inpatient collections, it is unlikely that any further immediate action is required. This incident should be monitored closely to ensure it does not deteriorate.
Materials and methods
The KIMMS program has four surveys a year, collecting data from the previous three months, with preanalytical and postanalytical incident reporting of 35 incident types that have been pre-determined by the RCPAQAP KIMMS Advisory Committee. Data is submitted via the RCPAQAP myQAP portal, where result entry instructions are included. Participants are asked to capture the number of episodes and the number of incidents per quarter of the year.
Data is then analysed using RCPAQAP in-house data analysis software, where each specific incident reported to KIMMS is calculated per 100,000 episodes (number of incidents divided by the number of episodes, multiplied by 100,000), known as the rate. Incident rates are compared to peer groups and flagged red if outside the best performers, which is set at 80% of all results. This summary will focus on 2024 data.
A single request usually covers an episode and may contain one or more samples. It may include more than one request, but the samples are collected simultaneously, i.e., samples received together in a bag with a request form equal one episode.
Participants range from small to large pathology laboratories, where size is determined by the number of episodes per quarter (small: < 100,000, medium: 100,000-500,000, large: > 500,000). This data is also collected for each submission. The participants include a mix of public and private organisations from across Australia and a couple of international sites.
Patient sources
The KIMMS program was renewed in 2023 to allow for further comparison by separating data into categories such as the patient source (Inpatient/Outpatient/Community/Emergency Department (ED)).
The Inpatient patient source refers to pathology testing performed on an admitted hospital patient (public or private hospital). The Outpatient source includes pathology requests and specimens that are collected while a patient attends a clinic run by a hospital. This includes day patients and cancer treatment centres. The Emergency Department patient source refers to pathology testing performed on a patient attending an Emergency Department (ED)/Emergency Room (ER)/Accident and Emergency (A&E)/Casualty ward/department. The community patient source is for when the pathology request is generated anywhere, but the specimens are collected in settings other than the previously mentioned patient sources. This includes doctors’ surgeries, designated collection centres (even within a hospital), nursing homes, home visits, and workplace collections.
The KIMMS program also offers a Mixed patient source option for those organisations that are unable to differentiate their data between patient sources. In the final calculations, these are analysed with a total of all results.
Results
The four 2024 surveys received an average of 111 responses, with 55,329,998 episodes recorded and 1,496,708 incidents identified.
Table 5 summarises the total episodes and incidents reported per survey.Table 6 summarises the key results for the top three incidents from all sites, andTable 7 shows the key results per size of organisation.Tables 8-10{ label needed for table-wrap[@id='t9'] }{ label needed for table-wrap[@id='t10'] } display the average 80th percentile cut-off rates and standard deviations (SDs) per patient source and the Mixed/Total category for the Test Request/Collection incident categories, as well as the Test Registration/Analytical/Postanalytical incident categories from the four 2024 KIMMS surveys.
| Total episodes | Total incidents | |
|---|---|---|
| Survey 1 | 11,930,241 | 274,277 |
| Survey 2 | 13,123,033 | 295,720 |
| Survey 3 | 14,832,081 | 484,514 |
| Survey 4 | 15,444,643 | 442,197 |
{ label needed for table-wrap[@id='t9'] }
{ label needed for table-wrap[@id='t10'] }
Figures 1-3{ label needed for fig[@id='f2'] }{ label needed for fig[@id='f3'] } are bar graphs highlighting the average 80th percentile cut-off rates per patient source and the Mixed/Total category for the Test Request/Collection incident categories, as well as the Test Registration/Analytical/Postanalytical incident categories from the four 2024 KIMMS surveys.
Discussion
The findings from the 2024 program are that the incident “No specimen received” appears to have the highest 80th percentile across the patient sources. The cost of a recollection is significant, both in organisational cost and patient time (24). More importantly, it is an error that can be reduced once identified. Suggested mechanisms to reduce this incident would be computer-generated, or manual checklists of collection tubes required for a given set of requested tests.
The commonest site of error is the ED, with an 80th percentile overall. The primary cause is the non-professional collector and the collection process (4,25-27). The solution could include better training of nursing and medical staff in the ED, though this has had poor long-term effects (25). The issue may be costly enough for the organisation to place phlebotomists in ED (28).
The 80th percentiles are consistent for all incidents throughout the year, meaning the occurrence of incidents is also consistent, validating the data collection process. The 80th percentiles are greater overall for incidents that fall under the Collection incident category, such as “No specimen received”, “Insufficient specimen”, “Specimen clotted” and “Specimen hemolysed”. Sites that perform greater than the 80th percentile have the incidents flagged for review. It is then up to the participants to calculate the risk of the incident to their organisation by referring to our KIMMS Matrix Table (https://rcpaqap.com.au/kimms-matrix-table/), which comprises consequence, probability and detectability. Specific incidences, such as WSIT, may have lower rates reported than the actual values, as the detectability of such incidences is low. This is why in the KIMMS risk matrix table, a higher value for detectability can be assigned to increase the overall risk score and severity of these incidents.
The KIMMS program has been operating since 2011 with several improvements (7,21,23). Significant findings and trends have been separately published (21,29). This experience of participants reporting using standardised definitions of incidents has led to consistent data collections as can be seen from year-on-year results. It has taken some years for this to occur, as there is much variation in what laboratories call episodes and what their capability is to accurately collect these data. The QIs used are relevant to Australian laboratory practice and share commonality with the IFCC set. It is difficult to compare results from different publications as many are one-off surveys rather than a system of regular data collection.
The limitations of this program’s data collection process include some manual data collection processes and likely missing data for some QIs, such as WSIT, because the laboratory is unable to identify these errors (30).
In conclusion, the KIMMS data provide valuable, regular and reproducible benchmarking data for the pre- and post-analytical phases of the total testing cycle. The incidents represent areas where the risk to patients can be reduced.