hrcak mascot   Srce   HID

Izvorni znanstveni članak
https://doi.org/10.11613/BM.2018.020711

Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization

Rania El Sharkawy ; Medical Research Institute, Alexandria University, Alexandria, Egypt
Sten Westgard ; Westgard QC Inc., Madison, USA
Ahmed M Awad ; Medical Research Institute, Alexandria University, Alexandria, Egypt
AbdelKarem Omneya I Ahmed ; Medical Research Institute, Alexandria University, Alexandria, Egypt
El Hadidi Iman ; Ain Shams University, Cairo, Egypt
Ahmed Gaballah ; Zagazig University, Zagazig, Egypt
Eman Shaheen ; Private laboratory, Helwan University, Alexandria, Egypt

Puni tekst: engleski, pdf (337 KB) str. 233-245 preuzimanja: 322* citiraj
APA 6th Edition
El Sharkawy, R., Westgard, S., Awad, A.M., Omneya I Ahmed, A., Iman, E.H., Gaballah, A. i Shaheen, E. (2018). Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization. Biochemia Medica, 28 (2), 233-245. https://doi.org/10.11613/BM.2018.020711
MLA 8th Edition
El Sharkawy, Rania, et al. "Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization." Biochemia Medica, vol. 28, br. 2, 2018, str. 233-245. https://doi.org/10.11613/BM.2018.020711. Citirano 17.01.2021.
Chicago 17th Edition
El Sharkawy, Rania, Sten Westgard, Ahmed M Awad, AbdelKarem Omneya I Ahmed, El Hadidi Iman, Ahmed Gaballah i Eman Shaheen. "Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization." Biochemia Medica 28, br. 2 (2018): 233-245. https://doi.org/10.11613/BM.2018.020711
Harvard
El Sharkawy, R., et al. (2018). 'Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization', Biochemia Medica, 28(2), str. 233-245. https://doi.org/10.11613/BM.2018.020711
Vancouver
El Sharkawy R, Westgard S, Awad AM, Omneya I Ahmed A, Iman EH, Gaballah A i sur. Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization. Biochemia Medica [Internet]. 2018 [pristupljeno 17.01.2021.];28(2):233-245. https://doi.org/10.11613/BM.2018.020711
IEEE
R. El Sharkawy, et al., "Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonization", Biochemia Medica, vol.28, br. 2, str. 233-245, 2018. [Online]. https://doi.org/10.11613/BM.2018.020711

Rad u XML formatu

Sažetak
Introduction: Analytical quality is an essential requirement for best practice in any medical laboratory. Lack of a harmonized approach for sigma
calculation is considered an obstacle in the objective comparability of analytical performance among laboratories adopting sigma metrics. It is
urgently needed that all laboratory professionals interested in the analytical quality to work hard towards harmonization protocol for sigma calculation
in order to properly select their analytical goals. This study aims at harmonization of Sigma metrics calculation in four accredited Egyptian
laboratories.
Materials and methods: This observational cross sectional study compared the sigma levels for certain biochemical parameters in the four participating
laboratories.
Results: Coefficient of variation (CV) and bias were determined for some biochemical analytes, data assayed by different automated analysers in
the four different accredited laboratories. The sigma level for the four medical laboratories was calculated for each biomedical parameter with changed
sigma level after total allowable error (Tea) unification among participating laboratories.
Conclusion: Each laboratory should select the TEa goal based on clear standardized criteria of selection without any subjective preferences as either
under or over estimation of Sigma metrics will affect the patient centred care negatively if laboratories use quality control procedures wrongly based
on incorrect Sigma metrics calculation with subsequent misleading medical decisions.

Ključne riječi
sigma; bias; coefficient of variation; harmonization; total allowable error

Hrčak ID: 201531

URI
https://hrcak.srce.hr/201531

▼ Article Information



Introduction

Analytical quality is an essential requirement for best practice in any medical laboratory. Patient-centred care, the main target of medical laboratories, depends on the key concepts of internal quality control (IQC) which was established by Levey and Jennings followed by the interpretative rules published by Westgard and colleagues and external quality assurance (EQA) programs, which established in the late nineties as a complementary pillar to IQC, provide a tool of peer comparison (1-5). Consequently, the minimization of analytical imprecision reflected as random errors via proper IQC plans and minimization of analytical bias seen as systematic errors through EQA programs are considered fundamental tools for any quality management system in laboratory medicine (6).

Quality decision specifications based on laboratory performance characteristics (bias and imprecision) were recommended many years ago as a mechanism to support quality in the medical laboratories. It is now necessary to determine how the performance of a measurement procedure relates to the medical requirements for interpreting results in order to determine the frequency to measure and evaluate quality control (QC) samples and results (7, 8). Sigma metrics (SM) have been used to assess quality in a quantitative manner. There are two different methodologies for assessing process performance in terms of Sigma metrics. The first method depends on counting the defects or errors which are expressed as defects per million (DPM); the DPM are subsequently converted to a SM scale of 0 to 6, with 6 being world class (3.4 defects per million) and 3 being the minimum level of performance (about 66,800 defects per million) (9).

The second method depends mainly on measuring the variation of the measurement process to predict its performance and evaluate how well a measurement procedure performs using the two pillars of performance characteristics (bias and precision) and the total allowable error (TEa) (10). The goal is to seek for 6-sigma (world class) quality, with the common minimum level of acceptable quality broadly considered to be 3-sigma (11).

The use of SM offers many advantages to laboratories as it helps in determining their IQC frequency; thus avoiding repeated IQC testing during periods of stable performance, consequently minimizing unnecessary costs and human-hour wastage.
In addition, it facilitates the comparison of the same assay performance across multiple systems (9, 12, 13).

To the best of our knowledge, this study represents the first study that tackles the variation in Sigma metrics calculation among accredited laboratories in Egypt. Since most of the laboratories are calculating the sigma and comparing the results while using different methods of calculating the sigma elements (bias and CV) as well as selecting suitable TEa, this variation might affect the comparability of analytical performance though they are all accredited. Also, it sheds the light over some key points in Sigma metrics calculation that allows laboratorians to make use of such valuable tool for assessment of method performance in a more objective manner.

This study aims at harmonization of Sigma metrics calculation following a standardized protocol in order to improve its utility for evaluation of the performance among accredited laboratories which is consequently reflected on the patient centred care in addition to facilitating comparability of sigma values in a more objective methodology.

Materials and methods

Study design

The current study is an observational cross sectional study which was conducted after approval of the Research ethical committees in each university and the whole study design was approved by the Medical Research Institute, ethics committee. The presented data are from four Egyptian International Organization for standardization (ISO) 15189:2012 accredited medical laboratories: Chemical Pathology department Medical Research Institute, Alexandria University (MRI laboratory) a hospital laboratory in Alexandria governorate, Zagazig University hospital laboratory located in Zagazig governorate, Ain Shams University hospital laboratory located in Cairo governorate and a private laboratory in Alexandria governorate.

Methods

Coefficient of variation (CV, from IQC records) and bias (from proficiency testing data) were determined for some biochemical analytes, data assayed by different automated analysers in the four different accredited laboratories then Sigma metrics calculation was performed.

Estimated parameters were glucose (Glc), urea, creatinine (CREA), uric acid (UA), cholesterol (CHOL), triglycerides (Tg), albumin (Alb) bilirubin, direct (BD), bilirubin, total (BT), total protein (TP), calcium (Ca), inorganic phosphates (Phos), magnesium (Mg) and potassium (K). Moreover, the following enzyme activities were measured: alanine aminotransferase (ALT), alkaline phosphatase (ALP), lactate dehydrogenase (LD) and gamma-glutamyltransferase (GGT). As well as two immunoassay parameters were evaluated: alpha fetoprotein (AFP) and carcinoembryonic antigen (CEA). Some parameters were not calculated by the four laboratories and presented in the current study to show the effect of different TEa on sigma calculation to emphasize on the idea of need for harmonization of the current sigma calculation.

The instruments used by the four laboratories: Olympus AU 400 (Beckman Coulter International SA, Nyon, Switzerland) was used in MRI laboratory where most of biochemical reagents were dedicated Beckman Coulter except bilirubin (Spectra, Cairo, Egypt), and creatinine (Randox, Antrim, United Kingdom). Immunoassay parameters were assayed on Immulite 1000 (Siemens Healthineers GmbH, Erlangen, Germany), while AVL 9180 (Roche Diagnostics GmbH, Mannheim, Germany) was used to assay the potassium. In Zagazig University hospital laboratory Cobas 8000 modular system (Roche Diagnostics GmbH, Mannheim, Germany) was used to assay both biochemical and immunoassay parameters while AVL 9180 (Roche Diagnostics GmbH, Mannheim, German) was used to assay the potassium. As for Ain Shams University hospital laboratory; Olympus AU 480 with Beckman Coulter dedicated reagents (Beckman Coulter International SA, Nyon, Switzerland) was used and AVL 9180 (Roche Diagnostics GmbH, Mannheim, Germany) was used to assay the potassium. The data collected from Ain Shams University hospital laboratory lacked of the immunoassay parameters. Finally the private laboratory used Cobas c501 (Roche Diagnostics GmbH, Mannheim, Germany) to assay biochemical parameters.

Internal QC data was extracted from the analysers records from January till May 2016 (130 QC run during 130 working days / one run per day). Control materials were run before each analytical run. Each laboratory has its customized internal quality control and calibrations protocol which was done according to each laboratory internal quality control policies and procedure. Each laboratory selected the TEa according to the current analytical laboratory performance.

Internal quality control (IQC) data (same lot for each laboratory and level 1 QC values determined by manufacturers) were used by the four laboratories to determine each parameter CV after exclusion of outliers (QC observations that violate 13S rule). External quality assurance (EQA) data were used by the four laboratories to determine bias for each analyte. The results of EQA samples for at least 3 months were included, the EQA programs used by the four laboratories were not accuracy based and the mean of comparator group is considered as consensus group peer data. The mean of comparators selected according to each laboratory method and instrument, so there were no real true values used in the current study by any of the participating laboratories to determine the bias of the studied biochemical parameters.

The participating laboratories assayed the BioRad monthly program as external quality assessment scheme which consisted of twelve monthly samples in each cycle. According to the manufacturer, the total number of samples for the entire cycle was provided at the same time. All submitted results for each analyte are grouped according to comparators (peer, method, and mode/all results) then an ISO 13528 robust statistical analysis was performed (14).

The approach used in the current study to calculate the Sigma metrics relied on method performance measurement. For laboratory measurements, the Sigma metrics is calculated by the following formula (8): Sigma metrics = (TEa – bias observed) / CV (coefficient of variation) observed. The studied parameters were sorted into 6 categories; world class performance (SM = 6 or more), excellent performance (SM = 5-6), good performance (SM = 4-5), marginal performance (SM = 3-4), poor performance (SM = 2-3) and unacceptable performance (SM is less than 2).

The CV is estimated from the QC data as previously described. It is critically important that the estimate of CV is done using QC data that represent all or most components of variability that occur over an extended time period. A CV that represents stable measurement performance can usually be estimated from the cumulative standard deviation (SD) over a 6 to 12-month period for a single lot of QC material (8). It is noteworthy to state that there was no unified protocol used in the current study to estimate either CV or mean % bias of the studied parameters and each laboratory calculate these performance characteristics according to each laboratory policy.

Two Sigma metrics were calculated for each parameter using 2 different CVs that were obtained from 2 levels IQC data.

The harmonization protocol (Annex 1) is a novel protocol that was suggested by the working group of the current study. It aims at checking most of the key points that are considered as potential sources of variability in Sigma metrics calculation.

Data presentation

The formula CV = (Standard deviation / mean) x 100 has been used for the calculation of coefficient of variation. Bias (%) calculation of single PT measurement using Bio-Rad EQAS programs was calculated using [(mean of all laboratories using same instrument/method – laboratory mean) / mean of all laboratories using same instrument and method] x 100. Mean bias (%) was calculated through the sum of all % bias of PT values of specific parameter / number of PT values. The Sigma metrics was calculated as (TEa – Bias observed / CV observed).

Results

Table 1 shows the performance characteristics of the parameters from Medical Research Institute, Alexandria University, Chemical Pathology Laboratory; Sigma metrics were calculated using the total allowable errors from the different sources as shown. Among the assayed parameters total bilirubin had the highest sigma (10.5) while magnesium had the lowest sigma value (- 0.7).

Table 1 Chemical Pathology department, Medical Research Institute, Alexandria University sigma metrics calculation
ParametersTEa sourceTEaBias (%) CV (%)Sigma
AFPBV minimal32.809.36.73.5
ALTBV desirable27.486.73.46.1
AlbBV desirable4.070.21.92.0
ALPBV desirable12.042.36.91.4
BDBV desirable44.505.67.55.2
BTBV desirable26.940.02.510.5
CalciumBV minimal3.822.02.30.8
CEARiliBÄK24.001.311.42.0
CholesterolBV desirable9.016.12.61.1
CreatinineBV desirable8.875.57.50.4
GlucoseBV desirable6.961.11.44.2
LDBV desirable11.350.54.72.3
MagnesiumBV minimal7.2010.04.3- 0.7
PhosphorousBV desirable10.115.13.51.2
PotassiumBV desirable5.612.15.10.7
Total proteinBV desirable3.630.23.31.0
TriglyceridesBV desirable25.991.53.17.9
Uric acidBV desirable11.970.71.95.9
UreaBV desirable15.551.91.97.2
GGTBV desirable22.111.11.01.9
TEa – total allowable error. CV – coefficient of variation. BV – biological variation. RiliBÄK - guidelines of the German medical association for the quality assurance of laboratory medical examinations. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Table 2 shows the performance characteristics of the parameters assayed in Zagazig University Hospital laboratory. Sigma metrics were calculated using the total allowable errors from the different sources as shown in the table. Triglycerides and direct bilirubin had the highest sigma (10.7) while calcium had the lowest sigma value (0.7) among the assayed parameters.

Table 2 Zagazig University Hospital laboratory Sigma metrics calculation
ParametersTEa sourceTEaBias (%)CV (%)Sigma
AFPBV desirable21.874.25.83.0
ALTBV desirable27.488.74.64.1
AlbCLIA10.000.62.04.6
ALPCLIA30.002.75.05.4
BDBV desirable44.502.73.910.7
BTBV desirable26.946.43.36.2
CalciumBV Minimal3.822.31.90.7
CholesterolBV desirable9.013.21.63.3
CreatinineBV desirable8.871.93.81.8
GlucoseCLIA10.001.72.13.9
LDCLIA20.001.93.45.3
MagnesiumBV Minimal7.210.54.11.6
PhosphorousBV Minimal15.161.34.03.4
PotassiumBV desirable5.611.12.41.8
Total proteinCLIA10.001.71.94.3
TriglyceridesBV desirable25.991.42.310.7
Uric acidBV desirable11.971.02.44.6
UreaBV desirable15.550.04.03.9
GGTBV desirable22.112.33.26.2
TEa – total allowable error. CV – coefficient of variation. BV – biological variation. CLIA - Clinical Laboratory Improvement Amendments. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Table 3 shows the performance characteristics of the parameters from Ain Shams University hospital laboratory; Sigma metrics were calculated using the total allowable errors from the different sources as shown in the table. Among the assayed parameters glucose had the highest sigma (4.9) while albumin had the lowest sigma value (0.4).

Table 3 Ain Shams University hospital laboratory Sigma metrics calculation
ParametersTEa sourceTEaBias (%)CV (%)Sigma
ALTCLIA20.005.96.72.1
AlbCLIA10.07.27.20.4
ALPCLIA30.003.98.03.3
BDBV desirable44.508.99.43.8
BTCLIA20.002.16.62.7
CalciumCLIA16.530.56.02.7
CholesterolCLIA10.001.76.31.3
CreatinineCLIA15.000.67.91.8
GlucoseCLIA10.001.11.84.9
MagnesiumCLIA25.008.08.42.0
PotassiumCLIA8.332.18.00.8
Total proteinCLIA10.001.46.91.2
Uric acidCLIA17.003.04.63.0
UreaCLIA9.006.36.00.5
GGTCLIA26.904.610.42.1
TEa – total allowable error. CV – coefficient of variation. BV – biological variation. CLIA - Clinical Laboratory Improvement Amendments. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Table 4 shows the performance characteristics of the parameters performed in a private laboratory in Alexandria governorate. Sigma metrics were calculated using the total allowable errors from the different sources as shown in the table. Among the assayed parameters GGT had the highest sigma (12.8) while calcium had the lowest sigma value (1.0).

Table 4 The private laboratory Sigma metrics calculation
ParametersTEa sourceTEaBias (%)CV (%)Sigma
ALTCLIA20.001.82.96.2
AlbCLIA10.000.32.04.7
ALPCLIA30.001.96.44.4
BDBV desirable44.500.86.46.8
BTCLIA20.002.13.55.1
CalciumBV desirable2.551.21.41.0
CholesterolCLIA10.000.01.66.3
CreatinineCLIA15.001.34.23.3
GlucoseCLIA10.002.91.83.9
LDCLIA20.003.13.45.0
MagnesiumCLIA25.001.34.25.6
PhosphorousBV desirable10.102.81.84.1
Total ProteinCLIA10.002.11.74.6
TriglyceridesCLIA25.000.02.012.6
Uric AcidCLIA17.002.02.46.1
UreaCLIA9.003.12.62.3
GGTBV desirable22.102.91.512.8
TEa – total allowable error. CV – coefficient of variation. BV – biological variation. CLIA - Clinical Laboratory Improvement Amendments. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Table 5 shows comparison between sigma levels after unifying the TEa source for all four laboratories.

Table 5 Comparison between different sigma levels after unifying the TEa source
ParametersTEa sourceTEaSIGMA MRISIGMA ZagazigSigma
AIN SHAMS
Sigma private lab
AFPCLIA31.873.44.8----
ALTCLIA20.003.92.42.16.2
AlbCLIA10.005.14.70.44.7
ALPCLIA30.004.05.53.34.4
BDRCPA20.001.94.41.23.0
BTCLIA20.007.84.12.75.1
CalciumCLIA12.204.55.22.07.8
CEARCPA12.000.9------
CholesterolCLIA10.001.54.31.36.3
CreatinineCLIA15.001.33.41.83.3
GlucoseCLIA10.006.33.94.93.9
LDCLIA20.004.25.3--5.0
MagnesiumCLIA25.003.56.02.05.6
PhosphorousRCPA10.151.22.2--4.1
Total proteinCLIA10.003.04.41.24.6
TriglyceridesCLIA25.007.610.3--12.6
Uric acidCLIA17.008.66.73.06.1
UreaRCPA21.4410.35.32.57.1
GGTRCPA12.000.93.00.76.0
Sigma re-calculated for the four participating laboratories using the same source of TEa. TEa – total allowable error. MRI – medical research institute. CLIA - Clinical Laboratory Improvement Amendments. RCPA - Royal College of Pathologists of Australasia.AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Tables 6-8{ label needed for table-wrap[@id='t7'] }{ label needed for table-wrap[@id='t8'] } show comparisons of values for bias, CV and sigma between all four laboratories.

Table 6 Comparison between bias (%) in different laboratories
ParametersMedical research instituteZagazig UniversityAin Shams UniversityThe private laboratory
AFP9.34.2----
ALT6.78.75.91.8
ALb0.20.67.20.3
ALp2.32.73.91.9
BD5.62.78.90.8
BT0.06.42.12.1
Calcium2.04.20.51.2
CEA1.3------
Cholesterol6.13.21.70.0
Creatinine5.51.90.61.3
Glucose1.11.71.12.9
LD0.51.9--3.1
Magnesium10.10.58.01.3
Phosphorous5.91.3--2.8
Potassium2.11.12.1--
Total Protein0.31.71.42.2
Triglycerides1.61.4--0.0
Uric Acid0.71.03.02.0
Urea1.90.06.33.1
GGT1.62.34.62.9
AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.
{ label needed for table-wrap[@id='t7'] }
Table 7 Comparison between CV (%) in different laboratories
ParametersMedical Research InstituteZagazig UniversityAin Shams UniversityThe private laboratory
AFP6.75.8----
ALT3.44.66.72.9
Alb1.92.07.22.0
ALP6.95.08.06.4
BD7.53.99.46.4
BT2.63.36.63.5
Calcium2.31.96.01.4
CEA11.4------
Cholesterol2.61.66.31.6
Creatinine7.53.87.94.2
Glucose1.42.11.81.8
LD4.73.4--3.4
Magnesium4.34.18.44.2
Phosphorous3.54.0--1.8
Potassium5.12.48.0--
Total protein3.31.96.91.7
Triglycerides3.12.3--2.0
Uric acid1.92.44.62.5
Urea1.94.06.02.6
GGT11.03.210.41.5
CV – coefficient of variation. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.
{ label needed for table-wrap[@id='t8'] }
Table 8 Comparison between sigma in different laboratories
ParametersMedical Research InstituteZagazig UniversityAin Shams UniversityThe private laboratory
AFP3.53.0----
ALT6.14.12.16.2
Alb2.04.60.44.7
ALP1.45.43.34.4
BD5.210.73.86.8
BT10.56.22.75.1
Calcium0.80.72.71.0
CEA2.0------
Cholesterol1.13.31.36.3
Creatinine0.41.81.83.3
Glucose4.23.94.93.9
LDH2.35.3--5.0
Magnesium- 0.71.62.05.6
Phosphorous1.23.4--4.1
Potassium0.71.80.8--
Total protein1.04.31.24.6
Triglycerides7.910.7--12.6
Uric acid5.94.63.06.1
Urea7.23.90.52.3
GGT1.96.22.112.8
AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. BD – billirubin, direct. BT- bilirubin, total. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.

Figures 1-4{ label needed for fig[@id='f2'] }{ label needed for fig[@id='f3'] }{ label needed for fig[@id='f4'] } demonstrate the precision and accuracy of the studied parameters for the four participating laboratories using the method decision chart while Figure 5 shows different sigma metrics in the four laboratories using the same total allowable error.

Figure 1 Method decision chart for Medical Research Institute, Alexandria University. D.Bili - direct bilirubin. T.Bili - total bilirubin. TP - total protein. ALT - alanine aminotransferase. Alk Phos - alkaline phosphatase. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase. CEA - carcinoembryonic antigen. AFP - alpha fetoprotein.
bm-28-2-020711-f1
{ label needed for fig[@id='f2'] }
Figure 2 Method decision chart of Zagazig University Hospital laboratory. D.Bili - direct bilirubin. T.Bili - total bilirubin. ALT - alanine aminotransferase. Alk Phos - alkaline phosphatase. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase. AFP - alpha fetoprotein. Trig - triglycerides. Chol - total cholesterol.
bm-28-2-020711-f2
{ label needed for fig[@id='f3'] }
Figure 3 Method decision chart for Ain Shams University hospital laboratory. D.Bili - direct bilirubin. T.Bili - total bilirubin. ALT - alanine aminotransferase. Alk Phos - alkaline phosphatase. TP - total protein. GGT - gamma glutamyltransferase.
bm-28-2-020711-f3
{ label needed for fig[@id='f4'] }
Figure 4 Method decision chart of the private laboratory. D.Bili - direct bilirubin. T.Bili - total bilirubin. ALT - alanine aminotransferase. Alk Phos - alkaline phosphatase. TP - total protein. GGT - gamma glutamyltransferase. LD - lactate dehydrogenase. Mg - magnesium. Trig - triglycerides.
bm-28-2-020711-f4
Figure 5 Different Sigma metrics in the four laboratories using the same total allowable error. AFP - alpha fetoprotein. ALT - alanine aminotransferase. Alb – albumin. ALP - alkaline phosphatase. CEA - carcinoembryonic antigen. LD - lactate dehydrogenase. GGT - gamma glutamyltransferase.
bm-28-2-020711-f5

Discussion

This study is conducted over one year for four accredited medical laboratories. The sigma values were calculated based on performance approach and were compared to each other in a trial to highlight the lack of objective method of comparability. Control of analytical performance is an essential procedure that shall be done by medical laboratories specially those who are seeking accreditation through method verification which by itself is a standardized process but has no harmonized approach.

Implementing harmonized QC procedures in a medical laboratory requires both knowledge and practical updates. For each of these updates a lot of considerations can be made and a lot of problems can be found. This is the main reason behind lack of harmonized approach in implementing QC procedures. This study is a trial to solve the main problem facing most of the medical laboratories as well as accreditation bodies; which is the lack of harmonized approach for most laboratory`s procedures which results in different healthcare services and outcomes in most laboratories even those which are accreditation by ISO 15189:2012.

Miller and Sandberg recommended that the choice of quality requirements that focus on patient centred care and the optimum clinical decision for each analyte, expressed as TEa based on the change in the analyte would need to be detected in order to make a clinical decision based on that change (8). Some analyte changes would affect the clinical decisions when are relatively large (up to 50% for alanine aminotransferase and lipase activities). However, for some of the analytes, a relatively small change will affect the clinical decision in the management of the case such as electrolytes (8).

Gami et al. studied how different parameters have different biological variation. High biological variation parameter such as triglyceride measured by any instrument will give acceptable sigma level. While electrolytes like sodium and potassium which are having low biological variation would give low results (15). In our results the same observation was obtained as well. For triglyceride was calculated by three laboratories very high sigma (Chemical Pathology department Medical Research Institute, Alexandria University, Zagazig University Hospital laboratory and the private lab from Alexandria), while on the other hand potassium that were calculated by three laboratories had very low sigma level (Chemical Pathology department Medical Research Institute, Alexandria University, Zagazig University Hospital laboratory and Ain Shams University hospital laboratory).

The variability of TEa sources as the biological variation data used by Ricos and her colleagues are completely different from the PT limits used by CLIA. Sometimes if we want use the same source like biological variation (BV) the TEa might not be available (e.g. direct billirubin) and if available vary due to updating of the studies used and even if all this source of variability are nullified for sake of harmonization. The BV will answer the questions of method performance in three different ways according to which subtype of BV used (optimal TEa which equal half the desirable TEa and one third the minimal TEa).

Precision and bias verification are considered the corner stones of the verification procedures and as mentioned previously both of them have no harmonized protocol, different materials and different targets to achieve, even in case of choosing the same inputs we might get different output for example in case of using the external quality assessment material to determine bias the mean percent bias will be different according to the number, levels, commutability and uncertainty of materials used as all of these characteristics differ from program to another.

Sigma metrics calculation harmonization will help laboratories not to waste time and efforts analysing SM values and changing TEa sources to fit for each analyte. After harmonization the laboratory managers’ efforts will be directed towards the possible causes of poor performance. Taking the calcium in this study as an example, its sigma was unaccepted in three out of four laboratories and this might be due to: improper reagent handling starting from shipment, storage, preparation or on board stability or poor calibration/quality control procedure (reconstitution vehicle, storage and or mixing), or personnel incompetency or lack of instrument preventive maintenance or insufficient environmental conditions monitoring.

After reaching the right root cause, we will have the opportunity to select the proper corrective action and eventually achieve the medical laboratories` ultimate goal which is the high quality patient care.

Comparing the sigma levels in four accredited medical laboratories (three universities and one private laboratory) as an initiative to harmonize the sigma calculations to the best of our knowledge, authors in this study suggested a harmonized protocol for sigma calculation (Annex 1).

Our results showed different sigma levels for different parameters that were calculated using different TEa selected by each laboratory resulting in different categories of performance. This was in agreement with Schoenmakers et al. who discussed the variables that affects the sigma calculations and concluded that the use of the road map based on sigma metrics leads to fast and easy implementation of optimal Westgard QC rules. This approach needs standardization in order to lead to better patient care and ultimately in reduction of costs (16).

We compared the sigma level of the participating laboratories after unified the selected TEa, most of parameters compared to CLIA TEa except those which had no CLIA limits were compared to RCPA TEa. Table 5 shows SM that was calculated in the four laboratories after harmonization of TEa source. These results highlighted how can the TEa source selection affects the sigma level significantly in a way that may obscure the analytical performance.

Comparing the SM using the same TEa as step towards harmonization gives more real indicator of performance in a more objective approach than using different TEa by each laboratory. For example, total bilirubin SM according to the data calculated by each lab had far worse performance in MRI lab (2.0) than in Zagazig lab (7.8) while after using one common TEa (CLIA) in both laboratories the SM in MRI (7.8) showed better performance than that of Zagazig (4.1). Another example was glucose which according to the data calculated by each lab had almost the same SM in MRI and Zagazig which proved to be wrong when the TEa harmonized in both laboratories (CLIA) and showed that glucose performance at MRI (6.3) was better than that at Zagazig (3.9). Moreover, the data calculated by each laboratory showed that the magnesium had poor performance at MRI and Zagazig laboratory and excellent performance at the private laboratory, but after unifying the source of TEa and recalculating the magnesium SM using the same TEa (CLIA) the performance of magnesium at Zagazig proved to be excellent and even better than that of the private laboratory (6 and 5.6, respectively) and even the magnesium SM at MRI showed some increase (3.5).

The limitation of this study included: the Sigma metric equation as formulated by Westgard (17) is a subject of debate, with an alternate calculation being proposed by Coskun et al. (18, 19). However, the most commonly used, published, and cited equation remains the Westgard formulation. This Westgard calculation was established through derivation of critical-systematic error equations (20) and was confirmed by comparison to CpK (21) Finally, numerous official standards committees have accepted the Westgard formulation, notably the International Federation of Clinical Chemistry (IFCC) committee for standardization of HbA1c; they recommend the use of the Westgard Sigma metrics formulation in assessing and selecting HbA1c methods (22, 23). Even the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Task and Finish Group on Total Error, while acknowledging the alternate Coskun formulation, accepted the Westgard equation as the standard (24). Other EQA programs, such as Dutch Foundation for Quality Assessment in Medical Laboratories (SKML), offer the Westgard Sigma metrics as a standard part of their survey reports (25). In 2009, a convocation of experts on quality control issued a collective opinion paper recommending the use of Sigma metrics in the Westgard formulation (26). Simply put, the Coskun formulation of the Sigma metrics is neither in wide acceptance nor wide use.

There are many variables that affect the comparability of estimated Sigma metrics among medical laboratories which include: the time interval upon which Sigma metrics is calculated, the different vendor systems providing external proficiency testing programs and quality control programs upon which bias and imprecision values are calculated and different environmental conditions. Also, the variability in the methods used for bias calculation. In addition to the different analytical or clinical benchmarks that are chosen for evaluation of TEa.

The laboratories participating in this study determined analytical bias through EQA programs where the bias was calculated through the difference between laboratory result and that of the EQA group mean against the group mean. Therefore, this is not a true value.

In conclusion, this study is considered the first to highlight the need for Sigma metrics harmonization. Therefore it is mandatory that all laboratory professionals interested in the analytical quality to harmonize the approach of sigma calculation with special empathizes on the bias and CV which are the main components of the sigma equation as well as to unify the methodology used among different laboratories. As for the bias calculation it is recommended to standardize the calculation by using duplicate readings of a number of materials with different concentrations to exclude the element of random error if the PT samples are used as a source for bias calculation. This in turn will help laboratories to find a unified objective tool to judge their method correctly. Finally, the TEa sources shall be vigorously reviewed and only approved sources shall be adopted for calculation. Each laboratory should select the TEa goal based on clear standardized criteria of selection without any subjective preferences as either under or over estimation of Sigma metrics will affect the patient centred care negatively if laboratories use quality control procedures wrongly based on incorrect Sigma metrics calculation with subsequent misleading medical decisions. Laboratories performance using different tolerance limit can’t be compared to each other using sigma approach. Further studies shall be conducted by the accredited laboratories in different sectors adapting the concept of harmonized approach. One of the most important outcomes of this study is the suggested harmonized protocol presented in Annex 1.

Supplementary material

Annex1a.

Annex1b.

References

1 

Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol. 1950;20:1059–66. DOI: http://dx.doi.org/10.1093/ajcp/20.11_ts.1059 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/14783086

2 

Henry RJ, Segalove M. The running of standards in clinical chemistry and the use of the control chart. J Clin Pathol. 1952;5:305–11. DOI: http://dx.doi.org/10.1136/jcp.5.4.305 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/13011214

3 

Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem. 1981;27:493–501. PubMed: http://www.ncbi.nlm.nih.gov/pubmed/7471403

4 

Westgard JO. Basic QC Practices. 3rd ed. Madison, WI: Westgard QC, 2010.

5 

Dennis P. Forty Years on: A History of the Australian Association of Clinical Biochemists. Mt Lawley: Australian Association of Clinical Biochemists, 2005.

6 

Linnet K, Boyd JC. Selection and analytical evaluation of methods - with statistical techniques. In: Burtis CA, Ashwood ER, Bruns DE, eds. Tietz Textbook of Clinical Chemistry. 4th ed. St. Louis, MO: Elsevier Saunders; 2006. p. 7-48.

7 

Westgard JO, Carey RN, Wold S. Criteria for judging precision and accuracy in method development and evaluation. Clin Chem. 1974;20:825–33. PubMed: http://www.ncbi.nlm.nih.gov/pubmed/4835236

8 

Miller WG, Sandberg S. Quality control the analytical examination. In: Rifai N, Gay-Lussac LJ, eds. Tietz textbook of clinical chemistry and molecular diagnostics. 6th ed. Philadelphia, PA: Elsevier Saunders; 2018.

9 

Westgard JO. Six sigma quality design and control: desirable precision and requisite QC for laboratory measurement processes. Madison, WI: Westgard QC, 2001.

10 

Hens K, Berth M, Armbruster D, Westgard S. Sigma metrics used to assess analytical quality of clinical chemistry assays: importance of the allowable total error (TEa) target. Clin Chem Lab Med. 2014;52:973–80. DOI: http://dx.doi.org/10.1515/cclm-2013-1090 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/24615486

11 

Westgard JO, Klee GG. Quality management. In: Burtis CA, Ashwood ER, Bruns DE, eds. Tietz Textbook of Clinical Chemistry. 4th ed. St. Louis, MO: Elsevier Saunders; 2006.

12 

Lo V. Sigma metrics as performance indicator contributes to effective cost and man-hour saving in chemical pathology laboratory. Clin Chem. 2016;62:S159–73.

13 

Klee GG, Westgard JO. Quality Management. In: Burtis AC, Ashwood ER, Brurns DE, eds. Tietz textbook of clinical chemistry and molecular diagnostics. 5th ed. Philadelphia, PA: Elsevier Saunders; 2012; p.163-204. https://doi.org/ DOI: http://dx.doi.org/10.1016/B978-1-4160-6164-9.00008-1

14 

EQAS Program User Guide. Version 12.2015 Available at: http://www.qcnet.com/Portals/0/PDFs/EQASUserGuide.pdf. Accessed February 5th, 2018.

15 

Gami B, Patel D, Chauhan K, Shah H, Haridas N. Sigma metrics as a quality marker for analyzing electrolytes in laboratory. Int J Adv Res (Indore). 2013;1:197–201.

16 

Schoenmakers CH, Naus AJ, Vermeer HJ, van Loon D, Steen G. Practical application of Sigma Metrics QC procedures in clinical chemistry. Clin Chem Lab Med. 2011;49:1837–43. DOI: http://dx.doi.org/10.1515/cclm.2011.249 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/21801029

17 

Westgard JO, Westgard SA. Quality control review: implementing a scientifically based quality control system. Ann Clin Biochem. 2016;53:32–50. DOI: http://dx.doi.org/10.1177/0004563215597248 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/26150675

18 

Coskun A, Oosterhuis WP, Serteser M, Unsal I. Sigma metric or defects per million opportunities (DPMO): the performance of clinical laboratories should be evaluated by the Sigma metrics at decimal level with DPMOs. Clin Chem Lab Med. 2016;54:e217–9. DOI: http://dx.doi.org/10.1515/cclm-2015-1219 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/26760310

19 

Coskun A, Serteser M, Serdar M, Aksungar F, Kilercik M, Unsal I. Z transformation is the gold standard for computing the sigma metric. Clin Biochem. 2016;49:732–3. DOI: http://dx.doi.org/10.1016/j.clinbiochem.2016.04.002 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/27079504

20 

Westgard JO, Westgard SA. The quality of laboratory testing today: an assessment of sigma metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. Am J Clin Pathol. 2006;125:343–54. DOI: http://dx.doi.org/10.1309/V50H4FRVVWX12C79 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/16613337

21 

Westgard JO, Burnett RW. Precision requirements for cost-effective operation of analytical processes. Clin Chem. 1990;36:1629–32. PubMed: http://www.ncbi.nlm.nih.gov/pubmed/2119916

22 

Weykamp C, John G, Gillery P, English E, Ji L, Lenters-Westra E, et al. Investigation of 2 models to set and evaluate quality targets for hb a1c: biological variation and sigma-metrics. Clin Chem. 2015;61:752–9. DOI: http://dx.doi.org/10.1373/clinchem.2014.235333 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/25737535

23 

Lenters-Westra E, English E. Evaluating new HbA1c methods for adoption by the IFCC and NGSP reference networks using international quality targets. Clin Chem Lab Med. 2017;55:1426–34. DOI: http://dx.doi.org/10.1515/cclm-2017-0109 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/28432844

24 

Oosterhuis WP, Bayat H. The use of error and uncertainty methods in the medical laboratory. Clin Chem Lab Med. 2018;56:209–19. DOI: http://dx.doi.org/10.1515/cclm-2017-0341 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/28796637

25 

Thelen MHM, Jansen RTP, Weykamp CW, Steigstra H, Meijer R, Cobbaert CM. Expressing analytical performance from multi-sample evaluation in laboratory EQA. Clin Chem Lab Med. 2017;55:1509–16. DOI: http://dx.doi.org/10.1515/cclm-2016-0970 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/28182577

26 

Burnett D, Ceriotti F, Cooper G, Parvin C, Plebani M, Westgard J. Collective opinion paper on findings of the 2009 convocation of experts on quality control. Clin Chem Lab Med. 2010;48:41–52. DOI: http://dx.doi.org/10.1515/CCLM.2010.001 PubMed: http://www.ncbi.nlm.nih.gov/pubmed/19922240


This display is generated from NISO JATS XML with jats-html.xsl. The XSLT engine is libxslt.

Posjeta: 539 *