Comparison of seven commercial RT-PCR kits with the NIV kit for the diagnosis of Covid-19

INTRODUCTION

Coronavirus disease 2019 (COVID-19) caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) spread across the globe in an unprecedented manner and was declared a pandemic on March 11, 2020 by the World Health Organization (WHO).¹ The disease presents with nonspecific clinical symptoms, and a definitive diagnosis can only be established in the laboratory. Laboratory testing is essential not only for diagnosis and management but also for containment and mitigation strategies to prevent further transmission.²

Over the course of time, numerous diagnostic technologies like nucleic acid amplification tests (NAAT) by real-time reverse transcription polymerase chain reaction (RT-PCR) and rapid antigen detection tests were approved for the laboratory diagnosis of COVID-19. However, RT-PCR remains the gold standard diagnostic test.^1,2

Multiple RT-PCR protocols for the detection of COVID-19 were published by WHO based on different target structural and nonstructural genes like envelope (E), nucleocapsid (N), RNA-dependent RNA polymerase (RdRp), open reading frame segments1 a/b (Orf1a/b), and the gene-encoding spike (S) protein. The E gene codes for nonstructural protein were specific to Sarbecovirus (β-CoV) while other structural and nonstructural genes target was specific to SARS-CoV-2. ³ As performance characteristics of molecular tests may vary with reagents, PCR, and instrumentation, an understanding of the analytical performance of different RT-PCR kits is essential for the proper interpretation of the results.⁴ Also, COVID-19 diagnostic tests have less accuracy in asymptomatic or Low-risk population and those person who may be have less viral load.^5,6 Multiple Indian Council of Medical Research (ICMR)-approved RT-PCR kits are currently available and are being used for the diagnosis of COVID-19.² This study was carried out with the aim to compare the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and agreement of the seven different RT-PCR kits for the diagnosis of COVID-19 with the ICMR-NIV kit.

MATERIAL AND METHODS

This study was performed at the molecular virology laboratory in the Department of Microbiology after obtaining ethical approval from the Institutional Ethics Committee (No: 439(88/2020) IEC). This was an observational, cross-sectional study in which 45 nasopharyngeal and oropharyngeal swab samples were incorporated fourbetween July 2021 and October 2021.

Collection of Samples: Forty-five nasopharyngeal and oropharyngeal swabs were collected in a Viral Transport Medium (VTM) tube from suspected COVID-19 patients. A repeat sample, after a gap of 2–3 days, was requested from the patients who had Ct values of ≥36. During the first wave of the COVID-19 pandemic, samples received for testing in the microbiology laboratory were for clinical symptomatic suspected COVID-19 diseases from the outpatients, inpatients, and patients of various intensive care units of the hospital.

Nucleic acid extraction: All the specimens were initially processed in Class II, Type A2 biosafety cabinet. Specimens were added into the lysis buffer, and RNA extraction was performed as per the manufacturer’s instructions using Easy Mag BioMerieux system (United States origin-based company).

RT-PCR kits: Eight different ICMR-approved kits (ICMR-NIV, LabGun^TM COVID-19 Assay, TIB MOLBIOL Roche, ARGENE SARS-CoV-2 R-GENE, BGI SARS-CoV-2, Perkin Elmer SARS-CoV-2, MyLab Patho Detect,^TM and COROSURE SARS-CoV-2) were used for testing these samples.² All the kits were based on TaqMan Fluorogenic probe-based chemistry except COROSURE SARS-CoV-2, which used SYBER Green dye and quantitative kit [Table 1].

ICMR-NIV RT-PCR kit has been indigenously developed by the National Institute of Virology (NIV), Pune, using internationally approved primer and probes.⁷ In this kit, coronaviruses under the subgenus Serbecovirus that includes 2019-nCoV, SARS-CoV, and bat SARS-like coronaviruses were used to generate a nonredundant alignment for screening of samples. Confirmatory assays using RdRp and ORF were designed based on their matching to the Wuhan virus as per inspection of the sequence alignment. Suspected human samples were first tested by (screening) E gene assay and then by confirmatory assay for the detection of RdRp and ORF gene in duplicates and other structural and nonstructural genes according to various SARS-Cov-2 detection kits. All samples were tested along with internal quality control, positive control, and negative control from various SARS-CoV-2 kits.

Amplification and detection: The RT-PCR tests were performed as per the manufacturer’s instructions described in each kit insert. Every run included a positive and negative control provided in the kit. For internal quality control, one known positive and negative sample was included in each run. All the RT-PCR assays were performed using BIORAD CFX-96 Real time system (Singapore origin-based company) according to the manufacturer’s instructions. The run was considered valid for any kit when the Ct value of controls was in the defined range, and the results for individual samples were recorded as per the Ct provided in the kit insert.

Statistical analysis: ICMR NIV, the Pune COVID-19 RT-PCR kit was used as the standard kit for the calculation of sensitivity, specificity, PPV, negative predictive value (NPV), and agreement of the different RT-PCR kits. The sensitivity and specificity of test kits were compared using the McNemar test. An inbuilt command in STATA 12E Statistical Software was used to obtain the kappa measure of integrated agreement between two COVID-19 RT-PCR Kits. None of the manufacturers were involved in the assessment and interpretation of the results.

RESULTS

Out of the 45 samples of suspected COVID-19 cases, both E and RdRp genes of SARS-CoV-2 were detected in 15 samples by ICMR-NIVCOVID-19 RT-PCR kit, and the remaining 30 samples were negative for SARS-CoV-2 genes. All the 15 samples that tested positive by the ICMR-NIV kit were also positive by other kits. However, out of the 30 samples that tested negative by the ICMR-NIV kit, 15 samples tested negative by all the kits, whereas 15 samples gave variable results with different kits [Table 2].

The sensitivity = number of true positives/number of true positives + number of false negatives; specificity: number of true negatives/number of true negatives + number of false positives; PPV = 100 × true positives/true positives + false positives; and NPV = 100×true negatives/false negatives+true negatives formulae were used to calculate the sensitivity, specificity, PPV, and NPV. All the kits showed 100% sensitivity and had 100% NPV when compared with the ICMR-NIV kit. However, specificity and PPV varied in comparison to the ICMR-NIV kit [Table 3].

The results of ARGENE-R, COROSURE, and MyLab PathoDetect^TM kits showed 100% concordance with the ICMR-NIV kit. Sensitivity, specificity, PPPV, NPV, and agreement of these kits were 100% (Kappa analysis 1.0000) [Tables 3 and 4].

With the BGI SARS-COV-2 RT-PCR kit (Cutoff Ct value ≤ 38), 26 samples tested positive and 19 tested negative. However, 11 samples that tested positive with Ct values ranging from 36 to 38 by the BGI kit were negative by the ICMR-NIV kit. Results of the BGI kit showed 73% specificity, 57% PPV, and moderate agreement (Kappa value 0.5352) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the LabGun ^TM COVID-19 RT PCR kit (Cutoff Ct value ≤ 40), 30 samples tested positive and 15 tested negative. However, 15 samples that tested positive with Ct values ranging from 36 to 40 by LabGun^TM were negative by the ICMR-NIV kit. Results of the LabGun ^TM kit showed 66% specificity, 50% PPV, and only fair agreement (Kappa value 0.4000) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the Perkin Elmer SARS-CoV-2 RT PCR kit (Cutoff Ct value ≤ 40), 24 samples tested positive and 21 tested negative. However, 9 samples that tested positive with Ct values of 36–40 by Perkin Elmer were negative by the ICMR-NIV kit. Results of the Perkin Elmer kit showed 81% Specificity, 68% PPV, and substantial agreement (Kappa value 0.6866) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the TIB MOLBIOL Roche SARS-CoV-2 RT PCR kit (Cutoff Ct value ≤ 38), 19 samples tested positive and 26 tested negative. However, 4 samples that tested positive with Ct values of 36–38 by the TIB MOLBIOL Roche kit were negative by the ICMR-NIV kit. Results of the TIB MOLBIOL Roche kit showed 88% Specificity, 78% PPV, and almost perfect agreement (Kappa value 0.8125) when compared with the ICMR-NIV kit [Tables 3 and 4].

Repeat samples were requested from 15 patients who had Ct ≥ 36, after a gap of 2–3 days. Only 14 patients submitted the repeat sample. Of these, 13 (92.8%) tested negative and 01 (7.2%) sample tested positive for SARS-CoV-2 by the respective kit and the ICMR-NIV kit.

DISCUSSION

Here we provide the comparison of seven commercially available RT-PCR kits with the ICMR-NIV kit for the diagnosis of COVID-19. These kits have been standardized to have different cutoff CTs ranging between 36 and 40 by the manufacturers. All the kits had 100% sensitivity and NPV, suggesting that these can correctly identify positive cases, and a negative report rules out infection. However, only three kits, namely, ARGENE-R, COROSURE, and Mylab PathoDetect^TM had 100% specificity, 100% PPV, and perfect agreement with the ICMR-NIV kit, suggesting that these can correctly identify negative cases, and a positive report indicates infection in the individual.

Four kits, namely, TIB MOLBIOL Roche, Perkin Elmer, BGI, and LabGun^TM had 88%, 81%, 73%, and 66% specificity, respectively. Manufacturer’s instructions should be followed for reporting of the results; however, our study revealed that Ct value ≥ 36 gave variable results with different kits. We found that 66.6% (30/45) of results were 100% concordant with NIV and COVID-19 RT PCR kit results hile 33.3% (15/45) showed variable results with 4 kits as compared to NIV, COVID-19 RT PCR kit results. On repeat testing of these (n = 14; one person did not submit the sample) samples, after 2–3 days, 13 (92.8%) tested negative, and 1 (7.2%) was positive for SARS-CoV-2 genes with the respective kit and NIV, COVID-19 RT PCR kit. Thirteen samples may either be “true negative” or in the late course of illness and hence became negative after 2–3 days. One that tested positive may have been in the early course of illness and hence tested positive after two days with respective kit and NIV, COVID-19 RT PCR kit. Overall, it can be stated from our obtained result that kits depending on a higher number of target genes show less false positive results. Our finding was comparable with the other study conducted in Bangladesh in 2023 published by Dip SD et al. ⁸

Variability of results in samples with Ct values > 36 suggests that these should be interpreted with caution. A false positive result in such cases may unnecessarily lead to quarantine/isolation of the individuals. In such cases, it is suggested that the test results should be reported as “Inconclusive” or “Indeterminate” and a repeat sample should be tested after a gap of 2–3 days to give the benefit of the doubt to the patient for appropriate management and public health authorities for the implementation of preventive measures. In addition, these may show inconsistent results in inter-laboratory comparison as different labs may be using different RT PCR kits.

All the kits were based on TaqMan Fluorogenic probe–based chemistry except COROSURE SARS-CoV-2, which uses SYBER Green dye. This kit showed 100% agreement with the NIV kit. The result was obtained faster (∼75 min) as compared to other kits (∼105 to 135 min). The SYBR green-based assay has been found to be equally sensitive to TaqMan assay for the diagnosis of West Nile Virus (WNV). Importantly, it also detected 100% of possible WNV target region variants.⁷ Probe-based assays are usually expensive, and the availability of SYBER Green dye–based assay may be an economical alternative for large-scale routine testing.

The main limitation of the study is its small sample size; however, considering the findings, this study suggests that all the seven COVID-19 RT-PCR kits can be used for routine diagnosis of COVID-19 patients. However, positive results having Ct values ≥36 should be interpreted with caution and a repeat sample should be asked to ascertain the presence of infection.

CONCLUSION

We also found that detection kits targeting more genes showed better accuracy, which yields less false positive results (<20%).