Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Filter by Categories
Article
Brief Report
Case Report
Commentary
Community Case Study
Editorial
Image
Images
Letter to Editor
Letter to the Editor
Mini Review
Obituary
Original Article
Perspective
Review Article
Reviewers; List
Short Communication
Task Force Report
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Filter by Categories
Article
Brief Report
Case Report
Commentary
Community Case Study
Editorial
Image
Images
Letter to Editor
Letter to the Editor
Mini Review
Obituary
Original Article
Perspective
Review Article
Reviewers; List
Short Communication
Task Force Report
View/Download PDF

Translate this page into:

Original Article
60 (
1
); 14-19
doi:
10.25259/ANAMS-2022-12-3-(794)

Comparison of seven commercial RT-PCR kits with the NIV kit for the diagnosis of Covid-19

Senior Consultant, Department of Microbiology, Atal Bihari Vajpayee institute of Medical Sciences and Ram Manohar Lohia Hospital, New Delhi, India
Department of Microbiology, University College of Medical Sciences and Guru Teg Bahadur Hospital, New Delhi, India
Department of Microbiology, Ram Manohar Lohia Hospital & Atal Bihari Vajpayee Institute of Medical Sciences, New Delhi, India
Statistical Officer, Ministry of health and family welfare, Government of India, New Delhi, India
Dr Ram Manohar Lohia Hospital and Atal Bihari Vajpayee Hospital, New Delhi, India.

* Corresponding author: Dr. Kirti Nirmal, Department of Microbiology, University College of Medical Sciences and Guru Teg Bahadur Hospital, Delhi, India. Email: doctorkirtinirmal@gmail.com

Licence
This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.

How to cite this article: Chhabra M, Nirmal K, Chauhan A, Athotra A, Kansra S, Shulania A, et al. Comparison of seven commercial RT-PCR kits with the NIV kit for the diagnosis of Covid-19. Ann Natl Acad Med Sci (India). 2024;60:14–9. doi: 10.25259/ANAMS-2022-12-3-(794)

Abstract

Objectives

Coronavirus disease 2019 (COVID-19) caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) spread across the globe in an unprecedented manner and was declared a pandemic on March 11, 2020 by the World Health Organization (WHO). This study was carried out with the aim to compare the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and agreement of the eight different RT-PCR kits for the diagnosis of COVID-19.

Material and Methods

This observational cross-sectional study was carried out in the Department of Microbiology of a tertiary care hospital in Central Delhi from July to October 2021. A total of 45 nasopharyngeal and/or oropharyngeal swabs in Viral Transport Medium (VTM) from suspected COVID-19 patients were received in the laboratory for RT-PCR. These samples were tested by eight different Indian Council of Medical Research (ICMR)-approved RT-PCR kits with different gene targets. The comparison was made with the National Institute of Virology (NIV), the Pune COVID-19 RT-PCR kit. Statistical analysis: sensitivity, specificity, PPV, and NPV were calculated for each kit and compared using the McNemar test. Agreement of different kits was evaluated using Kappa analysis.

Results

The results of the 45 samples of suspected COVID-19 cases were recorded as per the cycle threshold (Ct) provided in the kit insert. Of these, 15 samples detected both E and RdRp genes and 30 were negative for both the genes of SARS CoV-2 by NIV, the Pune COVID-19 RT-PCR kit. All kits showed 100% sensitivity and had 100% NPV when compared with the NIV kit. However, specificity, PPV, and agreement were variable as compared to the NIV kit.

Conclusion

The reporting should be carried out as per the manufacturer’s instructions. However, positive results with Ct values ≥ 36 showed variable results with different RT PCR kits and hence should be interpreted with caution.

Keywords

Agreement
COVID-19
RT-PCR Kits
SARS CoV-2

INTRODUCTION

Coronavirus disease 2019 (COVID-19) caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) spread across the globe in an unprecedented manner and was declared a pandemic on March 11, 2020 by the World Health Organization (WHO).1 The disease presents with nonspecific clinical symptoms, and a definitive diagnosis can only be established in the laboratory. Laboratory testing is essential not only for diagnosis and management but also for containment and mitigation strategies to prevent further transmission.2

Over the course of time, numerous diagnostic technologies like nucleic acid amplification tests (NAAT) by real-time reverse transcription polymerase chain reaction (RT-PCR) and rapid antigen detection tests were approved for the laboratory diagnosis of COVID-19. However, RT-PCR remains the gold standard diagnostic test.1,2

Multiple RT-PCR protocols for the detection of COVID-19 were published by WHO based on different target structural and nonstructural genes like envelope (E), nucleocapsid (N), RNA-dependent RNA polymerase (RdRp), open reading frame segments1 a/b (Orf1a/b), and the gene-encoding spike (S) protein. The E gene codes for nonstructural protein were specific to Sarbecovirus (β-CoV) while other structural and nonstructural genes target was specific to SARS-CoV-2. 3 As performance characteristics of molecular tests may vary with reagents, PCR, and instrumentation, an understanding of the analytical performance of different RT-PCR kits is essential for the proper interpretation of the results.4 Also, COVID-19 diagnostic tests have less accuracy in asymptomatic or Low-risk population and those person who may be have less viral load.5,6 Multiple Indian Council of Medical Research (ICMR)-approved RT-PCR kits are currently available and are being used for the diagnosis of COVID-19.2 This study was carried out with the aim to compare the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and agreement of the seven different RT-PCR kits for the diagnosis of COVID-19 with the ICMR-NIV kit.

MATERIAL AND METHODS

This study was performed at the molecular virology laboratory in the Department of Microbiology after obtaining ethical approval from the Institutional Ethics Committee (No: 439(88/2020) IEC). This was an observational, cross-sectional study in which 45 nasopharyngeal and oropharyngeal swab samples were incorporated fourbetween July 2021 and October 2021.

Collection of Samples: Forty-five nasopharyngeal and oropharyngeal swabs were collected in a Viral Transport Medium (VTM) tube from suspected COVID-19 patients. A repeat sample, after a gap of 2–3 days, was requested from the patients who had Ct values of ≥36. During the first wave of the COVID-19 pandemic, samples received for testing in the microbiology laboratory were for clinical symptomatic suspected COVID-19 diseases from the outpatients, inpatients, and patients of various intensive care units of the hospital.

Nucleic acid extraction: All the specimens were initially processed in Class II, Type A2 biosafety cabinet. Specimens were added into the lysis buffer, and RNA extraction was performed as per the manufacturer’s instructions using Easy Mag BioMerieux system (United States origin-based company).

RT-PCR kits: Eight different ICMR-approved kits (ICMR-NIV, LabGunTM COVID-19 Assay, TIB MOLBIOL Roche, ARGENE SARS-CoV-2 R-GENE, BGI SARS-CoV-2, Perkin Elmer SARS-CoV-2, MyLab Patho Detect,TM and COROSURE SARS-CoV-2) were used for testing these samples.2 All the kits were based on TaqMan Fluorogenic probe-based chemistry except COROSURE SARS-CoV-2, which used SYBER Green dye and quantitative kit [Table 1].

Table 1: Overview of RT-PCR kits used in this study for the detection of SARSCoV-2.
Kit name Kit chemistry Reaction time (min) Cycle threshold (Ct value) Gene target
Screening gene (Serbecovirus) Confirmatory gene
ICMR-NIV (Pune) TaqMan Fluorogenic probe 134 min ≤35 E RdRp ORF-1a/b
ARGENE SARS-CoV-2-R-Gene (France) TaqMan Fluorogenic probe 110 min ≤34 - N RdRp
BGI SARS-CoV-2 (China) TaqMan Fluorogenic probe 102 min ≤38 - ORF-1a/b (+) Sigmoid (S)-shaped curve
COROSURE SARS-CoV-2 (Faridabad) SYBER Green dye 72 min ≤37 -

S1 gene (+)

S 2gene (+)

(124 bp)

LabGunTM COVID-19 Assay (Korea) TaqMan Fluorogenic probe *(Limit of detection-100 copies/mL) 135 min ≤40 E RdRp
MylabPathoDetectTM (China) TaqMan Fluorogenic probe 105 min ≤38 E RdRp
Perkin Elmer SARS-CoV-2 (United States) TaqMan Fluorogenic probe 150 min ≤40 -

N

ORF-1a/b gene

TIB MOLBIOL Roche (Germany) TaqMan Fluorogenic probe 104 min ≤38 E RdRp

ICMR-NIV RT-PCR kit has been indigenously developed by the National Institute of Virology (NIV), Pune, using internationally approved primer and probes.7 In this kit, coronaviruses under the subgenus Serbecovirus that includes 2019-nCoV, SARS-CoV, and bat SARS-like coronaviruses were used to generate a nonredundant alignment for screening of samples. Confirmatory assays using RdRp and ORF were designed based on their matching to the Wuhan virus as per inspection of the sequence alignment. Suspected human samples were first tested by (screening) E gene assay and then by confirmatory assay for the detection of RdRp and ORF gene in duplicates and other structural and nonstructural genes according to various SARS-Cov-2 detection kits. All samples were tested along with internal quality control, positive control, and negative control from various SARS-CoV-2 kits.

Amplification and detection: The RT-PCR tests were performed as per the manufacturer’s instructions described in each kit insert. Every run included a positive and negative control provided in the kit. For internal quality control, one known positive and negative sample was included in each run. All the RT-PCR assays were performed using BIORAD CFX-96 Real time system (Singapore origin-based company) according to the manufacturer’s instructions. The run was considered valid for any kit when the Ct value of controls was in the defined range, and the results for individual samples were recorded as per the Ct provided in the kit insert.

Statistical analysis: ICMR NIV, the Pune COVID-19 RT-PCR kit was used as the standard kit for the calculation of sensitivity, specificity, PPV, negative predictive value (NPV), and agreement of the different RT-PCR kits. The sensitivity and specificity of test kits were compared using the McNemar test. An inbuilt command in STATA 12E Statistical Software was used to obtain the kappa measure of integrated agreement between two COVID-19 RT-PCR Kits. None of the manufacturers were involved in the assessment and interpretation of the results.

RESULTS

Out of the 45 samples of suspected COVID-19 cases, both E and RdRp genes of SARS-CoV-2 were detected in 15 samples by ICMR-NIVCOVID-19 RT-PCR kit, and the remaining 30 samples were negative for SARS-CoV-2 genes. All the 15 samples that tested positive by the ICMR-NIV kit were also positive by other kits. However, out of the 30 samples that tested negative by the ICMR-NIV kit, 15 samples tested negative by all the kits, whereas 15 samples gave variable results with different kits [Table 2].

Table 2: Variable cycle threshold (Ct) of the patient samples from different COVID-19 RT PCR kits.
S.No. Cycle threshold (Ct)
ICMR-NIV (Ct 35) BGI SARS-CoV-2 (Ct 38) COROSURE SARS-CoV-2 (Ct 37) ARGENE SARS-CoV-2-R-Gene (Ct 34) LabGunTM COVID-19 Assay (Ct 40) Mylab Patho DetectTM (Ct 38) Perkin Elmer SARS-CoV-2 (Ct 40) TIB MOLBIOL Roche (Ct 38)
1 26 22 30 30 19 25 28 27
2 29 32 31 28 29 29 26 32
3 30 32 28 28 30 30 30 33
4 28 30 32 30 29 28 29 32
5 26 26 30 32 23 26 28 31
6 24 28 30 30 26 26 31 28
7 28 30 33 28 29 28 30 33
8 28 32 34 30 28 28 29 31
9 34 30 30 34 32 31 31 34
10 31 28 34 32 30 31 32 32
11 29 30 30 30 28 29 28 32
12 21 24 25 24 20 22 21 24
13 24 20 24 28 23 23 21 28
14 30 30 30 30 29 31 28 33
15 20 22 24 24 20 21 20 22

The sensitivity = number of true positives/number of true positives + number of false negatives; specificity: number of true negatives/number of true negatives + number of false positives; PPV = 100 × true positives/true positives + false positives; and NPV = 100×true negatives/false negatives+true negatives formulae were used to calculate the sensitivity, specificity, PPV, and NPV. All the kits showed 100% sensitivity and had 100% NPV when compared with the ICMR-NIV kit. However, specificity and PPV varied in comparison to the ICMR-NIV kit [Table 3].

The results of ARGENE-R, COROSURE, and MyLab PathoDetectTM kits showed 100% concordance with the ICMR-NIV kit. Sensitivity, specificity, PPPV, NPV, and agreement of these kits were 100% (Kappa analysis 1.0000) [Tables 3 and 4].

Table 3: Sensitivity, Specificity, PPV, and NPV of various kits as compared to the NIV kit.
RT-PCR Kits Sensitivity % (95% Confidence interval) Specificity % (95% Confidence interval) PPV % (95% Confidence interval) NPV % (95% Confidence interval)
ARGENE SARS-COV-2-R-Gene 100 100 100 100
BGI SARS-COV-2 100 73 57 100
COROSURE SARS-COV-2 100 100 100 100
LabGunTM COVID-19 Assay 100 66 50 100
MylabPathoDetectTM 100 100 100 100
Perkin Elmer SARS-COV-2 100 81 68 100
TIB MOLBIOL Roche 100 88 78 100

Note: Sensitivity = number of true positives/number of true positives + number of false negatives

Specificity: number of true negative/number of true negative + number of false positive

Positive Predictive Value (PPV) = 100 × True positive/True positive + False positive)

Negative Predictive Value (NPV) = 100 × True negative/False negative + True Negative

Table 4: Kappa analysis for the measurement of agreement with the National Institute of Virology Kit.
COVID-19 RT-PCR kits Agreement with NIV RT-PCR kits (%) Expected Agreement (%) Kappa value Standard error z Prob > z
ARGENE SARS-COV-2-R-Gene 100.0 55.56 1.0000 0.1491 6.71 0.0000
BGI SARS-COV-2 75.56 47.41 0.5352 0.1320 4.05 0.0000
COROSURE SARS-COV-2 100.0 55.56 1.0000 0.1491 6.71 0.0000
LabGunTM COVID-19 Assay 66.67 44.44 0.4000 0.1193 3.35 0.0004
Mylab Patho DetectTM 100.0 55.56 1.0000 0.1491 6.71 0.0000
Perkin Elmer SARS-COV-2 84.44 50.37 0.6866 0.1416 4.85 0.0000
TIB MOLBIOL Roche 91.11 52.59 0.8125 0.1464 5.55 0.0000

Note: The kappa-statistic measure of agreement is scaled to be 0 when the amount of agreement is what would be expected to be observed by chance and 1 when there is perfect agreement.

With the BGI SARS-COV-2 RT-PCR kit (Cutoff Ct value ≤ 38), 26 samples tested positive and 19 tested negative. However, 11 samples that tested positive with Ct values ranging from 36 to 38 by the BGI kit were negative by the ICMR-NIV kit. Results of the BGI kit showed 73% specificity, 57% PPV, and moderate agreement (Kappa value 0.5352) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the LabGun TM COVID-19 RT PCR kit (Cutoff Ct value ≤ 40), 30 samples tested positive and 15 tested negative. However, 15 samples that tested positive with Ct values ranging from 36 to 40 by LabGunTM were negative by the ICMR-NIV kit. Results of the LabGun TM kit showed 66% specificity, 50% PPV, and only fair agreement (Kappa value 0.4000) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the Perkin Elmer SARS-CoV-2 RT PCR kit (Cutoff Ct value ≤ 40), 24 samples tested positive and 21 tested negative. However, 9 samples that tested positive with Ct values of 36–40 by Perkin Elmer were negative by the ICMR-NIV kit. Results of the Perkin Elmer kit showed 81% Specificity, 68% PPV, and substantial agreement (Kappa value 0.6866) when compared with the ICMR-NIV kit [Tables 3 and 4].

With the TIB MOLBIOL Roche SARS-CoV-2 RT PCR kit (Cutoff Ct value ≤ 38), 19 samples tested positive and 26 tested negative. However, 4 samples that tested positive with Ct values of 36–38 by the TIB MOLBIOL Roche kit were negative by the ICMR-NIV kit. Results of the TIB MOLBIOL Roche kit showed 88% Specificity, 78% PPV, and almost perfect agreement (Kappa value 0.8125) when compared with the ICMR-NIV kit [Tables 3 and 4].

Repeat samples were requested from 15 patients who had Ct ≥ 36, after a gap of 2–3 days. Only 14 patients submitted the repeat sample. Of these, 13 (92.8%) tested negative and 01 (7.2%) sample tested positive for SARS-CoV-2 by the respective kit and the ICMR-NIV kit.

DISCUSSION

Here we provide the comparison of seven commercially available RT-PCR kits with the ICMR-NIV kit for the diagnosis of COVID-19. These kits have been standardized to have different cutoff CTs ranging between 36 and 40 by the manufacturers. All the kits had 100% sensitivity and NPV, suggesting that these can correctly identify positive cases, and a negative report rules out infection. However, only three kits, namely, ARGENE-R, COROSURE, and Mylab PathoDetectTM had 100% specificity, 100% PPV, and perfect agreement with the ICMR-NIV kit, suggesting that these can correctly identify negative cases, and a positive report indicates infection in the individual.

Four kits, namely, TIB MOLBIOL Roche, Perkin Elmer, BGI, and LabGunTM had 88%, 81%, 73%, and 66% specificity, respectively. Manufacturer’s instructions should be followed for reporting of the results; however, our study revealed that Ct value ≥ 36 gave variable results with different kits. We found that 66.6% (30/45) of results were 100% concordant with NIV and COVID-19 RT PCR kit results hile 33.3% (15/45) showed variable results with 4 kits as compared to NIV, COVID-19 RT PCR kit results. On repeat testing of these (n = 14; one person did not submit the sample) samples, after 2–3 days, 13 (92.8%) tested negative, and 1 (7.2%) was positive for SARS-CoV-2 genes with the respective kit and NIV, COVID-19 RT PCR kit. Thirteen samples may either be “true negative” or in the late course of illness and hence became negative after 2–3 days. One that tested positive may have been in the early course of illness and hence tested positive after two days with respective kit and NIV, COVID-19 RT PCR kit. Overall, it can be stated from our obtained result that kits depending on a higher number of target genes show less false positive results. Our finding was comparable with the other study conducted in Bangladesh in 2023 published by Dip SD et al. 8

Variability of results in samples with Ct values > 36 suggests that these should be interpreted with caution. A false positive result in such cases may unnecessarily lead to quarantine/isolation of the individuals. In such cases, it is suggested that the test results should be reported as “Inconclusive” or “Indeterminate” and a repeat sample should be tested after a gap of 2–3 days to give the benefit of the doubt to the patient for appropriate management and public health authorities for the implementation of preventive measures. In addition, these may show inconsistent results in inter-laboratory comparison as different labs may be using different RT PCR kits.

All the kits were based on TaqMan Fluorogenic probe–based chemistry except COROSURE SARS-CoV-2, which uses SYBER Green dye. This kit showed 100% agreement with the NIV kit. The result was obtained faster (∼75 min) as compared to other kits (∼105 to 135 min). The SYBR green-based assay has been found to be equally sensitive to TaqMan assay for the diagnosis of West Nile Virus (WNV). Importantly, it also detected 100% of possible WNV target region variants.7 Probe-based assays are usually expensive, and the availability of SYBER Green dye–based assay may be an economical alternative for large-scale routine testing.

The main limitation of the study is its small sample size; however, considering the findings, this study suggests that all the seven COVID-19 RT-PCR kits can be used for routine diagnosis of COVID-19 patients. However, positive results having Ct values ≥36 should be interpreted with caution and a repeat sample should be asked to ascertain the presence of infection.

CONCLUSION

We also found that detection kits targeting more genes showed better accuracy, which yields less false positive results (<20%).

Ethical approval

The authors declare that they have taken the Institutional Ethics Committee approval and the approval number is 439(88/2020) IEC.

Declaration of patient consent

Patient’s consent not required as there are no patients in this study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Use of artificial intelligence (AI)-assisted technology for manuscript preparation

The authors confirm that there was no use of artificial intelligence (AI)-assisted technology for assisting in the writing or editing of the manuscript and no images were manipulated using AI.

References

  1. World Health Organization. Coronavirus Disease 2019 (COVID-19) Situation Report 154. Available from: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200622-covid-19.
  2. ICMR performance evaluation of commercial kits for real time pcrfor covid by icmridentified validation centres. Available from: https://www.finddx.org/covid-19/pipeline/.
  3. Molecular assays to diagnose COVID-19: Summary table of available protocols. Available at:https://www.who.int/publications/m/item/molecular-assays-to-diagnose-covid-19-summary-table-of-available-protocols.
  4. , , , , , . Detection of influenza H7N9 virus: All molecular tests are not equal. J Clin Microbiol. 2013;51:3835-38.
    [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
  5. , . Covid-19 molecular diagnostic testing - lessons learned. N Engl J Med. 2020;383:e97.
    [CrossRef] [PubMed] [Google Scholar]
  6. , , , , , . Laboratory preparedness for SARS-CoV-2 testing in India: Harnessing a network of virus research & diagnostic laboratories. Indian J Med Res. 2020;151:216-25.
    [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
  7. , , . SYBR green-based real-time quantitative PCR assay for detection of west nile virus circumvents false-negative results due to strain variability. J Clin Microbiol. 2004;42:1511-8.
    [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
  8. , , , , , . Evaluation of RT-PCR assays for detection of SARS-CoV-2 variants of concern. Sci Rep. 2023;13:2342.
    [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Show Sections