Concordance between PCR-based extraction-free saliva and nasopharyngeal swabs for SARS-CoV-2 testing

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a novel coronavirus that rapidly spread across the globe in late December 2019 and was declared a pandemic by the World Health Organization (WHO) in March 2020 [press conference WHO 11^th March 2020]. Containing the spread of SARS-CoV-2 has been a significant challenge worldwide mainly because both asymptomatic and symptomatic individuals can transmit the virus (Huff & Singh, 2020). A key approach to limiting cross-infection is robust testing and contact tracing. Currently, the gold standard for diagnosis is nucleic acid detection by reverse transcription quantitative PCR (RT-qPCR) (Adhikari et al., 2020). Nasopharyngeal swabs (NPS) were initially adopted as the preferred sampling procedure for SARS-CoV-2, due to established diagnostic practices for other respiratory infections (Li et al., 2013; Lieberman et al., 2009). However, NPS are invasive and may induce coughing and sneezing, increasing the risk of transmission to healthcare professionals (HCP) conducting the procedure (Kim et al., 2016). NPS discomfort can also be a barrier to repeated, frequent testing as swabbing can induce effects that can last up to 24 hours post procedure including epistaxis, headaches, earaches and rhinorrhea (Gupta et al., 2021).

Testing for SARS-CoV-2 in saliva mitigates many of the challenges associated with NPS sampling (Tan et al., 2021; Vaz et al., 2020; Wyllie et al., 2020). Although various different protocols for SARS-CoV-2 testing in saliva have been proposed, including colorimetric reverse transcription loop-mediated isothermal amplification (RT-LAMP) and lateral flow assays (Faustini et al., 2020; Lalli et al., 2021), RT-qPCR is the most common used modality (Caulley et al., 2021; Migueres et al., 2020; Teo et al., 2021) with a reported sensitivity between ~ 69 to 100% (Azzi et al., 2020; Kojima et al., 2020; Pasomsub et al., 2021; Skolimowska et al., 2020; To et al., 2020).

SalivaDirect is a nucleic acid extraction-free, cost effective and reliable method for detecting SARS-CoV2 which has been authorised for use by the FDA (Vogels et al., 2021, FDA press release 15^th August 2020). Using SalivaDirect, specimens can be self-collected in a sterile sample tube without a viral transport medium. Proteinase K addition and a short heat-treatment step precede RT-qPCR analysis (Vogels et al., 2021). Initial studies using SalivaDirect reported a significant positive agreement (94%) between paired saliva and NPS samples obtained from a hospital cohort of 37 asymptomatic HCP and 30 COVID-19-positive inpatients (Vogels et al., 2021). In another study, matched saliva and NPS samples obtained from 30 individuals with COVID-19 illustrated a 88.2% concordance when using the SalivaDirect protocol (Rodríguez Flores et al., 2021).

Data comparing the concordance of ‘gold standard’ NPS and the SalivaDirect protocol for ongoing testing as part of an infection prevention and control programme are limited. In the present study, we set out to compare matching NPS and extraction-free saliva samples for detection of SARS-CoV-2 viral RNA via RT-qPCR, to assess the suitability of saliva as an alternative specimen for COVID-19 testing. We studied two key demographics – asymptomatic university students and hospital inpatients admitted with respiratory symptoms due to COVID-19 related illnesses.

Methods

Results

A total of 308 individuals, 181 asymptomatic students and 127 symptomatic patients, were included in this study (Table 1).

We first tested the correlation between NPS and saliva testing results in the symptomatic cohort. SARS-CoV-2 was detected in 86 of 127 patient NPS, while it was undetected in 41. Saliva testing in the same cohort indicated that 90 patients tested positive for SARS-CoV-2, while 37 tested negative. The sensitivity of saliva compared to NPS was 94.2% (95% CI 87.1-97.4%), while the specificity was 78.1% (95% CI 63.3-88%) in our symptomatic cohort.

We next tested the correlation between NPS and saliva in the asymptomatic cohort. Of 181 students, one tested positive both in NPS and saliva samples, while 180 tested negative in both. Sensitivity (95% CI 5-100%) and specificity (95% CI 97.9-100%) were therefore both 100% in this cohort.

The overall concordance between NPS and saliva testing is shown in Table 2. In the combined cohort, the sensitivity of saliva compared to NPS was 94.3% (95% CI 87.2-97.5%), while the specificity was 95.9% (95% CI 92.4-97.8%). Analysis of the concordance between the NPS and saliva specimens demonstrated an overall 95.5% accuracy for the saliva test relative to NPS and a very high level of agreement (κ coefficient = 0.889, 95% CI 0.833–0.946) between the two specimens. Nonetheless, we found 14 discordant samples between saliva and NPS. SARS-CoV-2 was detected in nine saliva samples but not detected in their matched NPS. On the other hand, viral RNA was not detected in five saliva samples but detected in the matched NPS. All 14 discordant samples belonged to the symptomatic hospitalised cohort. Ct values were >30 in 13/14 of these discordant samples and >35 in 6/14 samples.

To further explore the discordant results, we extracted RNA from the 14 discordant saliva samples and retested the samples with the TaqPath COVID-19 CE-IVD RT-PCR kit. SARS-CoV-2 viral RNA detection was confirmed in 7/9 ‘false positive’ saliva samples, indicating these were actually true positives which the NPS test failed to detect. The remaining two saliva samples in which SARS-CoV-2 was undetected had Ct values ≥35 using the SalivaDirect protocol. All five ‘false negative’ also tested negative using the TaqPath protocol, indicating that in this case saliva testing failed to detect the virus which was instead detected by NPS testing. The RdRp Ct values from NPS from these samples were >30 using the Genesig Real-Time PCR assay.

We next compared the Ct values of the concordant positive saliva and NPS samples (Figure 1A) in the symptomatic cohort. N1 Ct values were reported for saliva samples according to SalivaDirect protocol, and RdRp Ct values for the NPS were reported, as it is the target gene of the Genesig Real-Time PCR assay. The overall mean and standard deviation (SD) Ct value for the positive NPS specimens and saliva samples was 26.36 (SD 7.03) and 26.49 (SD 6.04), respectively. The difference in mean Ct values (0.132) was not statistically significant (p=0.860). We also found a highly significant correlation between the Ct values of the positive saliva and NPS specimens (Pearson r = 0.475, p<0.0001, Figure 1B).

Figure 1. Comparison of Ct values from concordant positive saliva and NPS samples from the symptomatic cohort (n=81).

(A) The mean Ct values for saliva specimens are not significantly different than the mean for NPS specimens. The lines indicate samples from the same patient. (B) Correlation of Ct values between saliva (N1) and NPS (RdRp) PCR. The scatter plot shows the comparison of Ct values between the two methods.

Finally, we compared NPS and saliva test results obtained from the hospitalised symptomatic patients (127 individuals) with the results from their initial positive diagnostic SARS-CoV-2 swab, which was performed on admission to hospital. The average time between the initial diagnostic NPS and the paired (same day) research saliva and NPS was 4.73 days (95% CI 4.05-5.42) post initial diagnosis. As expected, the positive agreement of NPS and saliva testing decreased with time, but a similar pattern for saliva and NPS is notable (see Figure 2).

Figure 2. Percentage of sensitivity of saliva and repeat NPS in the symptomatic cohort (n=127).

X axis: the delay (in days) between the positive NPS on hospital admission and the collection of the matching saliva and NPS sample. Matching NPS/saliva samples taken > 7 days post admission were pooled and reported as ‘>7’ (average of 11.53 days, 95% CI 10.02-13.03).

Discussion

Use of saliva to detect SARS-CoV-2 represents a valid and accurate alternative to NPS sampling. Our results indicate a 94.3% sensitivity and 95.9% specificity of saliva when compared to a matching NPS taken on the same day in a combined cohort of symptomatic and asymptomatic individuals. Our results are consistent with the original publication of the SalivaDirect method, where the positive agreement was 94% in the hospitalised cohort, and sensitivity and specificity of saliva versus NPS in 3779 asymptomatic individuals were 89.5% and >99.9%, respectively (Vogels et al., 2021). A separate study reported a sensitivity of 88.2% of saliva samples assessed with SalivaDirect when compared to matching NPS samples taken from 30 COVID-19-positive individuals (Rodríguez Flores et al., 2021).

In samples where the virus was detected, we observed a strong correlation between viral gene Ct values across NPS and saliva samples, consistent with recent reports (Braz-Silva et al., 2020; Mahmoud et al., 2021). Saliva-positive samples often differ from NPS in terms of Ct values/viral load (Braz-Silva et al., 2021; Skolimowska et al., 2020; Wyllie et al., 2020), however in our hands NPS and saliva samples showed similar Ct values, possibly due to the fact that we tested for different viral genes for NPS (RdRp) and saliva (N1).

Although we found an overall accuracy of 95.5%, 14 samples from the symptomatic cohort were discordant between matching NPS and saliva samples, with five individuals testing positive by NPS only and nine by saliva only. The majority of these samples showed Ct values (of either RdRp for NPS or N1 for saliva) over 30, suggesting that a higher discordance between NPS and saliva testing results could be observed for high Ct values, as reported before (Kandel et al., 2020). When we extracted RNA from the 14 discordant saliva samples and tested them using an alternative PCR protocol, we found two ‘false positives’ in saliva. This confirmatory step in case of high Ct values from the SalivaDirect protocol could therefore be helpful to avoid false positives, which are proportionally greater in low prevalence settings (Basile et al., 2020; Healy et al., 2021).

In the hospitalised symptomatic cohort, we assessed the positivity rate in NPS, and saliva samples taken a variable amount of time after the first diagnosis. As expected, the positive agreement between the first diagnosis and the saliva or repeat NPS decreased over time, reflecting recovery and viral clearance. These findings are consistent with other longitudinal studies of COVID-19 testing (Smith et al., 2021; Wyllie et al., 2020). However, our results are the first to show that the drop in performance is consistent across saliva and NPS, thereby showing once again that the SalivaDirect accuracy is very similar to the gold-standard RT-qPCR from NPS.

Limitations of the current study include the presence of only one positive in the asymptomatic population which limits the value of the sensitivity calculation in that cohort. In addition, a more closely matched control group would have been beneficial in this study, as samples from a healthy student population were compared to the symptomatic hospitalised cohort. Lastly, the use of Ct values shows a trend of the viral load but does not allow exact quantification of viral copies/ml due to the absence of a standard curve included in the RT-qPCR analyses.

Overall, although the sensitivity and specificity are slightly lower, this work suggests that the SalivaDirect protocol represents a valid alternative to NPS. Other diagnostic assays are available including lateral flow antigen tests (LFAT). However, a pilot study applying LFAT in an asymptomatic population in the UK indicated a sensitivity of only ~49% when compared to gold standard NPS testing (Wise, 2020). More recent studies have reported sensitivities for rapid antigen testing compared to NPS-based molecular testing (i.e. RT-qPCR) of ~93% (Fernandez-Montero et al., 2021; Krüger et al., 2021; Van der Moeren et al., 2021) in the presence of low Ct values/high viral load, however these values dropped when including high Ct values/low viral load (71.4%, 79.8% and 82.2% in the three works, respectively). In our hands SalivaDirect sensitivity compared to NPS was retained for high Ct values by showing a sensitivity of 100% for Ct values lower than 30 and 94.3% overall. Most importantly, rapid antigen tests currently in use still require NPS as the sampling method, which is widely perceived as uncomfortable and could be a barrier for repeated testing. On the other hand, SalivaDirect employs saliva as input, which is less invasive and hence a more acceptable sampling method compared to NPS (Goldfarb et al., 2021; Kinloch et al., 2020) and whose collection does not require direct interaction between HCP and individuals. This offers several advantages, decreasing the potential for cross-infection as well as alleviating testing bottlenecks by decreasing the need for qualified HCP, personal protective equipment, and swab supply (Wyllie et al., 2020). Saliva can be self-collected, although individuals should be instructed in proper use of the self-collection tube, including its decontamination after saliva collection. These advantages are particularly relevant in the context of a surveillance programme where compliance rates play a crucial role in the long-term success of the screening initiative and individuals would benefit from a less invasive and accurate sampling method, easier to collect in a serial manner.

Data availability

DRYAD: Concordance between PCR-based extraction-free saliva and nasopharyngeal swabs for SARS-CoV-2 testing, https://doi.org/10.5061/dryad.ksn02v74n (De Santi, 2021)

This project contains the following underlying data:

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).