Concordance between PCR-based extraction-free saliva and nasopharyngeal swabs for SARS-CoV-2 testing

Introduction: Saliva represents a less invasive alternative to nasopharyngeal swab (NPS) for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) detection. SalivaDirect is a nucleic acid extraction-free method for detecting SARS-CoV2 in saliva specimens. Studies evaluating the concordance of gold standard NPS and newly developed SalivaDirect protocols are limited. The aim of our study was to assess SalivaDirect as an alternative method for COVID-19 testing. Methods: Matching NPS and saliva samples were analysed from a cohort of symptomatic (n=127) and asymptomatic (n=181) participants recruited from hospital and university settings, respectively. RNA was extracted from NPS while saliva samples were subjected to the SalivaDirect protocol before RT-qPCR analysis. The presence of SARS-Cov-2 was assessed using RdRp and N1 gene targets in NPS and saliva, respectively. Results: Overall we observed 94.3% sensitivity (95% CI 87.2-97.5%), and 95.9% specificity (95% CI 92.4-97.8%) in saliva when compared to matching NPS samples. Analysis of concordance demonstrated 95.5% accuracy overall for the saliva test relative to NPS, and a very high level of agreement (κ coefficient = 0.889, 95% CI 0.833–0.946) between the two sets of specimens. Fourteen of 308 samples were discordant, all from symptomatic patients. Ct values were >30 in 13/14 and >35 in 6/14 samples. No significant difference was found in the Ct values of matching NPS and saliva sample ( p=0.860). A highly significant correlation (r = 0.475, p<0.0001) was also found between the Ct values of the concordant positive saliva and NPS specimens. Conclusions: Use of saliva processed according to the SalivaDirect protocol represents a valid method to detect SARS-CoV-2. Accurate and less invasive saliva screening is an attractive alternative to current testing methods based on NPS and would afford greater capacity to test asymptomatic populations especially in the context of frequent testing.


Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a novel coronavirus that rapidly spread across the globe in late December 2019 and was declared a pandemic by the World Health Organization (WHO) in March 2020 [press conference WHO 11 th March 2020]. Containing the spread of SARS-CoV-2 has been a significant challenge worldwide mainly because both asymptomatic and symptomatic individuals can transmit the virus (Huff & Singh, 2020). A key approach to limiting cross-infection is robust testing and contact tracing. Currently, the gold standard for diagnosis is nucleic acid detection by reverse transcription quantitative PCR (RT-qPCR) (Adhikari et al., 2020). Nasopharyngeal swabs (NPS) were initially adopted as the preferred sampling procedure for SARS-CoV-2, due to established diagnostic practices for other respiratory infections (Li et al., 2013;Lieberman et al., 2009). However, NPS are invasive and may induce coughing and sneezing, increasing the risk of transmission to healthcare professionals (HCP) conducting the procedure (Kim et al., 2016). NPS discomfort can also be a barrier to repeated, frequent testing as swabbing can induce effects that can last up to 24 hours post procedure including epistaxis, headaches, earaches and rhinorrhea (Gupta et al., 2021).
SalivaDirect is a nucleic acid extraction-free, cost effective and reliable method for detecting SARS-CoV2 which has been authorised for use by the FDA (Vogels et al., 2021, FDA press release 15 th August 2020). Using SalivaDirect, specimens can be self-collected in a sterile sample tube without a viral transport medium. Proteinase K addition and a short heat-treatment step precede RT-qPCR analysis (Vogels et al., 2021). Initial studies using SalivaDirect reported a significant positive agreement (94%) between paired saliva and NPS samples obtained from a hospital cohort of 37 asymptomatic HCP and 30 COVID-19-positive inpatients (Vogels et al., 2021). In another study, matched saliva and NPS samples obtained from 30 individuals with COVID-19 illustrated a 88.2% concordance when using the SalivaDirect protocol (Rodríguez Flores et al., 2021).
Data comparing the concordance of 'gold standard' NPS and the SalivaDirect protocol for ongoing testing as part of an infection prevention and control programme are limited. In the present study, we set out to compare matching NPS and extraction-free saliva samples for detection of SARS-CoV-2 viral RNA via RT-qPCR, to assess the suitability of saliva as an alternative specimen for COVID-19 testing. We studied two key demographics -asymptomatic university students and hospital inpatients admitted with respiratory symptoms due to COVID-19 related illnesses.

Ethical considerations
This study was approved by the Ethics Committee of Royal College of Surgeons in Ireland (RCSI) (study code: REC202010011) and the National Research Ethics Committee for COVID-19 (20-NREC-COV-056). All subjects involved in this study provided written informed consent.

Sample collection
Research participants were recruited at two sites; a symptomatic patient cohort (n=127) was assembled at Beaumont Hospital, Dublin, Ireland and an asymptomatic 'student' cohort (n=181) at RCSI University of Medicine and Health Sciences, Dublin, Ireland. Recruitment took place during the period from November 2020 to March 2021 for the symptomatic population and in December 2020 and January 2021 for the asymptomatic student cohort.
Symptomatic individuals were inpatients at Beaumont Hospital who tested positive for SARS-CoV-2 on NPS-based diagnostic admission testing, carried out in the hospital's clinical microbiology laboratory (using either the CerTest Biotec VIASURE SARS-CoV-2 real time PCR detection kit or Cepheid Xpert Xpress SARS-CoV-2). Patients were recruited to the research study post admission to hospital. Individuals in the symptomatic arm provided a research NPS sample, (collected by a trained professional), and a saliva sample (collected by passive drooling). The research NPS and saliva were collected on the same day. Asymptomatic students were recruited via an in-house screening programme at RCSI. NPS samples were collected by a trained professional with a saliva sample collected by passive drooling immediately afterwards. In both cohorts, exclusion criteria for participation to the study were any of the following activities conducted in the half hour prior to saliva sample collection: smoking, drinking any liquids, food, nasal sprays, tooth brushing and/or mouth washing. We set these as exclusion criteria as these activities might leave residues in the saliva (thereby influencing the samples quality of the sample and associated handling in the laboratory) or affect the samples viral load in the sample (Eduardo et al., 2021;(Seneviratne et al., 2021).
NPS sample processing NPS from the symptomatic cohort were placed in 2 ml of Viral Transport Media and sent to the CEPHR Laboratory using a biomedical courier. Samples were aliquoted into 2 ml cryovials and stored at -80°C until further use. Using 250 µl of the biobanked NPS, RNA was extracted using the automated platform for nucleic acid extraction EX3600 (Liferiver Biotech, Shanghai, China) according to manufacturer's instructions, with a nucleic acid elution volume of 60 µl. Following RNA extraction, samples underwent RT-qPCR for SARS-CoV-2 with primers directed against the RNA-dependent RNA polymerase (RdRp) region of the viral genome, using the COVID-19 Genesig Real-Time PCR assay (Primerdesign Ltd, Hampshire, United Kingdom) on The LightCycler 480 PCR platform (Roche Diagnostics, Basel, Switzerland) with the following thermocycling conditions: 10 min 55°C, 2 min 95°C, 45 cycles of 10 sec 95°C/60 sec 60°C. Each run included a SARS-CoV-2 positive control (RNA), internal control, no-template control and a positive control of extraction (RNA from SARS-CoV-2 virus -2019-nCoV/Italy-INMI1). Samples with quantification cycle (Cq) values below 40 cycles were defined as SARS-CoV-2-detected.

Saliva sample processing
All saliva specimens (i.e. from both the symptomatic and asymptomatic cohorts) were processed in the COVID-19 Testing Laboratory at RCSI using the SalivaDirect protocol (Vogels et al., 2021). Briefly, 50 µl of each saliva sample was added to 2.5 µl of proteinase K (50 mg/ml, ThermoFisher Scientific), vortexed and incubated at 95°C for 5 minutes to ensure inactivation of the virus. The handling of the saliva samples (which were of a thick consistency) was performed with particular attention in order to avoid between-samples contamination. RT-qPCR amplification was performed using TaqPath 1-Step RT-qPCR Master Mix (4x, ThermoFisher Scientific), and the CDC N1 and RP sets of primers and probes as per SalivaDirect protocol at the following concentrations: N1_F primer (400nM), N1_R primer (400nM), and N1_probe (200nM); RP_F primer (150nM), RP_R primer (150nM), and RP_probe (200nM). The TaqPath RT-qPCR Master Mix (15 µL) was added to 5 µL of the "extraction-free" saliva sample and run on a QuantStudio 7 Pro Real-Time PCR with the following thermocycling conditions: 10 min 52°C, 2 min 95°C and 45 cycles of 10 sec 95°C/30 sec 55°C. Ct values lower than 40 were interpreted as detection of the gene and viral status was called as per SalivaDirect recommendation (Vogels et al., 2021).

RNA extraction from saliva and SARS-CoV-2 detection using TaqPath COVID-19 CE-IVD RT-PCR Kit
In some cases, i.e. when the saliva and NPS testing results were not concordant, saliva samples were extracted using the MagMax Viral/Pathogen II Nucleic Acid Isolation Kit (ThermoFisher Scientific) following the protocol described above for NPS processing. This was a quality control step introduced to reduce the risk of false positive or negative results from saliva processed according to SalivaDirect protocol. RT-qPCR amplification of the SARS-CoV-2 ORF1 gene, N gene and S gene was performed using TaqPath 1-Step Multiplex Master Mix (No ROX) (4X) and the TaqPath COVID-19 CE-IVD RT-PCR Kit as per manufacturers' instructions. Briefly, 15 µL of the prepared mix were added to 10 µL of the RNA extracted from each saliva sample and run on a QuantStudio 7 Pro Real-Time PCR with the following thermocycling conditions: 2 min 25°C, 10 min 53°C, 2 min 95°C, and 40 cycles of 3 sec 95°C/30 sec 60°C. Ct values lower than 37 were interpreted as indicating the expression of that gene and viral status was called as per TaqPath COVID-19 CE-IVD RT-PCR Kit manufacturers' instructions.

Statistical analyses
All statistical analyses were performed using GraphPad PRISM version 8.0.0 (GraphPad Software, San Diego, CA, USA). Sensitivity, specificity and 95% confidence intervals (CI) were calculated to assess diagnostic performance. Agreement between the NPS and saliva specimens for the virus detection ability was also assessed using Cohen's Kappa (κ coefficient). Paired t-tests were used to compare the Ct values between NPS and saliva. Correlation between NPS and saliva Ct values were quantified using the Pearson correlation coefficient (Pearson r). All p-values were two-sided and p <0.05 was considered significant.

Results
A total of 308 individuals, 181 asymptomatic students and 127 symptomatic patients, were included in this study (Table 1).
We first tested the correlation between NPS and saliva testing results in the symptomatic cohort. SARS-CoV-2 was detected in 86 of 127 patient NPS, while it was undetected in 41. Saliva testing in the same cohort indicated that 90 patients tested positive for SARS-CoV-2, while 37 tested negative. The sensitivity of saliva compared to NPS was 94.2% (95% CI 87.1-97.4%), while the specificity was 78.1% (95% CI 63.3-88%) in our symptomatic cohort.
We next tested the correlation between NPS and saliva in the asymptomatic cohort. Of 181 students, one tested positive both in NPS and saliva samples, while 180 tested negative in both. Sensitivity (95% CI 5-100%) and specificity (95% CI 97.9-100%) were therefore both 100% in this cohort.
The overall concordance between NPS and saliva testing is shown in Table 2. In the combined cohort, the sensitivity of saliva compared to NPS was 94.3% (95% CI 87.2-97.5%), while the specificity was 95.9% (95% CI 92.4-97.8%). Analysis of the concordance between the NPS and saliva specimens demonstrated an overall 95.5% accuracy for the saliva test relative to NPS and a very high level of agreement (κ coefficient = 0.889, 95% CI 0.833-0.946) between the two specimens. Nonetheless, we found 14 discordant samples between saliva and NPS. SARS-CoV-2 was detected in nine saliva samples but not detected in their matched NPS. On the other hand, viral RNA was not detected in five saliva samples but detected in the matched NPS. All 14 discordant samples belonged to the symptomatic hospitalised cohort. Ct values were >30 in 13/14 of these discordant samples and >35 in 6/14 samples.
To further explore the discordant results, we extracted RNA from the 14 discordant saliva samples and retested the samples with the TaqPath COVID-19 CE-IVD RT-PCR kit. SARS-CoV-2 viral RNA detection was confirmed in 7/9 'false positive' saliva samples, indicating these were actually true positives which the NPS test failed to detect. The remaining two saliva samples in which SARS-CoV-2 was undetected had Ct values ≥35 using the SalivaDirect protocol. All five 'false negative' also tested negative using the TaqPath protocol, indicating that in this case saliva testing failed to detect the virus which was instead detected by NPS testing. The RdRp Ct values from NPS from these samples were >30 using the Genesig Real-Time PCR assay.
We next compared the Ct values of the concordant positive saliva and NPS samples ( Figure 1A) in the symptomatic cohort. N1 Ct values were reported for saliva samples according to SalivaDirect protocol, and RdRp Ct values for the NPS were reported, as it is the target gene of the Genesig Real-Time PCR assay. The overall mean and standard deviation (SD) Ct value for the positive NPS specimens and saliva   samples was 26.36 (SD 7.03) and 26.49 (SD 6.04), respectively. The difference in mean Ct values (0.132) was not statistically significant (p=0.860). We also found a highly significant correlation between the Ct values of the positive saliva and NPS specimens (Pearson r = 0.475, p<0.0001, Figure 1B).
Finally, we compared NPS and saliva test results obtained from the hospitalised symptomatic patients (127 individuals) with the results from their initial positive diagnostic SARS-CoV-2 swab, which was performed on admission to hospital. The average time between the initial diagnostic NPS and the paired (same day) research saliva and NPS was 4.73 days (95% CI 4.05-5.42) post initial diagnosis. As expected, the positive agreement of NPS and saliva testing decreased with time, but a similar pattern for saliva and NPS is notable (see Figure 2).

Discussion
Use of saliva to detect SARS-CoV-2 represents a valid and accurate alternative to NPS sampling. Although we found an overall accuracy of 95.5%, 14 samples from the symptomatic cohort were discordant between matching NPS and saliva samples, with five individuals testing positive by NPS only and nine by saliva only. The majority of these samples showed Ct values (of either RdRp for NPS or N1 for saliva) over 30, suggesting that a higher discordance between NPS and saliva testing results could be observed for high Ct values, as reported before (Kandel et al., 2020). When we extracted RNA from the 14 discordant saliva samples and tested them using an alternative PCR protocol, we found two 'false positives' in saliva. This confirmatory step in case of high Ct values from the SalivaDirect protocol could therefore be helpful to avoid false positives, which are proportionally greater in low prevalence settings (Basile et al., 2020;Healy et al., 2021).
In the hospitalised symptomatic cohort, we assessed the positivity rate in NPS, and saliva samples taken a variable amount of time after the first diagnosis. As expected, the positive agreement between the first diagnosis and the saliva or repeat NPS decreased over time, reflecting recovery and viral clearance. These findings are consistent with other longitudinal studies of COVID-19 testing (Smith et al., 2021;Wyllie et al., 2020). However, our results are the first to show that the drop in performance is consistent across saliva and NPS, thereby showing once again that the SalivaDirect accuracy is very similar to the gold-standard RT-qPCR from NPS.
Limitations of the current study include the presence of only one positive in the asymptomatic population which limits the value of the sensitivity calculation in that cohort. In addition, a more closely matched control group would have been beneficial in this study, as samples from a healthy student population were compared to the symptomatic hospitalised cohort. Lastly, the use of Ct values shows a trend of the viral load but does not allow exact quantification of viral copies/ml due to the absence of a standard curve included in the RT-qPCR analyses.
Overall, although the sensitivity and specificity are slightly lower, this work suggests that the SalivaDirect protocol represents a valid alternative to NPS. Other diagnostic assays are available including lateral flow antigen tests (LFAT). However, a pilot study applying LFAT in an asymptomatic population in the UK indicated a sensitivity of only ~49% when compared to gold standard NPS testing (Wise, 2020 Most importantly, rapid antigen tests currently in use still require NPS as the sampling method, which is widely perceived as uncomfortable and could be a barrier for repeated testing. On the other hand, SalivaDirect employs saliva as input, which is less invasive and hence a more acceptable sampling method compared to NPS. (Goldfarb et al., 2021;Kinloch et al., 2020) and whose collection does not require direct interaction between HCP and individuals. This offers several advantages, decreasing the potential for cross-infection as well as alleviating testing bottlenecks by decreasing the need for qualified HCP, personal protective equipment, and swab supply (Wyllie et al., 2020). Saliva can be self-collected, although individuals should be instructed in proper use of the self-collection tube, including its decontamination after saliva collection. These advantages are particularly relevant in the context of a surveillance programme where compliance rates play a crucial role in the long-term success of the screening initiative and individuals would benefit from a less invasive and accurate sampling method, easier to collect in a serial manner.
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Methods
Saliva sample processing "Briefly, 50 µl of each saliva sample was added to 2.5 µl of proteinase K"-Mention the concentration or units of proteinase K added and Brand.
Mention about RT-qPCR cycle changes with NPS and saliva.

Discussion
Discuss what it may imply that by testing saliva this does not meet the established requirements (smoking, drinking fluids, etc.) It should be compared to currently used methods for rapid detection showing over 90% sensitivity. Why use SalivaDirect and not those?
Did the authors notice any disadvantages of using the SalivaDirect? If so, discuss it.

Is the study design appropriate and is the work technically sound? Yes
Are sufficient details of methods and analysis provided to allow replication by others?
longitudinal study. However, the asymptomatic cohort (students) only provided one same day NPS/saliva sample during their appointment as per routine in-house screening at our University (RCSI).

Methods
Saliva sample processing "Briefly, 50 μl of each saliva sample was added to 2.5 μl of proteinase K"-Mention the concentration or units of proteinase K added and Brand. Answer: This detail has now been added to the methods section text.
Mention about RT-qPCR cycle changes with NPS and saliva. Answer: The comparison between Cts from concordant NPS and saliva samples (for the symptomatic cohort) is addressed in the results section ("We next compared the Ct values of the concordant positive saliva and NPS samples ( Figure 1A) in the symptomatic cohort…") and shown in Figure 1. Although we did observe strong correlation between the Ct values in NSP and saliva as expected from the literature, we did not see significant differences in Ct count as other relevant publications have reported. This point has not been added to the discussion section, where we mention that it should be noted that we tested for different viral genes for NPS (RdRp) and saliva (N1).

Results
"Sensitivity (95% CI 5-100%) and specificity (95% CI 97.9-100%) were, therefore, both 100% in this cohort." 95%-CIs surrounding the point estimator of sensitivity is too wide (5-100%). Re-calculate values or clarify. Answer: The 95%-CIs surrounding the sensitivity (CI 5-100%) is correctly calculated. The wide interval is to be expected given that only one student tested positive in our asymptomatic cohort. We acknowledge in the discussion the limited value of this sensitivity value. We therefore combined the two cohorts of symptomatic and asymptomatic for the overall sensitivity (94.3%; 95% CI 87.2-97.5%) and specificity values (95.9%; 95% CI 92.4-97.8%).

Discussion
Discuss what it may imply that by testing saliva this does not meet the established requirements (smoking, drinking fluids, etc.). Answer: Smoking, drinking any liquids, food, nasal sprays, tooth brushing and/or mouth washing might affect the quality of the saliva sample and/or the validity of the saliva testing. For example, the use of mouth wash was shown to reduce the viral load in saliva samples (Seneviratne et al., 2021) and therefore using mouth wash prior to providing a saliva sample should be avoided. Presence of food, mucus and residues such as those from brushing teeth or smoking can decrease the quality of the samples and prove their handling more difficult due to particles presence in the saliva. A sentence about the effects of these activities on saliva testing has now been added to the method section.  PMID: 34127960). In our hands the sensitivity of saliva testing compared to NPS was maintained for high Cts -it was 100% for Cts in NPS lower than 30, and, as reported in our manuscript, 94.3% overall. But most importantly, the rapid antigen tests currently available still required a nasopharyngeal swab (often self-administered in case of asymptomatic screening programmes) which is widely perceived as uncomfortable and linked to coughing, sneezing, headache which can be a barrier to repeated testing (for example with younger students). Therefore, we propose PCR-based saliva testing as a valid alternative to NSP, especially when employing SalivaDirect protocol which skips the RNA extraction step (thereby saving money and time), is a relatively cheap and accurate method to detect SARS-CoV-2, especially for those individuals who will require repeated testing. Moreover, reproducible finding of PCR-based saliva testing is a prerequisite to developing and validating rapid antigen saliva tests, which so far haven't reached satisfactory diagnostic performance (Nagura-Ikeda et al., J Clin Microbiol. 2020 Aug 24;58(9):e01438-20; Audigé et al ., Microorganisms. 2021 Aug 10;9(8):1700). We have now added a sentence in the discussion section that reinforce the advantages of this approach for PCR-based SARS-CoV-2 testing.
Did the authors notice any disadvantages of using the SalivaDirect? If so, discuss it. Answer: We found no significant disadvantage in using SalivaDirect compared to RNA extraction-based saliva testing. However we did notice that the handling of the saliva samples was more challenging than the NPS (they were more gloopy, the colour and density varied greatly among the samples). We have now mention this in the methods section.