Keywords
SPIRIT, routinely collected data, reporting guideline, cohort, electronic health records, electronic patient records, registries, registry-based randomised controlled trial
This article is included in the HRB-TMRN gateway.
SPIRIT, routinely collected data, reporting guideline, cohort, electronic health records, electronic patient records, registries, registry-based randomised controlled trial
Randomized controlled trials (RCTs), the gold standard for the conduct of clinical research, have revolutionised the way that care is delivered to patients1,2. However, the frequently high cost of trials, the complex regulatory environment, and delays in incorporating trial findings into clinical practice threaten the ability of trials to continue to improve patient care into the future3–7.
A protocol is an essential document for any RCT. High quality protocols can assist efficient conduct, reporting, replicability and external review8. However, the completeness of the information provided in trial protocols is often inadequate8. One study carried out by Pildal and colleagues in 2005 found that many trials do not have clear allocation concealment according to their protocol9. In an effort to improve trial reporting quality, an international group of stakeholders launched the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Initiative in 2013 to provide evidence-based recommendations for the minimum set of items that need to be included in trial protocols. The core outputs of SPIRIT are the SPIRIT 2013 Statement, comprising of a 33-item checklist of minimum suggested protocol items with a diagram and an Explanation and Elaboration paper10. The SPIRIT website contains further information and resources11. The SPIRIT guidance has been instrumental in promoting high quality and transparent reporting of evaluations of interventions12. It provides the minimum guidance applicable for all clinical trial interventions and recognises that certain interventions may require extension or expansion of these items9,10,12.
In an observational cohort study, data collected from a group of individuals are gathered for the goal of conducting research13, whereas routinely collected data (RCD) refers to data collected for purposes other than research14. Recently, there is increasing interest and use of cohorts and RCD in RCTs because these data sources have the potential to improve efficiencies by facilitating recruitment, simplifying and reducing the cost of assessment of outcome measures and improving the applicability of trial findings15. Using RCD for RCTs offers novel concepts of testing health care interventions embedded in IT systems used in routine care and fosters conducting large pragmatic trials16. The use of cohorts and RCD provide the potential opportunity for low-cost long-term follow up minimising burden on participants17. To date, only a small number of trials access RCD to inform participant data, and few provide details on the use of such data sources18 but designs for trials conducted using cohorts or RCD including administrative databases, registries, and electronic health records (EHRs) are increasingly used in healthcare evaluation. One study carried out in the UK found that almost fifty percent of all publicly funded trials in the UK intend to collect data from RCD sources19.
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement was developed in 2004 to help improve the transparency of reporting of observational research20. In response, the REporting of studies Conducted using Observational Routinely-collected Data (RECORD) statement was developed as an extension to the STROBE statement to help address reporting items of a particular relevance to observational studies using RCD14. While, STROBE and RECORD have been instrumental in improving the transparency of reporting of research, both STROBE and RECORD are only intended to apply to observational research studies14. Thus, the recently developed CONSORT-ROUTINE extension extends the CONSORT guidance to aspects of trial reporting specific to trials conducted using cohorts and RCD data21 while our SPIRIT-ROUTINE extension aims to extend the SPIRIT guidance to protocols for trials conducted using cohorts and RCD. Guidance for what is expected in a protocol using RCD may lead to improvements in accessing data. For example, protocols that contain governance and technical information typically required by a data provider to release their data, will arguably expedite both information governance approvals and data provisioning processes and potentially lead to greater use of RCD22,23. There is currently no specific SPIRIT guidance on protocols for trials conducted using cohorts and RCD5.
The aim of this project is to outline the steps for the development of the SPIRIT-ROUTINE which will address this gap by extending the SPIRIT guidance for trials conducted using cohorts and RCD.
This project is registered with the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network since the 5th of May 202124. The development of this SPIRIT-ROUTINE extension will involve using a sequential approach, following previously published guidance on the development of protocols and the EQUATOR Network methodological framework25, consistent with the recently developed CONSORT Extension for Trials Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE)10; and similar to other SPIRIT extensions12,26. The development process involves 5 stages. Stage 1 consists of a project launch meeting to specify and finalise the scope of the extension and definition of cohorts and RCD. In Stage 2, a rapid review will be performed to identify possible modifications or new items to the original SPIRIT checklist. Potential items should have a specific relevance to trials using cohorts or RCD. We will then identify items for modification and places where additional items may be needed by reviewing the CONSORT-ROUTINE statement. Stage 3 will involve an online Delphi exercise, consisting of two rounds and involving key stakeholders to gather feedback on the draft checklist items. Stage 4 will involve a consensus meeting of the SPIRIT-ROUTINE team to suggest any new items and finalise all of the items to include in the extension, followed by stakeholder piloting of the checklist. Stage 5 will involve the publication preparation, dissemination and implementation of the final checklist.
A project operational team and a study steering committee have been established to deliver the project aims. The project operational team comprises, the lead Principal Investigator, Senior Research Fellow and Research Assistant. The SPIRIT-ROUTINE steering group was established to oversee the conduct and methodology of the project and comprises experts in trial methodology, including members who have conducted trials using cohorts and RCD; experts in the development of reporting guidelines and members of the SPIRIT and Trial Methodology Research Partnership (TMRP) Health Informatics and Outcomes working groups27. All members of the research team will advise on each stage of the project and will help draft and disseminate the final guideline document.
To create an initial ‘long list’ of items to consider for the SPIRIT-ROUTINE extension reporting guideline checklist, items from SPIRIT 2013 will first be examined to identify where modifications will be needed for trials conducted using cohorts and RCD, and items from the CONSORT-ROUTINE reporting guidelines will be investigated to identify supplementary items to SPIRIT items.
A search of the US National Library of Medicine’s clinical trial registry (ClinicalTrials.gov) will then be undertaken to find trial protocols using cohorts and RCD in Canada and the US (National Institute of Health (NIH) funded US trials) within the last five years. In addition, a similar search of the National Institutes of Health Research (NIHR) journals library in the UK will be undertaken. Search strategies are available in Extended data28.
These institutes operate within different geographies but with broadly similar objectives. In the US, the NIH, is the “nation’s medical research agency and aims to seek essential knowledge about the nature and behaviour of living systems and the application of that knowledge to help improve health and reduce illness and disability”29. The NIHR is the UKs largest funder of health and care research and the NIHR Health Technology Assessment programme contains published trial reports which are of high quality and provide detailed trial information30,31.
The results of each search will be individually downloaded into the citation management database Mendeley, and any duplicates will be removed. The eligibility of each protocol will be assessed through a two-stage process. Firstly, one reviewer will screen titles and abstracts. For an article to meet the criteria for full-text review, one reviewer must identify it as potentially eligible. Subsequently, a full-text review will be completed, with two investigators independently reviewing each article. Discrepancies after full-text review will be resolved by consensus, with a third investigator consulted if required. If the number of protocols identified is large, we will select a random sample for review and data extraction.
The eligibility criteria of each publication will be assessed as follows:
Inclusion criteria
Articles which are eligible will be examined to create a long list of items as follows: for trial protocols that describe aspects of methods or reporting of trials conducted using cohorts or RCD, we will examine the protocols and identify areas of trial design that are important to report. Potential items should be applicable to trials using cohorts or RCD and should clarify or alter an existing SPIRIT 2013 item or suggest a new element that should be separately reported as an item. Investigators will check elements for redundancy, and if either investigator or both deem it significant to report, the item will be included in the long list. The items identified will be added to an initial ‘long list’ of items, following the removal of duplicate records. Two investigators will independently extract the data from each protocol using a predesigned data extraction form, available in Extended data28. Before starting the Delphi process, we will remove items that are evidently not applicable to trials using cohorts or RCD.
The aim of the Delphi process is to evaluate the list of items for consideration to be included in the SPIRIT-ROUTINE extension and to identify any possible items that may not have been identified in the review. The Delphi survey will consist of two rounds and invitations to participate will be sent out via email for each round. Participants will complete the Delphi survey online using the COMET DelphiManager software32 and all participants will have a maximum of two weeks to complete each survey. The Delphi study participants will include members of the research team, clinical trialists, trial methodologists, guideline experts, Trial Methodology Research Network (TMRN) members and Patient and Public Involvement (PPI) contributors; participants may be added at the rapid review stage (e.g., authors of published trials using cohorts and RCD). While no official guidance on the minimum or perfect panel size for Delphi studies is available, we will aim to include a minimum of twenty participants, as this has been indicated by other studies to deliver reliable results5.
In round one, we will invite participants to rate items based on how valuable they are for the reporting of trial protocols on a Likert scale of 1–9, which will be labelled as: 1- 3= ‘not critical’ (items should not be part of the SPIRIT-ROUTINE extension checklist), 4–6= ‘no consensus’ (items should be discussed), 7–9 = ‘critical to include’ (item should be part of SPIRIT-ROUTINE extension checklist). All items rated from round one will be brought forward to the second round. In the second round, participants will be displayed the distribution of scores from other participants and the score that they attributed to each outcome, along with any comments from the previous round. They will be asked to reflect, and re-score if they want to, after having been shown the other participants’ views. DelphiManager enable this procedure by an easy setup followed by an inbuilt functionality to calculate the distribution of scores for a certain round33. In comparison to other online survey tools, the score distribution is then automatically presented to the participant in the next round and participants are reminded of their own score. Any item that is rated in the category ‘critical to include’ by more than 75% of the Delphi respondents will be deemed “consensus in”; likewise, if 75% or more score an item as ‘not critical’ it will be deemed ‘consensus out’. Participants will also be able to provide feedback when they rate items and can suggest any additional potential items, along with a rationale. New items will be added to the list for the second round if two or more participants suggest its inclusion, and it is not deemed to duplicate or have any major similarities with any other items already in the survey28. All responses will be anonymous and confidential. Results of the second round of the Delphi exercise will be presented during a consensus meeting to help inform the selection of checklist items.
A consensus meeting will be held with members of the study operational team, the SPIRIT-ROUTINE steering group and other key stakeholders including, guideline experts, clinical trialists, trial methodologists and PPI contributors. This meeting will be held with approximately 20 participants to establish consensus on the items to be included in the SPIRIT-ROUTINE extension checklist which will include developing item explanations and assessing consistency with the CONSORT-ROUTINE extension. This process will involve presentations of items by individuals with expertise, following a discussion. Firstly, the items in the Delphi survey which reached consensus will be discussed, following any possible objections. Subsequently, outstanding items will then be examined, and meeting participants will be provided with the opportunity to discuss each item. Participants will also be provided with the opportunity to discuss any items excluded during the Delphi process and can propose better explanations of any excluded items; this may involve attempting to alter the team’s conclusion about a particular item. We will arrive at consensus among meeting participants by implementing anonymous voting. Items with 75% or more of voters voting for its inclusion will be retained.
Knowledge translation is increasingly recognised to be an ‘intensely social process’ that depends on relationships between those who produce research evidence and users of that evidence3. The emerging and existing relationships between the primary investigator and her research team, members of the TMRP and collaborators will be used to disseminate the SPIRIT-ROUTINE extension.
The main outcome of the proposed research will be the development of the SPIRIT-ROUTINE extension for trials which utilise cohorts and RCD. The SPIRIT-ROUTINE extension will contribute to transparent reporting of these trials. The investigators will register their intent to proceed with this exercise with the EQUATOR network, and our proposed extension will be listed under “Reporting Guidelines Under Development”24. EQUATOR is an “umbrella organization that brings together researchers, journal editors, peer reviewers, research funders, and other collaborators with a mutual interest in improving the quality of research”34. We will use knowledge translation strategies consistent with previously successful efforts by SPIRIT and EQUATOR and members of our team, the majority of whom have such experience.
Strategies for knowledge translation may include:
(1) Involvement of key stakeholders including trialists, trial funders, trial methodologists, guideline experts, regulators, TMRN and Trial Methodology Research Partnership (TMRP), ethics board members, and PPI contributors in the team that is developing the extension with an extended group being recruited for the Delphi process including national and international experts
(2) Publication of the SPIRIT-ROUTINE extension in journals
(3) Dissemination via the SPIRIT group and EQUATOR network, including publication on their websites
(4) Presentations at conferences (e.g. submission to ICTMC 2022) and focused workshops on trials embedded in existing data sources
(5) Dissemination via the TMRN and TMRP with delivery of a Clinical Research Facility-Cork (CRF C/TMRN) webinar on the process of the development of a SPIRIT extension
(6) Dissemination will include presentation at the HRB-TMRN webinar and through relevant social media channels such as Twitter and YouTube
The proposed Delphi study abides by the ethical requirements of University College Cork aiming to assure rigour and responsibility in the conduct of research. Ethical approval for the Delphi survey will be sought at a later date from the Social Research Ethics Committee in University College Cork. All Delphi participants identified will be invited to complete each Delphi round via email. Informed consent will be assumed from Delphi participants who complete the survey. Participants will be made aware of the objectives of the Delphi survey in the email and that by completing the survey they will be giving permission for their anonymised responses to be used during the Delphi process, and to be accessed by members of the research team. Participants will also be made aware that their names will not be linked with the research materials, and will not be identifiable during the Delphi survey or in the reports that result from the research.
This SPIRIT-ROUTINE extension for trials conducted using cohorts and RCD aims to promote transparency and clarity and to reduce research waste due to inadequate reporting. Consistent with the recently developed CONSORT extension for trials conducted using cohorts and RCD5, this SPIRIT extension is being carried out with the long-term goal of improving the quality of reporting by establishing standards early in the process of uptake of these trial designs.
Open Science Framework: The development of a SPIRIT extension for trials conducted using cohorts and routinely collected data (SPIRIT-ROUTINE). https://doi.org/10.17605/OSF.IO/VQRUW28.
This project contains the following extended data:
• Appendix 1. SPIRIT-ROUTINE Electronic Search Strategies.docx (search strategy for Stage 2).
• Appendix 2. SPIRIT-ROUTINE Data extraction form.doc (data extraction form for protocols included in Stage 2).
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Medical Statistics
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Not applicable
References
1. Neal B, Wu Y, Feng X, Zhang R, et al.: Effect of Salt Substitution on Cardiovascular Events and Death. New England Journal of Medicine. 2021; 385 (12): 1067-1077 Publisher Full TextCompeting Interests: No competing interests were disclosed.
Reviewer Expertise: I am an expertise in the analysis of routinely collected data to examine the use and outcomes of prescribed medicines use in routine clinical care. I am also a leading authority on the ethics and governance of routinely collected data for population-based research.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 1 29 Jul 21 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with HRB Open Research
Already registered? Sign in
Submission to HRB Open Research is open to all HRB grantholders or people working on a HRB-funded/co-funded grant on or since 1 January 2017. Sign up for information about developments, publishing and publications from HRB Open Research.
We'll keep you updated on any major new updates to HRB Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)