Keywords
Internet-based, Randomised trials, Decentralised, Digital, E-health, Web-based, Young people, Children
Internet-based, Randomised trials, Decentralised, Digital, E-health, Web-based, Young people, Children
Digital technologies are increasingly important in most aspects of daily life. Health research is especially well poised to benefit from the possibilities these technologies offer. In the last decade, there has been rapid growth in randomised controlled trials (RCTs) and clinical trials using remote and digital methodologies1–3. The COVID-19 pandemic has spotlighted these remote methods by emphasizing how useful and innovative they can be. While traditional or centralised trials remain invaluable and necessary, decentralised trials may offer increased or different benefits to trial participants, researchers and health professionals. Decentralised trials include any study where data are collected in an alternative manner to a centralised trial. They often use alternative data collection locations, such as mobile or local health clinics, employ digital technologies, for example, wearable devices, or leverage online methods. Decentralised trials can potentially reach people in geographically remote areas, recruit larger and more diverse populations, improve trial participant retention, and may decrease trial costs and research waste2,4,5.
Recruitment and retention of trial participants are two of the most common hurdles faced by research teams conducting trials with adults6. Paediatric trials often encounter the same hurdles as those involving adults and have additional challenges, especially those of a logistical and ethical nature. In their Cochrane systematic review discussing strategies to improve recruitment in trials, Treweek et al. identified the lack of, and critical need for, the “development and evaluation”6(p22) of strategies for recruitment to paediatric trials. Pica and Bourgeois7 found that the two most common reasons researchers conducting paediatric RCTs cited for discontinuation or non-publication of a trial were participant recruitment and “conduct problems”7(p3), which they defined as technical or logistical issues in completing the trial. They found that 19% of registered paediatric trials went unfinished, and a further 30% went unpublished, resulting in substantial research waste and undue burden to their participants7. RCTs can be onerous for participants, which can affect their willingness to enrol or remain in them8. Consent forms for RCTs are customarily long documents written in legal jargon, and participants often do not fully comprehend or read them9. Given the opportunities online trials offer both participants and researchers for increased engagement and recruitment, and decreased research waste, it is important to examine how these tools can and are directed at paediatric research.
Price et al.10 published a systematic review of online trials with adults in 2019, outlining the need for more rigorous reporting of methodologies, but after a preliminary search of MEDLINE, Epistemonikos, and the Cochrane Database of Systematic Reviews, no such reviews, systematic or otherwise, were identified for online trials involving children. Overall, paediatric trials are published less often than adult trials despite paediatric disease making up 26.2% of all-cause disability-adjusted life years11. Groff et al.11 found of 4146 published RCTs, only 14.2% (591) were conducted with children and 18.3% (761) enrolled both children and adults. This may be because working with children introduces the added challenges of gaining consent from caregivers and children, recruiting children from non-traditional family groups where all caregivers must give consent, and differing international laws regarding the age of health or digital consent. As exclusively online, decentralised trials are still not common, and those with children even less so, there is a gap in the literature describing the methods and processes employed by them. This scoping review will identify, describe, and characterize how online, decentralised trials with children are conducted. The objectives of this scoping review are: (a) to determine what methods and tools have been used to create and conduct online trials with children and (b) to identify the gaps in the knowledge in this field.
Early in the field, Bull et al.12 conducted a randomised trial via social media with 16- to 25- year-olds. The researchers used multiple forms of recruitment, including online personal channels, blogs, websites, and in-person recruitment efforts and then conducted the rest of the trial online12. This type of research is termed ‘hybrid’, meaning it is conducted partially online and partially through other channels. Systematic reviews have reported on internet-based trials developed for young people and adolescents to address tobacco-, alcohol-, and drug- use, but many of these have an offline component, such as recruitment or consent acquisition4,13. Khan et al.14 published a systematic review of web-based interventions for children and young people with neurodevelopmental disorders and identified ten papers that met their inclusion criteria. Most of these trials were also hybrid as they recruited predominantly from clinics, but they demonstrate the trend towards decentralised trial development14.
While there are numerous publications from ‘hybrid’ trials, a growing body of work is being done entirely online. Notably, Rhodes et al.3 published recently on three of the studies they are developing or have already completed via their website ‘Discoveries Online’15. The paper describes their methods for overcoming the barriers involved in conducting their online studies using a webcam and participants’ guardians uploading their data for analysis by the research team3. Similarly, both MIT and Yale have developed websites for parents and their young children to take part in online trials, which can be accessed at the ‘LookIt’16 and the ‘Child Lab’ websites17, respectively. The website ‘Children Helping Science’18 hosts online trials from multiple groups, mostly academic, for parents and children to access and take part in. Gijbels et al.19 also reported on three ‘researcher-moderated’ virtual studies discussing the various elements needed to produce valid, trustworthy results. Although some trials will never lend themselves to being entirely online, the number that are accomplishing this is swiftly increasing.
Given the exploratory nature of our objectives, a scoping review is the most appropriate approach for this study20. The protocol’s organization and development are informed by the Preferred Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR)21, the JBI Manual for Evidence Synthesis’ scoping review chapter22, and Peters et al.23 and addresses the following steps: identifying a research question, identifying relevant studies, selecting studies, charting the data, and collating, summarising and reporting results. This scoping review protocol is registered with Open Science Framework (https://doi.org/10.17605/OSF.IO/WHJXY). A preprint of the protocol is available at medRXiv: doi: https://doi.org/10.1101/2022.04.15.22273742.
This scoping review intends to answer the question, “How have online, randomised trials been conducted with children?” The question was structured using the PCC mnemonic (Population, Concept, Context). The population is children, the concept is randomised and quasi-randomised trials, and the context is internet-based interventions. There had been no synthesis of these studies published at the time of our initial scoping search. For this study, children are defined as anyone under 18 years of age as per the UN General Assembly on The Rights of The Child24. We will describe the processes of recruitment and retention, ethics, randomisation, intervention, and data collection and analysis within included trials. Once described, we will categorise and summarise these methods.
The search strategy was constructed following the guidance of the JBI Manual for Evidence Synthesis22 and the PRISMA extension for Scoping Reviews25. One author (SL) undertook an initial scoping search of MEDLINE and the Cochrane Central Register of Controlled Trials (CENTRAL) to identify articles on the topic. The keywords, free text and indexed terms of relevant articles were used to develop a preliminary search strategy for MEDLINE (Ovid). This draft was then edited and refined by a different author (AC), who is trained as an information and knowledge translation specialist. The goal of the search strategy is to balance both specificity and sensitivity to capture as many relevant studies as possible and work within the constraints of time and resources available for this review.
The databases used for our search are MEDLINE (Ovid), Embase (Elsevier), CINAHL (Ebsco), and the Cochrane Central Register of Controlled Trials (CENTRAL). This is a new and rapidly developing field; therefore, we will perform a grey literature search of the following trial registries: the World Health Organization’s (WHO) International Clinical Trials Registry Platform (ICTRP), the EU Clinical Trials Register, and the NIH Clinical Trials Register. Two authors (NG & PW), experts in the fields of decentralised trials and digital health, respectively, will consult on the grey literature searches to identify trials not captured by traditional databases. We will also search Google and Google Scholar and manually screen the reference lists of all included trials for additional studies. To identify publications still in the review process we will search the preprint servers: medRxiv, JMIR Preprints, HRB Open Research, and Advance from SAGE.
The current search strategy, including all identified keywords, free text, and thesauri terms (MeSH, Emtree, CINAHL headings), has been translated for Embase, CINAHL and CENTRAL and will be adapted for all other information sources iteratively. We initially used the Polyglot translator tool to assist with translations of the search strategy from one database to another26. This automated translator has shown that it can decrease the time spent translating search strategies from one database to another27. We amended the translation in a two-step process where one author (SL) made the first modifications and then a second author (AC) further revised the translations. The cost of translation services is not within the budget for this review, so included studies will be limited to those published in the English language. We will not limit included studies by date or country of conduct. The search strategy devised for MEDLINE (Table 1) may be modified as the review develops and will be reported in the final review.
Ovid MEDLINE(R) ALL <1946 to February 16, 2022>
Following the search, we will import all identified citations into EndNote2028 reference management software, collate the libraries, and remove duplicates. The titles and abstracts will then be uploaded into the SysRev29 digital tool for screening. Two authors will perform an initial pilot test for inter-rater reliability (IRR) on 100 titles and abstracts. Inter-rater reliability will be calculated using Cohen’s kappa30. Once an agreement of 0.75 is achieved, the remaining titles and abstracts will be screened independently by one author (SL). The decision for a single reviewer to screen the remaining titles and abstracts was made in the interest of resource conservation. Taking into consideration an IRR of 0.75, the single reviewer’s familiarity with the objectives of the review, and the inherent exploratory nature of scoping reviews, we believe this is a valid and robust approach.
The next step in the selection process will be to identify full-text studies that meet the inclusion criteria. SysRev29 will also be used for full-text screening. A pilot test for this step will be completed independently by two authors for inter-rater reliability using Cohen’s kappa30. We will screen 50 full-texts and inclusion criteria may be modified and discussed during this process, and a third author consulted if needed. Once an IRR of 0.75 or greater is reached, one author (SL) will screen the remaining full-text studies independently in the interest of resource and budgetary constraints. Inter-rater reliability tests will be completed for 50 full text studies for every 1000 studies screened.
Inclusion criteria
♦ Randomised and quasi-randomised trials
♦ Trials with participants under the age of 18
♦ Trials enrolling participants under the age of 18 at trial commencement who turned 18 during the trial
♦ Trials where participants were an age-range cohort (for example 16–25-year-olds) and ≥60% of participants were under the age of 18
♦ Trials that enrolled caregiver/child(ren) dyads or groups where the children were active participants who contributed data
♦ Entirely online trials using digital technologies connected to the internet in each phase of the trial (For example, recruitment occurred via social media site advertisements and notice-board advertisements in primary care physicians’ offices which directed participants to a study website to enrol)
♦ Trials that implemented self-managed interventions that were offline, where the participants uploaded data via the internet (For example, if a video of a child performing a task is uploaded and then assessed by the researchers, or if a child performs 10 jumping jacks before doing their homework and then reports online on their ability to focus on their homework)
♦ Trials that advertised digitally to a school, clinic, or similar, inviting children and/or caregivers to participate on an individual basis
Exclusion criteria
♦ Trials where both caregivers and children were participants, but data was only gathered from caregivers (For example, where a behavioural intervention is implemented and the caregiver submits observations online but the child has no data input)
♦ Trials that employed any intervention that required clinical or researcher measurements offline or in-person
♦ Studies that used online interventions in educational, clinical, or similar group settings where the participants were recruited or interventions were overseen in-person (For example, an entire class takes part in a mandatory online intervention during school hours with the oversight of an educator or researcher)
♦ Posters and conference abstracts due to the paucity of method reporting
Exclusions of full-text trials and their justifications will be recorded and reported. The search results will be reported in full in the scoping review and presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping reviews (PRISMA-ScR) flow diagram25.
We will chart the data from included publications with a data charting tool developed by the authors (Table 2). Two reviewers will pilot and modify the tool using five randomly selected studies, iteratively. The reviewers will then discuss and agree on modifications needed to develop a final data charting tool. In the interest of resource conservation and given review purpose, a single reviewer will chart the remaining studies’ data.
We will conduct a risk of bias (RoB) assessment of included studies using the RoB1 tool outlined in the Cochrane Handbook for Systematic Reviews of Interventions31. This tool assesses selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases. To decrease the time spent assessing RoB, we will use RobotReviewer from Vortext Systems32. This freely available software, uses machine learning to semi-automate RoB assessments. RobotReviewer has shown an ability to decrease the workload of large reviews33. The creators of RobotReviewer stipulate that although machine learning can be extremely useful, it should be validated by the people using it33. We will validate results with two reviewers manually assessing RoB from the five original studies used for the extraction pilot and compare them to those from RobotReviewer32. Once we reach an IRR of 0.75 using Cohen’s kappa30, a single reviewer will complete the remaining studies’ RoB.
We will summarise the charted data and present it in the text of the full report and in a tabular fashion as it addresses the review’s objectives. Representations of digital tools and platforms used for each phase of the trials, participants' ages and characteristics, follow-up or retention methods used, consent verification methods used, and results of interest will likely be best presented in graphical form. However, because of the iterative nature of this review and breadth of the studies, other forms of data depiction may develop in the final report. It is notable that the aim of a scoping review is not to report or analyse the results of included studies and that we will collate results as they apply in the context of the review question.
This scoping review aims to identify and characterize how online randomised trials with children have been designed and implemented, and identify the gaps in the research. The review will employ a thorough and robust search strategy, chart the desired data, and summarise it in order to establish how these trials are managed. Our aim is that this review will allow for recommendations for future research directions and help guide researchers to improve the execution of these trials.
The challenges faced by researchers in these types of trials differ depending on the participants’ ages, the interventions used, and the primary objectives of the studies. Sensitivity, technological knowledge, and forethought are required to protect the child and find the appropriate digital tools for recruitment. In trials where maintaining the child's autonomy is critical, it can be a complex balance also to involve the caregivers to ensure legal and ethical integrity. Collecting and validating unique, individual data sets digitally requires additional verification steps compared to what a centralised trial might demand. Finally, and arguably most importantly, confirming informed consent and assent from both caregivers and children online calls for transparency, caution, and creativity in some cases. These additional challenges can be daunting for researchers to tackle; however, trialists have made innovative advances in online research with children in the last decade.
Our interest in this field stems from the ability of decentralised trials to reach a larger participant base, potentially decrease research waste, and expand inclusivity in paediatric trials. However, we acknowledge that internet-based trials inherently exclude some populations. There remains a digital divide through either disadvantage, infrastructure availability, or personal choice. We have chosen to focus on entirely online, decentralised trials to describe their benefits and challenges, and it is outside the scope of this work to address and include trials that are not internet-based.
Open Science Framework: PRISMA-ScR Checklist, and preliminary PRISMA 2020 Flow Diagram, for “Online randomised trials with children: A scoping review protocol”, https://doi.org/10.17605/OSF.IO/WHJXY34.
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Yes
Competing Interests: I am one of the lead investigators in a project called CHISEL that seeks to improve methods in paediatric trials. That project is independent from the paper reviewed here, and while it has a focus on paediatric trial methods it does not have focus on online methods specifically. Beyond being in the same space of research (child health trials) there is no further conflict of interest or connection.
Reviewer Expertise: Child health, complex intervention methods, evidence synthesis methods
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Information for patients, including information about trials.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 1 20 Jun 22 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with HRB Open Research
Already registered? Sign in
Submission to HRB Open Research is open to all HRB grantholders or people working on a HRB-funded/co-funded grant on or since 1 January 2017. Sign up for information about developments, publishing and publications from HRB Open Research.
We'll keep you updated on any major new updates to HRB Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)