A concept analysis of <i>‘trial recruitment’ </i>using the hybrid model – Phase 1 findings

Hannah Delaney; Declan Devane; Andrew Hunter; Shaun Treweek; Nicola Mills; Carrol Gamble; Valerie Smith

doi:10.12688/hrbopenres.13173.1

Home Browse A concept analysis of ‘trial recruitment’ using the hybrid model –...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Note

A concept analysis of ‘trial recruitment’ using the hybrid model – Phase 1 findings

[version 1; peer review: 2 approved with reservations]

Hannah Delaney^1-3, Declan Devane^1,2, Andrew Hunter^2,4, [...] Shaun Treweek⁵, Nicola Mills⁶, Carrol Gamble⁷, Valerie Smith³

Hannah Delaney^1-3, Declan Devane^1,2, [...] Andrew Hunter^2,4, Shaun Treweek⁵, Nicola Mills⁶, Carrol Gamble⁷, Valerie Smith³

PUBLISHED 21 Dec 2020

Author details Author details

¹ Health Research Board - Trials Methodology Research Network, National University of Ireland, Galway, Galway, Ireland
² School of Nursing and Midwifery, National University of Ireland, Galway, Galway, Ireland
³ School of Nursing and Midwifery, Trinity College Dublin, Dublin, Ireland
⁴ Qualitative Research in Trials Centre (QUESTS), National University of Ireland, Galway, Galway, Ireland
⁵ Health Services Research Unit, University of Aberdeen, Aberdeen, UK
⁶ Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
⁷ Clinical Trials Research Centre, University of Liverpool, Liverpool, UK

Hannah Delaney
Roles: Conceptualization, Data Curation, Formal Analysis, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Declan Devane
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Andrew Hunter
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Writing – Original Draft Preparation, Writing – Review & Editing

Shaun Treweek
Roles: Conceptualization, Writing – Review & Editing

Nicola Mills
Roles: Conceptualization, Writing – Review & Editing

Carrol Gamble
Roles: Conceptualization, Writing – Review & Editing

Valerie Smith
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: The International Committee of Medical Journal Editors (ICMJE) requires trials submitted for publication to be registered before enrolment of the first participant; however, there is ambiguity around the definition of recruitment and in anchoring the trial start date, end date, recruitment and enrolment, temporally to trial processes. There is potential for variation in how recruitment is reported and understood in trial protocols and trial reports. We report on Phase 1 of a concept analysis of ‘trial recruitment’ and develop a preliminary operational definition of ‘trial recruitment’.
Methods: A concept analysis using the hybrid model. We searched randomised and non-randomised trial reports published between January 2018 and June 2019. Included studies were sourced from the five top journals in the category of medicine with the highest impact factor. We examined how recruitment was defined temporally to four time points; screening, consent, randomisation, and allocation.
Results: Of the 150 trial reports analysed, over half did not identify a clear time point of when recruitment took place in relation to any of screening/consent/randomisation/allocation. The majority of the assessed trials provided a time frame in relation to the trial (i.e. start/end date), the process that this time frame referred to differed between studies. There was variation across studies in the terminology used to describe entry to the trial and often multiple terms were used interchangeably.
Conclusion: There is ambiguity around temporal descriptions of ‘trial recruitment’ in health care journals. Informed by the findings of Phase 1, we developed a preliminary temporal operational definition of trial recruitment based on i) trial recruitment of an individual or cluster and ii) the trial recruitment period. In Phase 2 this definition will be discussed in focus groups with healthcare workers involved in designing/implementing/reporting on trials; to contribute to the final phase (analytical phase) of this concept analysis.

Keywords

Concept analysis, Trial recruitment, Trial report, Trial enrolment

Corresponding author: Hannah Delaney

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board [TMRN-2017-2].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 Delaney H et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Delaney H, Devane D, Hunter A et al. A concept analysis of ‘trial recruitment’ using the hybrid model – Phase 1 findings [version 1; peer review: 2 approved with reservations]. HRB Open Res 2020, 3:92 (https://doi.org/10.12688/hrbopenres.13173.1) First published: 21 Dec 2020, 3:92 (https://doi.org/10.12688/hrbopenres.13173.1) Latest published: 01 Apr 2022, 3:92 (https://doi.org/10.12688/hrbopenres.13173.2)

Background

Non-reporting of completed trials and selective outcome reporting in trials can result in a biased assessment of the global body of evidence that inform health care decisions. Since 2004, the International Committee of Medical Journal Editors (ICMJE) has required that trials submitted for publication must be registered before the enrolment of the first trial participant. The ICJME ‘does not define the timing of first participant enrolment, but best practice dictates registration by the time of first participant consent’¹ (p. e1). In addition, the World Health Organisation’s (WHO) ‘International Standards for Clinical Trial Registries’² define prospective trial registration as ‘the registration of a trial before the recruitment of the first participant’ (p.8) and date of the first enrolment as the ‘anticipated or actual date of enrolment of the first participant’ (p.28) but the temporal relationship between an invitation to potential participant, consent and randomisation is not defined. For example, the International Standard Randomised Controlled Trial Number (ISRCTN) registry defines a recruitment start date as ‘the date, or planned date, of recruitment of the first participant to the study’³ (p. e1). The clincialtrials.gov registry refers to a study start date and defines this as ‘the estimated date on which the clinical study will be open for recruitment of participants, or the actual date on which the first participant was enrolled’⁴ (p. e1), thus separating recruitment from enrolment whereby a participant is ‘enrolled’ following completion of the informed consent process.

In summary, ambiguity in anchoring the trial start date, end date, recruitment and enrolment temporally to trial processes (e.g. invitation, consent, and randomisation) has the potential for variation in how recruitment is reported and understood in trial registries, trial protocols and trial reports.

Aim

To report Phase 1 of a concept analysis of ‘trial recruitment’ using the hybrid model⁵. The aim of Phase 1 is to develop a preliminary operational definition of ‘trial recruitment’, which is then further explored and refined following Phases 2 and 3.

Methods

Study design

A concept analysis typically involves synthesising evidence on a concept and distinguishing it from other similar/related concepts to help resolve inconsistencies in the knowledge base⁶. Concept analysis offers a means of defining or clarifying concepts, contextually, while also assisting to elucidate patterns of usage which can become a precursor of theory and knowledge development⁷. There are various methods available for formal concept analyses⁸. We chose Schwartz-Barcott and Kim’s hybrid model to analyse the concept of ‘trial recruitment’, because it is considered beneficial in helping resolve ambiguity surrounding a concept and is facilitative of concept expansion and purification⁹. The model consists of three major phases; 1) the theoretical phase, 2) the fieldwork phase, and 3) the analytic phase (Figure 1). This paper reports on Phase 1.

Figure 1. Phases of the hybrid model of concept analysis⁵.

Searching the literature

The theoretical phase aims to comprehensively source and analyse relevant literature to acquire a deep understanding of the concept under study; that is, how the concept has been defined, used, and ways that it has been or might be measured⁵. To gain a contemporary understanding of the concept of ‘trial recruitment,’ we searched randomised (parallel, cluster, and other randomised designs, including pilot and feasibility trials) and non-randomised (i.e. quasi) trial reports published between January 2018 and June 2019. Included studies were sourced from the five top journals in the category of medicine¹⁰ that had the highest impact factor (Table 1). We excluded trial protocols, studies reporting secondary analyses of original/primary trial data, trials not yet started, ongoing studies, meta-analyses/systematic reviews and single-arm studies.

Table 1. Top five impact factor medical journals 2019¹⁰.

Journal Title	Impact factor (2019)
New England Journal of Medicine (N Engl J Med)	55.873
Lancet	45.217
Journal of the American Medical Association (JAMA)	35.289
Annals of Internal Medicine (Ann Intern Med)	17.81
British Medical Journal (BMJ)	17.445

The search strategy (available as Extended data¹¹) was executed in June 2019, using the Cochrane Collaboration’s EMBASE ‘trial’ search string¹² combined with the respective journal titles, and limited by year 2018-2019 and ‘article’ publication type.

Dealing with meaning and measurement

The following data were extracted and used to analyse the concept of ‘trial recruitment’; study characteristics (data source, the aim of the study, location of study, and health condition); implicit or explicit temporal descriptions and definitions of the trial start date, end date, trial duration, gaining consent, recruitment, enrolment, and randomisation. Once data were extracted, significant points of contrast and similarity were explored. This type of comparison gives the researcher an insight into the degree of consensus among users of the concept of ‘trial recruitment’ and can help ascertain the degree of intersubjectivity of meaning⁵. Anticipating that few explicit definitions of trial recruitment might exist, Schwartz-Barcott and Kim recommend analysis of the authors’ writings to determine implied definitions of the concept under study, using the format given in Table 2 as a guide⁵.

Table 2. Sample format for organising and analysing definitions⁵.

Reference	Explicit	Implicit	Examples	Comments

Data analysis

The CONSORT flow diagram¹³ recommends that five main time points should be reported when presenting the progress of participants through a trial (enrolment, randomisation, allocation, follow-up and analysis). As we were concerned explicitly with recruitment in this analysis, we focused on enrolment, randomisation and allocation. For our analysis, we examined how recruitment was defined temporally to four time points, these are aligned with the CONSORT time points; 1. screening (i.e. CONSORT enrolment/eligibility assessment), 2. consent (i.e. CONSORT enrolment/exclusion), 3. randomisation and 4. allocation (see Figure 2).

Figure 2. CONSORT flow diagram¹³ - edited to include the four time points analysed for this study.

Findings of Phase 1

Results of the search

Searches yielded 2867 records, and no duplicates were found. Following title and abstract screening, 1659 records were excluded based on our predefined inclusion and exclusion criteria. Given the scope of our inclusion criteria, we were confident that the majority of the 1208 records would be included following full-text screening. For this reason, we decided to do full-text screening and data extraction concurrently, dividing the papers equally between three authors (HD, VS and AH). After piloting the data extraction form with a subset of 10 of the 1208 records, and considering the similar reporting format across the five included journals, we selected a 20% random sample of records from each of the five journals, resulting in the inclusion of 241 studies on which to base the concept analysis (see Extended data¹¹). Although we anticipated extracting data from all 241 included studies, at 150 records we had reached a point where no further novel data were being captured (see Figure 3 for further details). For this reason, we concluded data extraction with these 150 trial reports and based the theoretical analysis on the data extracted from these 150 records as we believed this offered data sufficiency in meeting the aim of Phase 1 of this concept analysis. We recognise, however, in omitting the additional 91 records, that the proportions reported in our findings may have been impacted on but not necessarily on the overall conclusions derived from the analysis. 148 of the records reported on randomised trials and two reported on non-randomised trials.

Figure 3. PRISMA flow diagram¹⁴

Characteristics of included studies

Of the included trials, 21 reported on trials in oncology, with the remaining trials reporting in the medical areas of: cardiology (n=6), psychiatry (n=6), diabetes (n=5), stroke (n=5), dermatology (n=5), HIV (n=5), paediatrics (n=4), ophthalmology (n=4), and other (n=89) (see Extended data¹¹). The majority of trials were carried out in multiple countries (n=55). Twenty-nine were based in America and 15 in the United Kingdom. The remaining trials were conducted in: Asia (n=9), Australia (n=7), Africa (n=5), Netherlands (n=5), France (n=5), Germany (n=4), Switzerland (n=3), Norway (n=2), Canada (n=2), not stated (n=3), and one in each of Hong Kong, Ireland, Poland, Portugal, Russia, South Africa.

Temporal descriptions of ‘recruitment’

Of the 150 trials analysed, over half (n=76) did not identify a clear time point of when recruitment took place in relation to any of screening, consent, randomisation, allocation (see Figure 4). Twenty-five of the trial reports referred to recruitment as taking place after consent and before randomisation (explicit n=15, implicit n=10); 21 as the point between screening and randomisation (explicit n=10, implicit n=11) with the timing of consent unspecified; and nine referred to recruitment as the point between screening and consent (explicit n=3, implicit n=6). The remaining trials defined recruitment at the time-point before screening (n=5, 3 explicit and 2 implicit); between randomisation and allocation (n=1, explicit). Three studies referred to recruitment generally as including screening, consent and randomisation (explicit n=1, implicit n=2), 10 were categorised as ‘other’: in seven of these trial reports the order of trial processes differed to the order identified in the CONSORT flow diagram and three trials referred to recruitment taking place at randomisation, but the timing of randomisation was unclear.

Figure 4. At what time point is recruitment defined?

The majority of the assessed trials (n=138) provided a time frame in relation to the trial (i.e. start and end date); however, the process that this time frame referred to differed between studies (see Table 3). For instance, 24 studies included the start and end date of the duration of the trial such as¹⁵; '...multicentre phase 3 trial was conducted from August 4, 2011, to June 20, 2017’ (p.599). Twenty-two studies stated the start and end date of the randomisation period, such as¹⁶; 'Between Oct 1, 2012, and June 20, 2014, we randomly assigned 155 participants...' (p.41). Others included dates between which ‘enrolment’ (n=18), ‘recruitment’ (n=15), and ‘screening’ (n=13) took place. Forty of the trials reported on the start and end date of multiple processes, for instance:

Table 3. Reported start and end date.

Reported on the start and end date for:	Total studies
Multiple recruitment processes (i.e. the time frame provided referred to more than one process, such as ‘enrolment and randomisation’)	40
Trial duration (i.e. providing a start and end date for the ‘study’ or ‘trial’ period)	24
Period of randomisation (i.e. reporting the start and end date for when ‘randomisation’ took place)	22
Period of enrolment (i.e. reporting the start and end date for when ‘enrolment’ took place)	18
Period of recruitment (i.e. reporting the start and end date for when ‘recruitment’ took place)	15
No start/end date reported	12
Screening period (i.e. reporting the start and end date for when ‘screening’ took place)	13
Other (i.e. reporting a time frame for trial processes not related to ‘recruitment’ such as data collection and rounds of treatment)	6
Total	150

‘During the study period (August 2015 and May 2017), 151 patients were screened, 117 underwent randomization’¹⁷ (p.2301)
‘Between July 2, 2013, and May 10, 2016, 80 patients were enrolled, randomly assigned, and started their allocated treatment'¹⁸ (p.328)

The studies categorised as ‘other’ (n=6) reported on the start and end date of other processes such as data collection (n=1), rounds of treatment (n=2), and the use of the same start and end date with differing terminology (n=3); for instance, enrolment and recruitment were used interchangeably. Further findings on the variation in language are presented in the next section.

Variation in terminology

There was variation across the studies in the terminology used to describe entry (the point at which a participant was considered to have ‘joined’ a trial) to the trial (see Table 4). Of the 150 analysed trials, just over a third (n=52) used the term ‘enrolment’, and 34 did not use a specific term to describe entry to the trial. Thirty studies used multiple terms; this was mostly in the form of ‘recruitment’ used interchangeably with another term such as ‘enrolment’ (n=19), ‘randomisation’ (n=3), randomisation and enrolment (n=2), screening (n=1), screening and randomisation (n=1). Other studies used the term ‘randomisation’ interchangeably with ‘accrued’ (n=1) and ‘enrolment’ (n=1). One study used the terms ‘included’ and ‘enrolment’ interchangeably. Table 4 and Table 5 illustrate the variation across the studies in the terminology used to describe the entry of participants to the trial.

Table 4. Terminology.

Term used to describe entry to the trial	Total studies
Enrolment	52
No specific term used	34
Multiple terms used	30
Recruitment	29
Other	5
Total	150

Table 5. Variation in terminology.

Journal [reference]	Healthcare area	Description
N Engl J Med¹⁹	Lung Disease	‘863 infants were enrolled during the period from April 2010 through August 2013’(p.149)
Lancet²⁰	Osteoperosis	‘...we enrolled post-menopausal women with at least two moderate or one severe vertebral fracture and a bone mineral density’ (p.30) ‘We enrolled 680 patients in each group…’ (p.30)
Lancet²¹	Oncology	‘Of 601 patients assessed for eligibility, a total of 452 patients… were recruited and randomly assigned’ (p.233) ‘...601 patients assessed for eligibility, of whom 452 patients were enrolled and 226 were randomly assigned ’ (p.229)
JAMA²²	Anaesthesia	'Patients undergoing anaesthesia with RSI were enrolled from February 2014 until February 2017...' (p.E1) '...Recruitment began in February 2014 and ended in February 2017’ (p.E2)
Lancet²³	Adolescent health	‘Of the 112 eligible schools, 75 were randomly selected to participate in the trial...' (p.2471) '...we recruited 75 schools' (p.2471)
JAMA²⁴	Retinopathy of prematurity	‘Patients were recruited between September 2014 and August 2016. 20 infants were screened and 19 were randomized’…(p.278) ‘20 patients were screened and 19 were enrolled' (p.279)
Lancet²⁵	Inflammatory diseases	‘Between Oct 6, 2015, and Nov 30, 2016, 166 patients were screened, of whom 102 were randomly assigned ...’ (p.1330) ‘...Patients were recruited between Oct 6, 2015, and Nov 30, 2016’ (p.1335)
BMJ Open²⁶	Critical care	‘...an enrolment of 114 patients was planned....’ (p.3) ‘One hundred fourteen patients were included in this study with 57 patients randomised in each group’ (p.3)

Conclusion and working definition for Phase 2

The theoretical Phase 1 of our concept analysis has revealed that there is ambiguity around temporal descriptions of ‘trial recruitment’ in health care journals, and varying terminology is used when reporting on trial recruitment.

Sixty-one of the analysed trials identified a time point, in relation to the four main trial processes (screening, consent, randomisation, allocation), at which trial recruitment took place. The majority of these studies identified trial recruitment as being between consent and randomisation or between screening and randomisation (with time of consent unclear) as the time point of actual recruitment. Over half of the trials analysed (n=76) did not identify a clear time point of when trial recruitment took place. Our analysis also revealed a variation in terminology used to describe entry to the trial, and often multiple terms were used interchangeably. Enrolment (n=52) and recruitment (n=29) were the most common terms used, but the use of numerous terms was also frequent in the trial reports (n=30).

There are some limitations of this study to be noted. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed here are randomised trials. However, we included both randomised and non-randomised trials in our search strategy and the selection of trials for inclusion in analysis was based on a random sample of records from each of the five journals.

Considering these findings, we have developed a preliminary temporal operational definition of trial recruitment based on i) trial recruitment of an individual or cluster as ‘the time point after screening and consent and before randomisation’ and ii) trial recruitment period as ‘the time point after screening and consent of the first participant, and before randomisation of the last participant’. According to Schwartz-Barcott and Kim⁵, once an initial definition is developed from the findings of the theoretical phase of the analysis, a ‘further detailed examination’ of the definition is required. This occurs in Phase 2 during fieldwork. Our definition from Phase 1 will be discussed in focus groups with healthcare workers involved in designing, implementing and reporting on trials. Further discussion around temporal descriptions and reporting of ‘trial recruitment’ will take place during these focus groups. The use of varying terminology when reporting on trial recruitment will also be further explored in the focus group discussions. The findings from the focus groups will be combined with the findings as above, in the final phase of this concept analysis process; Phase 3, the analytical phase.

Data availability

Underlying data

Figshare: https://doi.org/10.6084/m9.figshare.13109870.v1¹¹

This project contains the following underlying data:

Delaney et al. 2020_Concept Analysis_Extracted Data.xlsx

Extended data

Figshare: https://doi.org/10.6084/m9.figshare.13109870.v1¹¹

This project contains the following extended data:

Delaney et al. 2020_search strategy.pdf
Delaney et al. 2020_records per journal.pdf
Delaney et al. 2020_characteristics of included studies.pdf

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Faculty Opinions recommended

References

1. International Committee of Medical Journal Editors: Publishing and Editorial Issues Related to Publication in Medical Journals – Clinical Trials. Reference Source
2. World Health Organisation: International standards for clinical trial registries 2012. Reference Source
3. International Standard Randomised Controlled Trial Number (ISRCTN) registry: Definitions. Reference Source
4. Clinicaltrials.gov Registry: Protocol Registration Data Element Definitions for Interventional and Observational Studies. Reference Source
5. Schwartz-Barcott D, Kim HS: Chapter 9: An expansion and elaboration of the hybrid model of concept development. In: Rodgers BL, Knafl KA. (eds.) Concept Development in Nursing. Foundations, Techniques and Applications. Second Edition, Philadelphia: WB. Saunders; 2000; 129–159.
6. Knafl K, Deatrick J: Chapter 2: Knowledge Synthesis and Concept Development in Nursing. In: Rodgers BL, Knafl KA. (eds.) Concept Development in Nursing. Foundations, Techniques and Applications. Second Edition, Philadelphia: WB. Saunders; 2000; 39–54.
7. Cronin P, Ryan F, Coughlan M: Concept analysis in healthcare research. Int J Ther Rehabil. 2010; 17(2): 62–68. Publisher Full Text
8. Beecher C, Devane D, White M, et al.: Concept development in Nursing and Midwifery: An overview of methodological approaches. Int J Nurs Pract. 2019; 25(1): e12702. PubMed Abstract | Publisher Full Text
9. Jasemi M, Valizadeh L, Zamanzadeh V, et al.: A Concept Analysis of Holistic Care by Hybrid Model. Indian J Palliat Care. 2017; 23(1): 71–80. PubMed Abstract | Publisher Full Text | Free Full Text
10. Impact Factors: Medical Journal Impact Factors 2019. [Accessed 26^th June 2019]. Reference Source
11. Delaney H: A Concept Analysis of trial recruitment. Extended Data. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.13109870.v1
12. Lefebvre C, Manheimer E, Glanville J: Chapter 6: Searching for studies. In: Higgins J, Green S (eds). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. updated March 2011. The Cochrane Collaboration, 2011. Reference Source
13. Schulz KF, Altman DG, Moher D, et al.: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340: c332. PubMed Abstract | Publisher Full Text | Free Full Text
14. Moher D, Liberati A, Tetzlaff J, et al.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text
15. Tinnitus Retraining Therapy Trial Research Group, Scherer RW, Formby C: Effect of Tinnitus Retraining Therapy vs Standard of Care on Tinnitus-Related Quality of Life: A Randomized Clinical Trial. JAMA Otolaryngol Head Neck Surg. 2019; 145(7): 597–608. PubMed Abstract | Publisher Full Text | Free Full Text
16. Fowler D, Hodgekins J, French P, et al.: Social recovery therapy in combination with early intervention services for enhancement of social recovery in patients with first-episode psychosis (SUPEREDEN3): a single-blind, randomised controlled trial. Lancet Psychiatry. 2018; 5(1): 41–50. PubMed Abstract | Publisher Full Text | Free Full Text
17. Raghu G, van den Blink B, Hamblin MJ, et al.: Effect of Recombinant Human Pentraxin 2 vs Placebo on Change in Forced Vital Capacity in Patients With Idiopathic Pulmonary Fibrosis: A Randomized Clinical Trial. JAMA. 2018; 319(22): 2299–2307. PubMed Abstract | Publisher Full Text | Free Full Text
18. Wildiers H, Tryfonidis K, Dal Lago L, et al.: Pertuzumab and trastuzumab with or without metronomic chemotherapy for older patients with HER2-positive metastatic breast cancer (EORTC 75111-10114): an open-label, randomised, phase 2 trial from the Elderly Task Force/Breast Cancer Group. Lancet Oncol. 2018; 19(3): 323–336. PubMed Abstract | Publisher Full Text
19. Bassler D, Shinwell ES, Hallman M, et al.: Long-Term Effects of Inhaled Budesonide for Bronchopulmonary Dysplasia. N Engl J Med. 2018; 378(2): 148–157. PubMed Abstract | Publisher Full Text
20. Kendler D, Marin F, Zerbini CAF, et al.: Effects of teriparatide and risedronate on new fractures in post-menopausal women with severe osteoporosis (VERO): a multicentre, double-blind, double-dummy, randomised controlled trial. Lancet. 2018; 391(10117): 230–240. PubMed Abstract | Publisher Full Text
21. Horwitz S, O’Connor OA, Pro B, et al.: Brentuximab vedotin with chemotherapy for CD30-positive peripheral T-cell lymphoma (ECHELON-2): a global, double-blind, randomised, phase 3 trial. Lancet. 2019; 393(10168): 229–240. PubMed Abstract | Publisher Full Text | Free Full Text
22. Birenbaum A, Hajage D, Roche S, et al.: Effect of Cricoid Pressure Compared With a Sham Procedure in the Rapid Sequence Induction of Anesthesia: The IRIS Randomized Clinical Trial. JAMA Surg. 2019; 154(1): 9–17. PubMed Abstract | Publisher Full Text | Free Full Text
23. Shinde S, Weiss HA, Varghese B, et al.: Promoting school climate and health outcomes with the SEHER multi-component secondary school intervention in Bihar, India: a cluster-randomised controlled trial. Lancet. 2018; 392(10163): 2465–2477. PubMed Abstract | Publisher Full Text
24. Stahl A, Krohne TU, Eter N, et al.: Comparing Alternative Ranibizumab Dosages for Safety and Efficacy in Retinopathy of Prematurity: A Randomized Clinical Trial. JAMA Pediatr. 2018; 172(3): 278–286. PubMed Abstract | Publisher Full Text | Free Full Text
25. van Vollenhoven RF, Hahn BH, Tsokos GC, et al.: Efficacy and safety of ustekinumab, an IL-12 and IL-23 inhibitor, in patients with active systemic lupus erythematosus: results of a multicentre, double-blind, phase 2, randomised, controlled study. Lancet. 2018; 392(10155): 1330–1339. PubMed Abstract | Publisher Full Text
26. Bridey C, Thilly N, Lefevre T, et al.: Ultrasound-guided versus landmark approach for peripheral intravenous access by critical care nurses: a randomised controlled study. BMJ Open. 2018; 8(6): e020220. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2020

Author details Author details

Hannah Delaney
Roles: Conceptualization, Data Curation, Formal Analysis, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Declan Devane
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Andrew Hunter
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Writing – Original Draft Preparation, Writing – Review & Editing

Shaun Treweek
Roles: Conceptualization, Writing – Review & Editing

Nicola Mills
Roles: Conceptualization, Writing – Review & Editing

Carrol Gamble
Roles: Conceptualization, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board [TMRN-2017-2].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 01 Apr 2022, 3:92

https://doi.org/10.12688/hrbopenres.13173.2

version 1

Published: 21 Dec 2020, 3:92

https://doi.org/10.12688/hrbopenres.13173.1

© 2020 Delaney H et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

1038

downloads

Citations

SEE MORE DETAILS

CITE

how to cite this article

Delaney H, Devane D, Hunter A et al. A concept analysis of ‘trial recruitment’ using the hybrid model – Phase 1 findings [version 1; peer review: 2 approved with reservations]. HRB Open Res 2020, 3:92 (https://doi.org/10.12688/hrbopenres.13173.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 21 Dec 2020

Views

Reviewer Report 09 Feb 2021

Jennifer Hanratty, Centre for Evidence and Social Innovation, Queen's University Belfast, Belfast, UK; Campbell UK & Ireland, Queen's University Belfast, Belfast, UK

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14299.r28734

As a systematic review expert with experience of evaluating and conducting trials I am very well qualified to review this paper and I thank the authors for an interesting read.

This study analysed reporting of trial recruitment timing in leading medical journals and found that reporting is unclear, incomplete or absent. This work is a timely reminder that incomplete or unclear reporting in RCTs is an issue that requires remedy. I commend the team for thoroughly documenting the problem.

Abstract: the team uses both "enrolment" and "recruitment" interchangeably in the abstract (and the background section). Is this deliberate or should it be revised to consistently use "recruitment", or perhaps offer some explication of the difference between the two terms (if any)?

Background: This situates the work well and cites relevant and current guidance. I would like to see a statement and some elaboration on why "variation in how recruitment is reported and understood" is a problem – aside from being a major frustration for systematic reviewers. The opening line appropriately mentions bias due to non-reporting or selective reporting but this is not directly linked to reporting on recruitment. I’d really like the authors to explicitly answer the questions; why is this a problem? What are the implications of poor reporting or indeed poor practice – why does it matter?

Methods: This is very clearly presented and I found the succinct introduction to concept analysis particularly informative. I am curious as to why trial protocols were excluded as they are a vital part of the reporting and often contain supplementary detail not contained in the trial report. Could the authors explain why protocols were excluded? I would be interested to know too if authors consulted the trial protocol of included published trials to get more information on reporting of recruitment and to compare reporting between protocol and publication.

The selection of a subsample of trials is appropriate and well documented. In the PRISMA Flow diagram, box "no further novel data captured" – should this be 91 rather than 150?

The line "148 of the records reported on randomised trials and two reported on non-randomised trials." would fit better in Characteristics of included studies. I would also like to know what proportion were cluster, parallel, pragmatic RCTs and if any were feasibility studies or pilot trials to get a fuller sense of the data set. Is it possible to explore whether or not the definition of recruitment was related to the type of trial?

I suggest a change of wording "Of the 150 trials analysed, over half (n=76) did not identify a clear time point of when recruitment took place in relation to any of screening, consent, randomisation, allocation" to "Of the 150 trials analysed, over half (n=76) did not clearly identify when recruitment took place in relation to any of screening, consent, randomisation, allocation".

I prefer reporting to be consistent in referring to proportions, percent or number and not mixing different metrics e.g., "Of the 150 analysed trials, just over a third (n=52) used the term ‘enrolment’, and 34 did not use a specific term to describe entry to the trial." Could be "Of the 150 analysed trials, just over a third (n=52) used the term ‘enrolment’, and just over one fifth (n=34) did not use a specific term to describe entry to the trial."

Conclusions: The lack of comparison between RCTs and non-RCTs is unfortunate but unsurprising given the focus on medical journals. A better approach may have been to focus only on randomised trials or to specifically search for non-RCTs, which would likely mean expanding the search to include non-medical journals/databases to capture non-RCTs as they are less common in medicine than other areas of health and social care.

I understand that the focus is on the term "trial recruitment" but could you also consider that the term "enrolment" might have its own distinct meaning and explore this in the next phase of the study? Given that the term "enrolment" was actually more common in your data set, had you considered that enrolment might be the more useful term instead of recruitment or do you think they are two distinct terms with distinct meanings? The preliminary definition is very useful.

As above, I would like a see the authors draw out the importance of good clear reporting, why terminology is important, and preliminary recommendations for how to fix the problem – not just providing a definition but telling the reader what you might do next once you have settled on an agreed definition.

Overall, I found this article to be informative and a very useful practical approach to documenting and exploring the problem of reporting of trial recruitment. Some minor changes will enhance the quality and usefulness of the article and I look forward to reading about the next phase of this work.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Systematic reviews and evidence synthesis, evaluation (including trials).

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

01 Apr 2022

Author Response

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the ... Continue reading Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

Abstract: We now use the term ‘recruitment’ consistently in the abstract but note also that the term ‘recruitment’ is often referred to interchangeably in reports as ‘enrolment’. We have added a further sentence to the Background (Section 1.0) to explain this further.

Background: We have added a sentence to the Background to emphasize why variation in how recruitment is reported can be problematic.

Methods:
In response to your comment on the exclusion of protocols: Thank you for this important point. We did consider trial protocols, however, we wished to analyse ‘actual’ recruitment data rather than what might be planned. We have clarified this with an added sentence to the paper (end of paragraph in ‘Dealing with meaning and measurement’ section).

We have corrected Figure 3.

We have moved the line "148 of the records reported on randomised trials and two reported on non-randomised trials" to the ‘Characteristics of included studies’ section.

In response to your query about cluster, parallel and pragmatic RCT’s: Thank you for raising this important point, however, unfortunately we did not extract these specific types of data as our focus was on definition and description of recruitment irrespective of trial design. We do see, in hindsight, that this provides a limitation, and have noted this in the added text on limitations in in the Conclusion.

We have revised the wording around proportions and ‘clearly identify’ in line with your suggestions.

Conclusion:
We have now acknowledged the limitation that the type of randomised trial might also potentially influence the definition of recruitment in trial reports.

In response to your comment about ‘enrolment’: We appreciate that the terms ‘enrolment’ and ‘recruitment’ are often used interchangeably and indeed ‘enrolment’ was a commonly used term in the analysed reports. As our aim was to develop a definition for ‘recruitment,’ which is not to say that ‘recruitment’ is a more useful term than ‘enrolment’ but the hope is that by providing clarity on at least one term, this will reduce ambiguity in trial reporting, and potentially encourage those reporting on trials to consistently use ‘recruitment’ based on this definition.

In response to your comment about emphasising the importance of clear trial reporting and preliminary recommendations: We have added text to the end of the Conclusion which we hope addresses this point.
Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

Abstract: We now use the term ‘recruitment’ consistently in the abstract but note also that the term ‘recruitment’ is often referred to interchangeably in reports as ‘enrolment’. We have added a further sentence to the Background (Section 1.0) to explain this further.

Background: We have added a sentence to the Background to emphasize why variation in how recruitment is reported can be problematic.

Methods:
In response to your comment on the exclusion of protocols: Thank you for this important point. We did consider trial protocols, however, we wished to analyse ‘actual’ recruitment data rather than what might be planned. We have clarified this with an added sentence to the paper (end of paragraph in ‘Dealing with meaning and measurement’ section).

We have corrected Figure 3.

We have moved the line "148 of the records reported on randomised trials and two reported on non-randomised trials" to the ‘Characteristics of included studies’ section.

In response to your query about cluster, parallel and pragmatic RCT’s: Thank you for raising this important point, however, unfortunately we did not extract these specific types of data as our focus was on definition and description of recruitment irrespective of trial design. We do see, in hindsight, that this provides a limitation, and have noted this in the added text on limitations in in the Conclusion.

We have revised the wording around proportions and ‘clearly identify’ in line with your suggestions.

Conclusion:
We have now acknowledged the limitation that the type of randomised trial might also potentially influence the definition of recruitment in trial reports.

In response to your comment about ‘enrolment’: We appreciate that the terms ‘enrolment’ and ‘recruitment’ are often used interchangeably and indeed ‘enrolment’ was a commonly used term in the analysed reports. As our aim was to develop a definition for ‘recruitment,’ which is not to say that ‘recruitment’ is a more useful term than ‘enrolment’ but the hope is that by providing clarity on at least one term, this will reduce ambiguity in trial reporting, and potentially encourage those reporting on trials to consistently use ‘recruitment’ based on this definition.

In response to your comment about emphasising the importance of clear trial reporting and preliminary recommendations: We have added text to the end of the Conclusion which we hope addresses this point.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

01 Apr 2022

Author Response

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the ... Continue reading Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

Abstract: We now use the term ‘recruitment’ consistently in the abstract but note also that the term ‘recruitment’ is often referred to interchangeably in reports as ‘enrolment’. We have added a further sentence to the Background (Section 1.0) to explain this further.

Background: We have added a sentence to the Background to emphasize why variation in how recruitment is reported can be problematic.

Methods:
In response to your comment on the exclusion of protocols: Thank you for this important point. We did consider trial protocols, however, we wished to analyse ‘actual’ recruitment data rather than what might be planned. We have clarified this with an added sentence to the paper (end of paragraph in ‘Dealing with meaning and measurement’ section).

We have corrected Figure 3.

We have moved the line "148 of the records reported on randomised trials and two reported on non-randomised trials" to the ‘Characteristics of included studies’ section.

In response to your query about cluster, parallel and pragmatic RCT’s: Thank you for raising this important point, however, unfortunately we did not extract these specific types of data as our focus was on definition and description of recruitment irrespective of trial design. We do see, in hindsight, that this provides a limitation, and have noted this in the added text on limitations in in the Conclusion.

We have revised the wording around proportions and ‘clearly identify’ in line with your suggestions.

Conclusion:
We have now acknowledged the limitation that the type of randomised trial might also potentially influence the definition of recruitment in trial reports.

In response to your comment about ‘enrolment’: We appreciate that the terms ‘enrolment’ and ‘recruitment’ are often used interchangeably and indeed ‘enrolment’ was a commonly used term in the analysed reports. As our aim was to develop a definition for ‘recruitment,’ which is not to say that ‘recruitment’ is a more useful term than ‘enrolment’ but the hope is that by providing clarity on at least one term, this will reduce ambiguity in trial reporting, and potentially encourage those reporting on trials to consistently use ‘recruitment’ based on this definition.

In response to your comment about emphasising the importance of clear trial reporting and preliminary recommendations: We have added text to the end of the Conclusion which we hope addresses this point.
Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

Abstract: We now use the term ‘recruitment’ consistently in the abstract but note also that the term ‘recruitment’ is often referred to interchangeably in reports as ‘enrolment’. We have added a further sentence to the Background (Section 1.0) to explain this further.

Background: We have added a sentence to the Background to emphasize why variation in how recruitment is reported can be problematic.

Methods:
In response to your comment on the exclusion of protocols: Thank you for this important point. We did consider trial protocols, however, we wished to analyse ‘actual’ recruitment data rather than what might be planned. We have clarified this with an added sentence to the paper (end of paragraph in ‘Dealing with meaning and measurement’ section).

We have corrected Figure 3.

We have moved the line "148 of the records reported on randomised trials and two reported on non-randomised trials" to the ‘Characteristics of included studies’ section.

In response to your query about cluster, parallel and pragmatic RCT’s: Thank you for raising this important point, however, unfortunately we did not extract these specific types of data as our focus was on definition and description of recruitment irrespective of trial design. We do see, in hindsight, that this provides a limitation, and have noted this in the added text on limitations in in the Conclusion.

We have revised the wording around proportions and ‘clearly identify’ in line with your suggestions.

Conclusion:
We have now acknowledged the limitation that the type of randomised trial might also potentially influence the definition of recruitment in trial reports.

In response to your comment about ‘enrolment’: We appreciate that the terms ‘enrolment’ and ‘recruitment’ are often used interchangeably and indeed ‘enrolment’ was a commonly used term in the analysed reports. As our aim was to develop a definition for ‘recruitment,’ which is not to say that ‘recruitment’ is a more useful term than ‘enrolment’ but the hope is that by providing clarity on at least one term, this will reduce ambiguity in trial reporting, and potentially encourage those reporting on trials to consistently use ‘recruitment’ based on this definition.

In response to your comment about emphasising the importance of clear trial reporting and preliminary recommendations: We have added text to the end of the Conclusion which we hope addresses this point.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 29 Jan 2021

Hans Lund, Section for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14299.r28598

The authors state that "ambiguity in anchoring the trial start date, end date, recruitment and enrolment temporally to trial processes (e.g. invitation, consent, and randomisation) has the potential for variation in how recruitment is reported and understood in

The authors state that "ambiguity in anchoring the trial start date, end date, recruitment and enrolment temporally to trial processes (e.g. invitation, consent, and randomisation) has the potential for variation in how recruitment is reported and understood in trial registries, trial protocols and trial reports." and thus in phase 1 aims to perform a concept analysis of ‘trial recruitment’ and develop a preliminary operational definition of ‘trial recruitment’. However, there is no systematic evaluation of earlier similar studies, and thus a limited justification of this study. The aim of the study makes it even more importantly to identify and consider earlier similar studies as the authors are looking for how a certain concept has been used and defined.
Whenever a concept is analyzed and discussed it would be helpful to identify, describe and use the existing theories about the concept. Having this theoretical background would be both helpful in analyzing the results of the study and helps the readers to evaluate the relevance of the interpretation of the data.
It may have been relevant to perform an analysis where the results randomized versus non-randomized included studies were performed.
All in all a very important and well-performed study.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Rehabilitation - Systematic Reviews - Evidence-Based Medicine - Evidence-Based Research

CITE

Report a concern

Author Response 01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

01 Apr 2022

Author Response

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text ... Continue reading Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

1. We appreciate this important comment, thank you, however, we have not identified any previous similar studies including any previous concept analyses that have specifically analysed the concept of ‘trial recruitment’ and how it might be operationally defined. We have now emphasised this in the final paragraph of the Background section. This text is also expanded and further supported with revisions in line with comment 2 (see below).

2. Similar to comment 1, we did not identify existing theories that explicitly analysed the concept of ‘trial recruitment’, however, we have now added text on previous studies that used theoretical frameworks in trying to understand or enhance trial recruitment strategies. This additional text and references (final paragraph of the Background section), we hope gives further justification for undertaking a formal concept analysis.

3. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed are randomised trials (n=148/150 are RCTs). We also refer to this issue in the limitations sections of the Conclusion.
Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

1. We appreciate this important comment, thank you, however, we have not identified any previous similar studies including any previous concept analyses that have specifically analysed the concept of ‘trial recruitment’ and how it might be operationally defined. We have now emphasised this in the final paragraph of the Background section. This text is also expanded and further supported with revisions in line with comment 2 (see below).

2. Similar to comment 1, we did not identify existing theories that explicitly analysed the concept of ‘trial recruitment’, however, we have now added text on previous studies that used theoretical frameworks in trying to understand or enhance trial recruitment strategies. This additional text and references (final paragraph of the Background section), we hope gives further justification for undertaking a formal concept analysis.

3. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed are randomised trials (n=148/150 are RCTs). We also refer to this issue in the limitations sections of the Conclusion.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

01 Apr 2022

Author Response

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text ... Continue reading Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

1. We appreciate this important comment, thank you, however, we have not identified any previous similar studies including any previous concept analyses that have specifically analysed the concept of ‘trial recruitment’ and how it might be operationally defined. We have now emphasised this in the final paragraph of the Background section. This text is also expanded and further supported with revisions in line with comment 2 (see below).

2. Similar to comment 1, we did not identify existing theories that explicitly analysed the concept of ‘trial recruitment’, however, we have now added text on previous studies that used theoretical frameworks in trying to understand or enhance trial recruitment strategies. This additional text and references (final paragraph of the Background section), we hope gives further justification for undertaking a formal concept analysis.

3. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed are randomised trials (n=148/150 are RCTs). We also refer to this issue in the limitations sections of the Conclusion.
Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

1. We appreciate this important comment, thank you, however, we have not identified any previous similar studies including any previous concept analyses that have specifically analysed the concept of ‘trial recruitment’ and how it might be operationally defined. We have now emphasised this in the final paragraph of the Background section. This text is also expanded and further supported with revisions in line with comment 2 (see below).

2. Similar to comment 1, we did not identify existing theories that explicitly analysed the concept of ‘trial recruitment’, however, we have now added text on previous studies that used theoretical frameworks in trying to understand or enhance trial recruitment strategies. This additional text and references (final paragraph of the Background section), we hope gives further justification for undertaking a formal concept analysis.

3. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed are randomised trials (n=148/150 are RCTs). We also refer to this issue in the limitations sections of the Conclusion.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 01 Apr 22	read	read
Version 1 21 Dec 20	read	read

Hans Lund, Western Norway University of Applied Sciences, Bergen, Norway
Jennifer Hanratty, Queen's University Belfast, Belfast, UK; Queen's University Belfast, Belfast, UK

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

20 Views

13 Apr 2022 | for Version 2

Jennifer Hanratty, Centre for Evidence and Social Innovation, Queen's University Belfast, Belfast, UK; Campbell UK & Ireland, Queen's University Belfast, Belfast, UK

20 Views Cite this report Responses(0)

Approved

Thank you for your additions and clarifications.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Systematic reviews and evidence synthesis, evaluation (including trials).

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

20 Views

01 Apr 2022 | for Version 2

Hans Lund, Section for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway

20 Views Cite this report Responses(0)

Approved

No further comments.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Rehabilitation - Systematic Reviews - Evidence-Based Medicine - Evidence-Based Research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

56 Views

09 Feb 2021 | for Version 1

Jennifer Hanratty, Centre for Evidence and Social Innovation, Queen's University Belfast, Belfast, UK; Campbell UK & Ireland, Queen's University Belfast, Belfast, UK

56 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Systematic reviews and evidence synthesis, evaluation (including trials).

Respond to this report

Responses (1)

Author Response

01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, in sections below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

Abstract: We now use the term ‘recruitment’ consistently in the abstract but note also that the term ‘recruitment’ is often referred to interchangeably in reports as ‘enrolment’. We have added a further sentence to the Background (Section 1.0) to explain this further.

Background: We have added a sentence to the Background to emphasize why variation in how recruitment is reported can be problematic.

Methods:
In response to your comment on the exclusion of protocols: Thank you for this important point. We did consider trial protocols, however, we wished to analyse ‘actual’ recruitment data rather than what might be planned. We have clarified this with an added sentence to the paper (end of paragraph in ‘Dealing with meaning and measurement’ section).

We have corrected Figure 3.

We have moved the line "148 of the records reported on randomised trials and two reported on non-randomised trials" to the ‘Characteristics of included studies’ section.

In response to your query about cluster, parallel and pragmatic RCT’s: Thank you for raising this important point, however, unfortunately we did not extract these specific types of data as our focus was on definition and description of recruitment irrespective of trial design. We do see, in hindsight, that this provides a limitation, and have noted this in the added text on limitations in in the Conclusion.

We have revised the wording around proportions and ‘clearly identify’ in line with your suggestions.

Conclusion:
We have now acknowledged the limitation that the type of randomised trial might also potentially influence the definition of recruitment in trial reports.

In response to your comment about ‘enrolment’: We appreciate that the terms ‘enrolment’ and ‘recruitment’ are often used interchangeably and indeed ‘enrolment’ was a commonly used term in the analysed reports. As our aim was to develop a definition for ‘recruitment,’ which is not to say that ‘recruitment’ is a more useful term than ‘enrolment’ but the hope is that by providing clarity on at least one term, this will reduce ambiguity in trial reporting, and potentially encourage those reporting on trials to consistently use ‘recruitment’ based on this definition.

In response to your comment about emphasising the importance of clear trial reporting and preliminary recommendations: We have added text to the end of the Conclusion which we hope addresses this point.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

55 Views

29 Jan 2021 | for Version 1

Hans Lund, Section for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway

55 Views Cite this report Responses(1)

Approved With Reservations

The authors state that "ambiguity in anchoring the trial start date, end date, recruitment and enrolment temporally to trial processes (e.g. invitation, consent, and randomisation) has the potential for variation in how recruitment is reported and understood in trial registries, trial protocols and trial reports." and thus in phase 1 aims to perform a concept analysis of ‘trial recruitment’ and develop a preliminary operational definition of ‘trial recruitment’. However, there is no systematic evaluation of earlier similar studies, and thus a limited justification of this study. The aim of the study makes it even more importantly to identify and consider earlier similar studies as the authors are looking for how a certain concept has been used and defined.
Whenever a concept is analyzed and discussed it would be helpful to identify, describe and use the existing theories about the concept. Having this theoretical background would be both helpful in analyzing the results of the study and helps the readers to evaluate the relevance of the interpretation of the data.
It may have been relevant to perform an analysis where the results randomized versus non-randomized included studies were performed.
All in all a very important and well-performed study.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Rehabilitation - Systematic Reviews - Evidence-Based Medicine - Evidence-Based Research

Respond to this report

Responses (1)

Author Response

01 Apr 2022

Valerie Smith, Trinity College Dublin and National University of Ireland Galway, Ireland

Many thanks for taking the time to consider our paper. We appreciate your valuable input and have replied to your comments, numbered below. Please see our amendments to the text in the revised paper. This revised version now also includes the findings and methods for Phase 2 of the Concept Analysis (and a final definition of ‘trial recruitment’).

1. We appreciate this important comment, thank you, however, we have not identified any previous similar studies including any previous concept analyses that have specifically analysed the concept of ‘trial recruitment’ and how it might be operationally defined. We have now emphasised this in the final paragraph of the Background section. This text is also expanded and further supported with revisions in line with comment 2 (see below).

2. Similar to comment 1, we did not identify existing theories that explicitly analysed the concept of ‘trial recruitment’, however, we have now added text on previous studies that used theoretical frameworks in trying to understand or enhance trial recruitment strategies. This additional text and references (final paragraph of the Background section), we hope gives further justification for undertaking a formal concept analysis.

3. We acknowledge that trial design could potentially impact on the variation and type of terminology used when reporting trials, for instance whether or not a trial is randomised and whether the trial includes a run-in period. We did not extract data relating to trial run-in periods and the majority of the trials analysed are randomised trials (n=148/150 are RCTs). We also refer to this issue in the limitations sections of the Conclusion.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. International Committee of Medical Journal Editors: Publishing and Editorial Issues Related to Publication in Medical Journals – Clinical Trials. Reference Source

[2] 2. World Health Organisation: International standards for clinical trial registries 2012. Reference Source

[3] 3. International Standard Randomised Controlled Trial Number (ISRCTN) registry: Definitions. Reference Source

[4] 4. Clinicaltrials.gov Registry: Protocol Registration Data Element Definitions for Interventional and Observational Studies. Reference Source

[5] 5. Schwartz-Barcott D, Kim HS: Chapter 9: An expansion and elaboration of the hybrid model of concept development. In: Rodgers BL, Knafl KA. (eds.) Concept Development in Nursing. Foundations, Techniques and Applications. Second Edition, Philadelphia: WB. Saunders; 2000; 129–159.

[6] 6. Knafl K, Deatrick J: Chapter 2: Knowledge Synthesis and Concept Development in Nursing. In: Rodgers BL, Knafl KA. (eds.) Concept Development in Nursing. Foundations, Techniques and Applications. Second Edition, Philadelphia: WB. Saunders; 2000; 39–54.

[7] 7. Cronin P, Ryan F, Coughlan M: Concept analysis in healthcare research. Int J Ther Rehabil. 2010; 17(2): 62–68. Publisher Full Text

[8] 8. Beecher C, Devane D, White M, et al.: Concept development in Nursing and Midwifery: An overview of methodological approaches. Int J Nurs Pract. 2019; 25(1): e12702. PubMed Abstract | Publisher Full Text

[9] 9. Jasemi M, Valizadeh L, Zamanzadeh V, et al.: A Concept Analysis of Holistic Care by Hybrid Model. Indian J Palliat Care. 2017; 23(1): 71–80. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Impact Factors: Medical Journal Impact Factors 2019. [Accessed 26^th June 2019]. Reference Source

[11] 11. Delaney H: A Concept Analysis of trial recruitment. Extended Data. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.13109870.v1

[12] 12. Lefebvre C, Manheimer E, Glanville J: Chapter 6: Searching for studies. In: Higgins J, Green S (eds). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. updated March 2011. The Cochrane Collaboration, 2011. Reference Source

[13] 13. Schulz KF, Altman DG, Moher D, et al.: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340: c332. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Moher D, Liberati A, Tetzlaff J, et al.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Tinnitus Retraining Therapy Trial Research Group, Scherer RW, Formby C: Effect of Tinnitus Retraining Therapy vs Standard of Care on Tinnitus-Related Quality of Life: A Randomized Clinical Trial. JAMA Otolaryngol Head Neck Surg. 2019; 145(7): 597–608. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Fowler D, Hodgekins J, French P, et al.: Social recovery therapy in combination with early intervention services for enhancement of social recovery in patients with first-episode psychosis (SUPEREDEN3): a single-blind, randomised controlled trial. Lancet Psychiatry. 2018; 5(1): 41–50. PubMed Abstract | Publisher Full Text | Free Full Text

[17] 17. Raghu G, van den Blink B, Hamblin MJ, et al.: Effect of Recombinant Human Pentraxin 2 vs Placebo on Change in Forced Vital Capacity in Patients With Idiopathic Pulmonary Fibrosis: A Randomized Clinical Trial. JAMA. 2018; 319(22): 2299–2307. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Wildiers H, Tryfonidis K, Dal Lago L, et al.: Pertuzumab and trastuzumab with or without metronomic chemotherapy for older patients with HER2-positive metastatic breast cancer (EORTC 75111-10114): an open-label, randomised, phase 2 trial from the Elderly Task Force/Breast Cancer Group. Lancet Oncol. 2018; 19(3): 323–336. PubMed Abstract | Publisher Full Text

[19] 19. Bassler D, Shinwell ES, Hallman M, et al.: Long-Term Effects of Inhaled Budesonide for Bronchopulmonary Dysplasia. N Engl J Med. 2018; 378(2): 148–157. PubMed Abstract | Publisher Full Text

[20] 20. Kendler D, Marin F, Zerbini CAF, et al.: Effects of teriparatide and risedronate on new fractures in post-menopausal women with severe osteoporosis (VERO): a multicentre, double-blind, double-dummy, randomised controlled trial. Lancet. 2018; 391(10117): 230–240. PubMed Abstract | Publisher Full Text

[21] 21. Horwitz S, O’Connor OA, Pro B, et al.: Brentuximab vedotin with chemotherapy for CD30-positive peripheral T-cell lymphoma (ECHELON-2): a global, double-blind, randomised, phase 3 trial. Lancet. 2019; 393(10168): 229–240. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Birenbaum A, Hajage D, Roche S, et al.: Effect of Cricoid Pressure Compared With a Sham Procedure in the Rapid Sequence Induction of Anesthesia: The IRIS Randomized Clinical Trial. JAMA Surg. 2019; 154(1): 9–17. PubMed Abstract | Publisher Full Text | Free Full Text

[23] 23. Shinde S, Weiss HA, Varghese B, et al.: Promoting school climate and health outcomes with the SEHER multi-component secondary school intervention in Bihar, India: a cluster-randomised controlled trial. Lancet. 2018; 392(10163): 2465–2477. PubMed Abstract | Publisher Full Text

[24] 24. Stahl A, Krohne TU, Eter N, et al.: Comparing Alternative Ranibizumab Dosages for Safety and Efficacy in Retinopathy of Prematurity: A Randomized Clinical Trial. JAMA Pediatr. 2018; 172(3): 278–286. PubMed Abstract | Publisher Full Text | Free Full Text

[25] 25. van Vollenhoven RF, Hahn BH, Tsokos GC, et al.: Efficacy and safety of ustekinumab, an IL-12 and IL-23 inhibitor, in patients with active systemic lupus erythematosus: results of a multicentre, double-blind, phase 2, randomised, controlled study. Lancet. 2018; 392(10155): 1330–1339. PubMed Abstract | Publisher Full Text

[26] 26. Bridey C, Thilly N, Lefevre T, et al.: Ultrasound-guided versus landmark approach for peripheral intravenous access by critical care nurses: a randomised controlled study. BMJ Open. 2018; 8(6): e020220. PubMed Abstract | Publisher Full Text | Free Full Text

A concept analysis of ‘trial recruitment’ using the hybrid model – Phase 1 findings

Abstract

Keywords

Background

Aim

Methods

Study design

Figure 1. Phases of the hybrid model of concept analysis⁵.

Searching the literature

Table 1. Top five impact factor medical journals 2019¹⁰.

Dealing with meaning and measurement

Table 2. Sample format for organising and analysing definitions⁵.

Data analysis

Figure 2. CONSORT flow diagram¹³ - edited to include the four time points analysed for this study.

Findings of Phase 1

Results of the search

Figure 3. PRISMA flow diagram¹⁴

Characteristics of included studies

Temporal descriptions of ‘recruitment’

Figure 4. At what time point is recruitment defined?

Table 3. Reported start and end date.

Variation in terminology

Table 4. Terminology.

Table 5. Variation in terminology.

Conclusion and working definition for Phase 2

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

A concept analysis of ‘trial recruitment’ using the hybrid model – Phase 1 findings

Abstract

Keywords

Background

Aim

Methods

Study design

Figure 1. Phases of the hybrid model of concept analysis5.

Searching the literature

Table 1. Top five impact factor medical journals 201910.

Dealing with meaning and measurement

Table 2. Sample format for organising and analysing definitions5.

Data analysis

Figure 2. CONSORT flow diagram13 - edited to include the four time points analysed for this study.

Findings of Phase 1

Results of the search

Figure 3. PRISMA flow diagram14

Characteristics of included studies

Temporal descriptions of ‘recruitment’

Figure 4. At what time point is recruitment defined?

Table 3. Reported start and end date.

Variation in terminology

Table 4. Terminology.

Table 5. Variation in terminology.

Conclusion and working definition for Phase 2

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!

Figure 1. Phases of the hybrid model of concept analysis⁵.

Table 1. Top five impact factor medical journals 2019¹⁰.

Table 2. Sample format for organising and analysing definitions⁵.

Figure 2. CONSORT flow diagram¹³ - edited to include the four time points analysed for this study.

Figure 3. PRISMA flow diagram¹⁴