Using behavioural science to enhance use of core outcome sets in trials: protocol

Karen Matvienko-Sikar; Molly Byrne; Mike Clarke; Jamie Kirkham; Jan Kottner; Katie Mellor; Fiona Quirke; Ian J. Saldanha; Valerie Smith; Elaine Toomey; Paula Williamson

doi:10.12688/hrbopenres.13510.1

Home Browse Using behavioural science to enhance use of core outcome sets in trials:...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Using behavioural science to enhance use of core outcome sets in trials: protocol

[version 1; peer review: 1 approved, 1 approved with reservations]

Karen Matvienko-Sikar ¹, Molly Byrne², Mike Clarke³, [...] Jamie Kirkham⁴, Jan Kottner⁵, Katie Mellor⁶, Fiona Quirke⁷, Ian J. Saldanha⁸, Valerie Smith⁹, Elaine Toomey¹⁰, Paula Williamson¹¹

Karen Matvienko-Sikar ¹, Molly Byrne², [...] Mike Clarke³, Jamie Kirkham⁴, Jan Kottner⁵, Katie Mellor⁶, Fiona Quirke⁷, Ian J. Saldanha⁸, Valerie Smith⁹, Elaine Toomey¹⁰, Paula Williamson¹¹

PUBLISHED 24 Mar 2022

Author details Author details

¹ School of Public Health, University College Cork, Cork, Ireland
² School of Psychology, National University of Ireland, Galway, Galway, Ireland
³ Centre for Public Health, School of Medicine, Dentistry and Biomedical Sciences, Queen’s University Belfast, Belfast, UK
⁴ Centre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
⁵ Charité-Universitätsmedizin Berlin, Institute of Clinical Nursing Science, Berlin, Germany
⁶ Centre for Statistics in Medicine, University of Oxford, Oxford, UK
⁷ College of Medicine, Nursing & Health Sciences, Áras Moyola, National University of Ireland, Galway, Galway, Ireland
⁸ Center for Evidence Synthesis in Health; Department of Health Services, Policy, and Practice, Department of Epidemiology, Brown University School of Public Health, Providence, USA
⁹ School of Nursing and Midwifery, Trinity College Dublin, Dublin, Ireland
¹⁰ School of Allied Health, University of Limerick, Limerick, Ireland
¹¹ Trials Methodology Research Partnership, Department of Health Data Science, University of Liverpool, Liverpool, UK

Karen Matvienko-Sikar
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Molly Byrne
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Mike Clarke
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Jamie Kirkham
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Jan Kottner
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Katie Mellor
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Fiona Quirke
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Ian J. Saldanha
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Valerie Smith
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Elaine Toomey
Roles: Funding Acquisition, Methodology, Writing – Review & Editing

Paula Williamson
Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Core outcome sets (COS) represent agreed-upon sets of outcomes, which are the minimum that should be measured and reported in all trials in specific health areas. Use of COS can reduce outcome heterogeneity, selective outcome reporting, and research waste, and can facilitate evidence syntheses. Despite benefits of using COS, current use of COS in trials is low. COS use can be understood as a behaviour, in that it is something trialists do, or not do, adequately. The aim of this study is to identify stakeholder-prioritised strategies, informed by behaviour change theory, to increase COS use in trials.

Methods: The project will be conducted in three stages, informed by the behaviour change wheel (BCW). The BCW is a theoretically based framework that can be used to classify, identify, and develop behaviour change strategies. In Stage 1, barriers and enablers to COS use will be extracted from published studies that examined trialist’s use of COS. Barriers and facilitators will be mapped to the components of COM-B model (capability, opportunity, and motivation), which forms part of the BCW framework. Stage 2 will build on Stage 1 findings to identify and select intervention functions and behaviour change techniques to enhance COS use in trials. Stage 3 will involve an online stakeholder consensus meeting including trialists, healthcare professionals, and patient/public representatives. The purpose of the meeting is to prioritise identified intervention approaches that will inform future research to increase COS use.

Discussion: The findings of this study will provide an understanding of the behavioural factors that influence COS use in trials, what strategies might be used to target these factors to increase COS use, and what strategies key stakeholders perceive as especially important in future research to enhance COS use in trials.

Keywords

Core outcome sets, trials, trial methodology

Corresponding author: Karen Matvienko-Sikar

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2022 Matvienko-Sikar K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Matvienko-Sikar K, Byrne M, Clarke M et al. Using behavioural science to enhance use of core outcome sets in trials: protocol [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2022, 5:23 (https://doi.org/10.12688/hrbopenres.13510.1) First published: 24 Mar 2022, 5:23 (https://doi.org/10.12688/hrbopenres.13510.1) Latest published: 20 Nov 2023, 5:23 (https://doi.org/10.12688/hrbopenres.13510.2)

Core outcome sets (COS) are agreed-upon sets of outcomes that should be measured and reported in all trials in specific health areas¹. COS are usually developed using input from broad stakeholders, such as researchers (including trialists), healthcare professionals, patient/public representatives, and research funders². COS are used and/or recommended for use by these stakeholders^1,2. COS use involves trialists including the COS in the trial design, measuring the COS outcomes during the trial, and reporting the COS outcomes in the final trial report. Use of COS facilitates evidence syntheses^1,3,4 and can reduce outcome heterogeneity⁵, selective outcome reporting⁶, and research waste⁷. Despite the benefits of using COS in trials, low COS use has been demonstrated across multiple areas of health research^8–11. Low use of COS is problematic because it means that methodological improvements associated with COS are not finding their way into trial conduct quickly enough.

Previous research has examined potential reasons for low COS use in trials, including identification of barriers and facilitators to COS use among researchers with trials registered on the International Standard Randomised Controlled Trial Number (ISRCTN) Registry¹¹, researchers who submitted funding applications to the National Institute for Health Research Health Technology Assessment (NIHR HTA) programme¹², researchers named as chief investigators of NIHR HTA funded trials¹³, and researchers who published a trial report in a major medical journal (e.g., The Lancet, BMJ)⁸. Identified barriers to COS use include poor knowledge about existence of COS, and perceived outcome measurement issues, including patient burden^8,13. Facilitators for COS use include good trialist knowledge about what COS are and how to use them, perceived importance of COS in trials, and having funder or professional recommendations or requirements to use COS in funded research^8,13.

Whether or not trialists use COS in their trials can be understood as a behaviour because COS use is something trialists do, or not do. As such, COS use in trials might be modified and increased through theoretically informed behaviour change strategies. The Behaviour Change Wheel (BCW) provides a useful theoretically based framework¹⁴ to identify and develop behaviour change strategies for COS use. The BCW framework includes the COM-B model, in which behaviour is predicated on an individual’s capability, opportunity, and motivation to engage in that behaviour^14,15. Following the COM-B model, for a behaviour to occur, an individual must have the physical and psychological capability, social and physical opportunity, and reflective and automatic motivation to engage in the behaviour^14,15. In addition to the COM-B model, the BCW also includes systematic guidance on identifying targeted intervention components and content in the form of intervention functions and behaviour change techniques (BCTs)^14–16. Adopting a behavioural science approach to understanding COS use behaviours and developing strategies to increase COS use using the BCW has the potential to maximise benefits of COS and improve the quality of trials and evidence-based practice. Incorporating views from relevant stakeholders, including trialists, healthcare professionals, and patients and the public, in such an approach further ensures that developed strategies are very likely to be relevant, feasible, and acceptable to those who should use COS.

The overall project aim is to identify stakeholder-prioritised strategies, informed by the BCW, to increase trialist use of COS in trials. The findings of this project will provide an understanding of the behavioural factors that influence COS use in trials, what strategies might be used to target these factors to increase COS use, and what strategies key stakeholders perceive as particularly important to enhance future COS use in trials.

Methods

We will achieve the overall project aim by examining behavioural factors identified in the previous research^8,11–13 that influence whether or not trialists use COS. The focus is on the individual trialist behaviour, rather than broader system behaviours and factors, which are being examined in a separate project¹⁷. We will then map these factors to behaviour change strategies that will subsequently be prioritised by stakeholders (i.e., researchers, clinicians, journal editors, funding bodies, and patient/public representatives). This project will be conducted in three sequential stages to identify and prioritise behaviour change strategies that could enhance COS use in trials.

Stage 1. Identification of behavioural barriers and facilitators to COS use

Stage 1 is informed by the first phase of the BCW, which involves understanding the behaviour to be examined (i.e., use of COS in trials). This includes defining the components of the behaviour in terms of who, what, where, when, and how often the behaviour is done. Existing data from recently published examinations of COS use^8,11–13 and research team expertise will be used to understand and define use of COS in trials for these components.

Following on from behavioural specification of COS use in trials, barriers to and enablers of this behaviour will be extracted from four recent studies (all published since 2019) that examined trialist use of COS:

1. A review and survey of COS use in a cohort of trials published in major medical journals⁸.
2. A review and survey of COS use in funding applications submitted to the NIHR HTA programme¹².
3. A survey of trialists named as the contact person for trials registered on the ISRCTN Registry¹¹.
4. A qualitative study of trialist barriers and facilitators to COS use¹³.

Barriers and facilitators will be extracted verbatim from the four papers and will be coded to the components of COM-B framework^14,15 to identify behavioural components influencing trialist use of COS. These components include capability (physical and psychological), opportunity (physical and social) and motivation (automatic and reflective). The previously conducted qualitative study of trialist barriers and facilitators to COS use¹³, utilised the COM-B framework to guide analysis, and so will inform the approach taken in the current study. One investigator will conduct initial coding using a standardised coding form (Extended Data). All coding will be verified by a second investigator, with any discrepancies discussed to reach consensus, involving a third investigator as needed; all investigators involved in coding will have prior experience in coding using the COM-B framework. The findings from this coding will be synthesised narratively and using matrices, guided by COM-B as a deductive framework to identify behavioural components influencing COS use in trials.

Stage 2. Identification of behavioural intervention strategies to enhance COS use in trials

The BCW framework^14,15 will be applied to identify and select intervention functions to enhance COS use by trialists in trials. Intervention functions are ‘broad categories of means by which an intervention can change behaviour’ (e.g., incentivisation, training)¹⁴. Intervention functions map on to COM-B components¹⁴ and can be used in isolation or together to develop behavioural strategies. Stage 2 will use the findings of Stage 1 to identify potential intervention functions by mapping identified behavioural components to corresponding intervention functions. For example, the behavioural component ‘psychological capability’ maps on to the intervention functions ‘training’, ‘education’, and ‘enablement’¹⁴. Where multiple intervention functions are identified, we will ensure that the selected intervention functions are affordable, practical, effective/cost-effective, acceptable, safe, and equitable (the APEASE criteria)¹⁴. Two investigators will independently apply the APEASE criteria to each identified intervention function, with APEASE criteria rated as ‘yes’, ‘no’, or ‘unsure’; any disagreements will be resolved by discussion, involving a third investigator as needed. Rationale for each decision made using the APEASE criteria will be also documented on the standardised APEASE template (Extended Data).

We will also identify potential intervention content in terms of BCTs. BCTs are irreducible, observable, and replicable active ingredients of an intervention designed to change behaviour that can be mapped from identified intervention functions¹⁴. BCTs will be identified using the BCT Taxonomy Version 1 (BCTTv1)¹⁶. Each BCT will be operationalised by translating it from the BCTTv1 definition to a concrete application; for example, the BCT ‘demonstration of the behaviour’ may be operationalised as delivering a workshop for trialists demonstrating how COS can be included in trials. As with intervention functions, two investigators will independently screen BCTs using the APEASE criteria to evaluate identified BCTs based on the APEASE criteria¹⁴. Rationale for each decision made using the APEASE criteria will be documented on a standardised template (Extended data) and any disagreements will be resolved by discussion, involving a third investigator as needed.

Stage 3. Prioritisation of intervention strategies to enhance COS use in trials

We will conduct an online stakeholder consensus meeting¹⁸ to prioritise identified intervention approaches that will inform future research to increase COS use in trials. The prioritisation meeting will be conducted using the Nominal Group Technique (NGT) approach¹⁹ and following recent guidance from the Core Outcome Measures for Effectiveness Trials (COMET) Initiative¹⁸.

Participants: Members of relevant trials methodology and outcome specific groups will be invited to participate in the consensus meeting via an email invitation circulated to all group members, including those sitting on group executive and steering committees as well as group mailing lists. Relevant gatekeepers for groups will be contacted to facilitate dissemination of the email invitation where needed. Groups to be included are the Health Research Board Trial Methodology Research Network (HRB TMRN) and the Medical Research Council National Institute for Health Research Trial Methodology Research Partnership (MRC-NIHR-TMRP), both of which represent national and transnational networks and partnerships of researchers working in trial methodology. Members of the Red Hat Group, which includes several initiatives related to improving choice of outcomes in health research (e.g., Outcome Measures in Rheumatoid Arthritis Clinical Trials [OMERACT] and Consensus-based Standards for the Selection of Health Measurement Instruments [COSMIN]) will also be recruited²⁰. Healthcare professionals and patient/public representatives will be identified from existing networks, trials and/or outcomes research by the study team and invited to participate via direct email contact. The meeting will include 10 to 15 participants, with recruitment focused on diversity of expertise and representativeness across stakeholder groups. This sample size represents a feasible number to facilitate online participant discussions and input to achieve consensus on priority approaches for future research¹⁸.

Consensus meeting: Before the meeting, participants will receive a lay language summary of the intervention functions and BCTs identified in Stage 2. Participants will receive an information leaflet outlining the purpose and process of the stakeholder meeting, including the COMET plain language summary of the NGT (available at the COMET Initiative website www.comet-initiative.org). Intervention functions and the associated BCTs will be presented to participants by the meeting facilitator. Participants will then engage in break-out group discussions about the intervention functions and BCTs, including consideration of the ease of implementation and anticipated effectiveness of the presented approaches. The number of intervention functions and BCTs discussed by each break-out group will be dependent on the preceding stages; if a large number of intervention functions and/or BCTs are identified, these will be randomly allocated for discussion across the break-out groups. Participants will also be encouraged to suggest any additional ideas about behavioural strategies at this stage. Following the break-out discussions, each break-out group will present back to the full group. Further discussion will then take place to clarify and/or further discuss any points raised in the break-out groups. Participants will then rank intervention functions and BCTs by perceived importance for future research, considering ease of implementation, feasibility, and potential effectiveness. These rankings will be presented to and discussed with the participants, and a final round of ranking will follow with the aim of reaching consensus on a prioritised list of strategies to target in future research.

The meeting will conclude with a planning activity for future research. This activity will also use structured break-out group discussions followed by full-group discussions. The discussion will address potential future studies (e.g., topics, methodological approaches), funding opportunities (e.g., current or future national and international funding announcements), and collaborators.

Ethical considerations

All research activities will be conducted following the University College Cork (UCC) Code of Research Conduct ethical approval and in accordance with General Data Protection Regulations (GDPR). Stages one and two do not involve any potential ethical issues because they relate to reviewing and synthesising evidence from the existing literature. For the Stage three consensus meeting, ethical approval will be sought and obtained from the Social Research Ethics Committee in University College Cork. Participants will be provided with full study information before the study, in which it will be made clear that participation is voluntary, that the meeting will be recorded for data collection purposes (the recording will not be shared publicly), and that all ranking data will be recorded and stored anonymously. The recording will be deleted after completion of the project.

Dissemination

This study is registered on the Open Science Framework (DOI:10.17605/OSF.IO/GWYZS); accompanying data and materials will also be made openly available upon study completion on the Open Science Framework. The study findings will be submitted for publication in a peer-reviewed journal and disseminated through presentations to the working groups and the MRC-NIHR TMRP and the HRB-TMRN, at conferences, and through social media.

Discussion

Use of COS in trials can benefit evidence syntheses^1,3,4, in addition to reducing outcome heterogeneity⁵, selective outcome reporting⁶, and research waste⁷. This project will provide information on stakeholder-prioritised strategies, informed by behavioural science, to enhance COS use in trials. The results will therefore provide the foundation for future methodology research and development and implementation of strategies to maximise COS use in trials.

Study status

Stage one of the study commenced in March 2022.

Data availability

Extended data

Open Science Framework: Enhancing COS Use in Trials, https://doi.org/10.17605/OSF.IO/BN3YQ (Matvienko-Sikar et al., 2022)

This project contains the following extended data:

Supplementary File 1: Coding barriers and facilitators to COM-B components
Supplementary File 2. Selection of Intervention Functions
Supplementary File 3. Selection of Behaviour Change Techniques (BCTs)

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Faculty Opinions recommended

References

1. Williamson PR, Altman DG, Bagley H, et al.: The COMET Handbook: version 1.0. Trials. 2017; 18(Suppl 3): 280. PubMed Abstract | Publisher Full Text | Free Full Text
2. Gargon E, Gorst SL, Matvienko-Sikar K, et al.: Choosing important health outcomes for comparative effectiveness research: 6th annual update to a systematic review of core outcome sets for research. Wiysonge CS editor. PLoS One. 2021; 16(1): e0244878. PubMed Abstract | Publisher Full Text | Free Full Text
3. Williamson PR, Altman DG, Blazeby JM, et al.: Developing core outcome sets for clinical trials: issues to consider. Trials. 2012; 13: 132. PubMed Abstract | Publisher Full Text | Free Full Text
4. Webbe J, Sinha I, Gale C: Core Outcome Sets. Arch Dis Child Educ Pract Ed. 2018; 103(3): 163–166. PubMed Abstract | Publisher Full Text
5. Schmitt J, Lange T, Kottner J, et al.: Cochrane Reviews and Dermatological Trials Outcome Concordance: Why Core Outcome Sets Could Make Trial Results More Usable. J Invest Dermatol. 2019; 139(5): 1045–1053. PubMed Abstract | Publisher Full Text
6. Dwan K, Altman DG, Arnaiz JA, et al.: Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. Siegfried N editor. PLoS One. 2008; 3(8): e3081. PubMed Abstract | Publisher Full Text | Free Full Text
7. Glasziou P, Altman DG, Bossuyt P, et al.: Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014; 383(9913): 267–276. PubMed Abstract | Publisher Full Text
8. Matvienko-Sikar K, Avery K, Blazeby J, et al.: Use of Core Outcome Sets was Low in Clinical Trials Published in Major Medical Journals. J Clin Epidemiol. 2022; 142: 19–28. PubMed Abstract | Publisher Full Text
9. Hughes KL, Clarke M, Williamson PR: A systematic review finds core outcome set uptake varies widely across different areas of health. J Clin Epidemiol. 2021; 129: 114–123. PubMed Abstract | Publisher Full Text | Free Full Text
10. Saldanha IJ, Dodd S, Gorst SL, et al.: More than half of systematic reviews have relevant core outcome sets. J Clin Epidemiol. 2021; 136; 168–179. PubMed Abstract | Publisher Full Text | Free Full Text
11. Bellucci C, Hughes K, Toomey E, et al.: A Survey of Knowledge, Perceptions and Use of Core Outcome Sets among Clinical Trialists. Trials. 2021; 22(1): 937. PubMed Abstract | Publisher Full Text | Free Full Text
12. Hughes KL, Kirkham JJ, Clarke M, et al.: Assessing the impact of a research funder’s recommendation to consider core outcome sets. Derrick GE editor. PLoS One. 2019; 14(9): e0222418. PubMed Abstract | Publisher Full Text | Free Full Text
13. Hughes KL, Williamson PR, Young B: In-depth qualitative interviews identified barriers and facilitators that influenced chief investigators’ use of core outcome sets in randomised controlled trials. J Clin Epidemiol. 2021; 144: 111–120. PubMed Abstract | Publisher Full Text
14. Michie S, Atkins L, West R: The Behaviour Change Wheel: A Guide to Designing Interventions. Silverback Publishing; 2014. Reference Source
15. Michie S, van Stralen MM, West R: The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Sci. 2011; 6(1): 42. PubMed Abstract | Publisher Full Text | Free Full Text
16. Michie S, Richardson M, Johnston M, et al.: The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Implement Sci. 2013; 46(1): 81–95. PubMed Abstract | Publisher Full Text | Free Full Text
17. Williamson PR, Clarke M, Hughes K, et al.: Improving core outcome set uptake in trials: Project advertisment. Reference Source
18. Gorst SL, Barrington H, Brookes ST, et al.: Online consensus meetings for COS development: issues to consider. Reference Source
19. McMillan SS, King M, Tully MP: How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3): 65562. PubMed Abstract | Publisher Full Text | Free Full Text
20. Gargon L, Facile R, Nieuwlaat R, et al.: The Red Hat Group: an initiative to promote broader uptake of core outcome sets through the healthcare research ecosystem. Reference Source
21. Matvienko-Sikar K: Enhancing COS use in trials. 2022. http://www.doi.org/10.17605/OSF.IO/BN3YQ

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 24 Mar 2022