Using behavioural science to enhance use of core outcome sets in trials: protocol

Reviewer comment 1. Thank you for the opportunity to participate in this review. The protocol paper describes a programme of research with three stages aimed at identifying theory-indicated behaviour change strategies to promote the use of core outcome sets (COS) when reporting medical research. The research described is novel and important. Authors have provided a comprehensive overview of the different research activities planned and their methodology. However, I lack specific details on the research protocol and I have also some suggestions to expand the discussion section. I hope my comments below will help authors to improve this submission.
Author response 1. Thank you for your feedback on the protocol manuscript and for your useful suggestions for improvement, which have now been incorporated.

Reviewer comment 2. Introduction
While the rationale is sound and well-presented, more information could be added to help readers understand what authors mean by ‘core outcome sets’. Introducing one or two examples would be helpful in this regard. Moreover, it would be good to clarify whether COS only apply to interventional research or also include all medical research (e.g., observational).
 Author Response 2. We have now added the following to the introduction to provide more information and clarification on COS:
“COS are not necessarily the only outcomes that should be included in trials, but they represent the minimum set of outcomes to be measured and reported. For instance, a recently published COS for post-COVID-19 condition, included 11 outcomes such as fatigue, pain, survival, and post-exertion symptoms. COS are also applicable for use in other study designs, such as observational studies and clinical audit¹.”

Reviewer comment 3. Authors state that “The Behaviour Change Wheel (BCW) provides a useful theoretically based framework to identify and develop behaviour change strategies for COS use”, but it is not clearly specified why is that the case. The selection of a given theoretical framework is critical as it will determine subsequent work according to the protocol, so it should be better justified. For example, a unique advantage of the BCW is that it helps intervention designers to not just better understand behaviour but also how to act upon it (i.e., provides a systematic approach on how to select and implement a wide range of intervention functions based on what is known about the behaviour).
Author Response 3. We agree that clearly explaining the choice of theoretical framework is important and have added the following to the introduction to better clarify this choice:
“Using the BCW is particularly useful as it facilitates mapping of barriers and facilitators to behavioural intervention functions and BCTs. As such, it can provide a systematic approach to selecting and implementing a wide range of intervention functions based on what is known about the behaviour.”

Reviewer comment 4. Methods –Stage 1
I lack further details on the four studies that will be used as part of this stage. Why have these been selected? Do they represent a sizeable portion of all literature on COS influences? Have these studies been conducted by the research team, or other researchers? What is the rationale to select studies published since 2019? These details are relevant as this constitutes the source data from which the entire project will be based on.
Author 4. The four studies included represent the research that had been conducted to the date of the protocol on use and uptake of broad cohorts of COS, rather than use of COS in specific health areas. This information across a broad cohort of health areas is needed to develop intervention strategies that are potentially useful across health areas. Some, but not all, members of the research team were involved in these papers, though the first authors of three papers (Dr Hughes, Ms Bellucci) are not involved in this project. This was not a deciding factor in inclusion of these papers, rather a reflection of the work of members of the research team in this area. Similarly, studies were not required to be published since 2019 but the studies identified were, thus providing up to date information. We have included the following for further clarification:
“These papers were identified in a recent review of COS uptake as addressing why trialists use, or do not use, COS. They represent up to date research that had been conducted on use and uptake of broad cohorts of COS, rather than use of COS in specific health areas. This information across a broad cohort of health areas is needed to develop intervention strategies that are potentially useful across health areas.”

Reviewer comment 5. I understand the COM-B will be used deductively to code the source data. Are authors using any specific qualitative analysis deductive approach (e.g., directed content analysis)? What are the authors’ epistemology stance when it comes to these qualitative analyses? Do authors plan to use the TDF to expand on the COM-B components and provide a more nuanced framework for coding (and later mapping of intervention functions / BCTs)?
Response 5. We are not conducting specific qualitative deductive analyses beyond coding the included qualitative data to the COM-B framework. The epistemological stance taken in analysis of the qualitative data is a realist approach whereby we are seeking to identify and understand underlying mechanisms of sources of behaviour, in the form of barriers and facilitators. The COM-B was chosen as the single framework for use in this study to enable development of broad strategies, tied to specific behaviour change techniques that could be used. Future studies may build upon this work, utilising other frameworks such as the TDF, but this is not part of the current study.
The following has been added to Stage 1 in the Methods to clarify the epistemological stance:
“A realist approach to coding will be taken, whereby we are seeking to identify and understand underlying mechanisms of sources of behaviour, in the form of barriers and facilitators.”

Reviewer comment 6.
Methods –Stage 2
It is not clear to me why a prioritisation effort is taking place here to select between different intervention functions when you have a specific phase (stage 3) to precisely do that. Why not simply present all intervention functions to the experts as part of the consensus meeting? Please clarify.
Response 6. The consensus meeting (stage 3) has now been removed from the project. This has resulted from the decision to not conduct prioritisation of behavioural intervention components due to the heterogeneity in the behavioural intervention components identified. The research team felt that this heterogeneity would have limited meaningful ability to prioritise these techniques for trialists who are unfamiliar with them. Further, by providing a range of potential strategies, researchers can choose those which are best suited for their areas and COS.


Reviewer comment 7. Regarding the rating for the APASE criteria, I understand you will use different categories (‘yes’, ‘no’, or ‘unsure’), but it is not clear how this will lead to an overall inclusion/exclusion decision (e.g., will you use the number of ‘yes’ per intervention function / BCT as a criterion?)
Response 7.
Decisions around inclusion or exclusion of intervention functions/BCTs are based on discussion as informed by the yes/no/unsure ratings. A numeric approach is not used to reach decisions, as individual criterion can result in intervention approaches being unfeasible. For instance, if an intervention is likely effective but is not affordable, it may be untenable. The following has been added to clarify this in the methods:
“Final decisions on inclusion and exclusion of intervention functions based on APEASE criteria will be made based on team review and discussion to reach consensus.“
“As for intervention functions, decisions on inclusion or exclusion of BCTs will be based on team review and discussion to reach consensus.”

Reviewer comment 8. When it comes to BCT definitions and labels, apart from the original taxonomy perhaps the research team wants to also cross-check the recently published ontology of behaviour change interventions (see below), as I understand some BCTs have been updated in this new resource.
Marques, M. M., Wright, A. J., Corker, E., Johnston, M., West, R., Hastings, J., ... & Michie, S. (2023). The Behaviour Change Technique Ontology: Transforming the Behaviour Change Technique Taxonomy v1. Wellcome Open Research, 8.
Response 8. Thank you, the team are aware of the recently published ontology by Dr Marta Marques and colleagues, and this is consulted in the project. Reference to the ontology is now included in the methods section in relation to identifying BCTs.

Reviewer comment 9. Methods –Stage 3
Although not explicitly mentioned, I understand the consensus meeting will be conducted in English, with most (all?) participants from English-speaking countries. Will this hamper the broad applicability of findings?
Could you also please specify the expected duration of the meeting? It seems there are many aspects to be covered.
Will the APEASE criteria will be somehow introduced here? For consistency with stage 2.
Response 9. As noted above, the consensus meeting has now been removed from the project.

Reviewer comment 10. Discussion
While I agree that the use of COS will potentially improve evidence synthesis, I also see some limitations which could be interesting to elaborate further to provide a broader picture. To my understanding, COS basically refers to a checklist of outcomes that need to be reported for a given health condition. That is, COS do not provide a controlled vocabulary for such outcomes to ensure the same labels are used consistently to describe the same outcomes. This is unlike other evidence synthesis approaches (e.g., ontologies such as the BCIO) that use pre-defined entities and have the added benefit of being computer-friendly, which could be very useful to support evidence synthesis automation and leverage on AI / machine learning. In addition, COS are focused on outcomes so do not seem to address other important aspects of clinical trials that would also benefit from standardisation and a more comprehensive reporting (e.g., population targeted, methods, etc). I understand this research is not about COS but rather the use of COS (i.e., it’s a behaviour change question rather than an evidence synthesis question), but I still think authors could use the introduction and/or the discussion to provide a broader overview of COS and highlight similarities (and differences) with other evidence synthesis improvement approaches (e.g., ontologies), as well as comment on how COS relates, or could be interoperable with, common reporting guidelines (e.g., CONSORT statement).
Michie, S., West, R., Finnerty, A. N., Norris, E., Wright, A. J., Marques, M. M., ... & Hastings, J. (2020). Representation of behaviour change interventions and their evaluation: Development of the Upper Level of the Behaviour Change Intervention Ontology. Wellcome open research, 5.
Author response 10. Thank you very much for this important comment. We fully agree that this is an important topic but feel it is beyond the scope of the work described in this protocol.