An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol

Molly Byrne; Jenny McSharry; Oonagh Meade; Kim L. Lavoie; Simon L. Bacon

doi:10.12688/hrbopenres.12795.2

Home Browse An international, Delphi consensus study to identify priorities for...

ALL Metrics

Views

128

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Revised

An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol

[version 2; peer review: 2 approved]

Molly Byrne ¹, Jenny McSharry¹, Oonagh Meade¹, Kim L. Lavoie^2,3, Simon L. Bacon^3,4

Molly Byrne ¹, Jenny McSharry¹, [...] Oonagh Meade¹, Kim L. Lavoie^2,3, Simon L. Bacon^3,4

PUBLISHED 23 Jul 2018

Author details Author details

¹ Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland
² Department of Psychology, University of Quebec at Montreal, Montreal, QC, H2L 2C4, Canada
³ Montreal Behavioural Medicine Centre, CIUSSS-NIM- Hopital du Sacre-Coeur de Montreal, Montreal, QC, H4B 1R6, Canada
⁴ Department of Exercise Science, Concordia University, Montreal, QC, H3G 1M8, Canada

Molly Byrne
Roles: Conceptualization, Data Curation, Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jenny McSharry
Roles: Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Oonagh Meade
Roles: Methodology, Writing – Review & Editing

Kim L. Lavoie
Roles: Funding Acquisition, Methodology, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Simon L. Bacon
Roles: Investigation, Methodology, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Effective behaviour change interventions are needed to impact important health outcomes, including morbidity and mortality. However, the uptake and impact of behavioural interventions have been limited by methodological challenges. The International Behavioural Trials Network (IBTN) was established in 2013 to facilitate global improvement in methodological quality of behavioural trials. There has been no formal process, within the network or in the broader literature, to define the most important research priorities to achieve this aim. In this project, we will conduct an international, Delphi consensus study to identify and achieve consensus on priorities for methodological research in behavioural trials among IBTN members.
Methods: Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions. The IBTN Research Prioritisation team (the authors) will review all items generated, removing duplicates and merging similar topics, and generate a ‘long-list’ of items. This long-list will be sent to the 15 IBTN core members for approval.
We will then administer two online Delphi surveys to all IBTN members. In the first survey, respondents will be asked to rate the importance of each item on a nine-point scale and rank their top five priorities. In the second survey, respondents will receive feedback on others’ responses and a reminder of their own responses in survey 1, and will be asked to re-rate items and re-select their ‘top five’.
Discussion: Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.

Keywords

Behaviour change interventions, Research prioritisation, Randomised controlled trials, Methodological research, Delphi study

Corresponding author: Molly Byrne

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board Ireland [RL-2013-8], Research Leadership Award 2013 to MB.
Ireland Canada University Foundation James M. Flaherty Visiting Professor Award 2016-17 to MB.
The IBTN was developed with funding from the Canadian Institutes of Health Research [MPE 309504] and is supported by SLB's CIHR SPOR Chair [SMC 383472] and KLL's UQAM Behavioral Medicine Chair and Salary Award from the Fonds de la recherche du Québec – Santé (FRQS).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2018 Byrne M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Byrne M, McSharry J, Meade O et al. An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol [version 2; peer review: 2 approved]. HRB Open Res 2018, 1:11 (https://doi.org/10.12688/hrbopenres.12795.2) First published: 12 Mar 2018, 1:11 (https://doi.org/10.12688/hrbopenres.12795.1) Latest published: 23 Jul 2018, 1:11 (https://doi.org/10.12688/hrbopenres.12795.2)

Revised Amendments from Version 1

Based on reviewers’ reports on version 1, we made the following changes to our manuscript for version 2:
1. Some words have been changed, e.g. in the abstract, ‘brainstorm’ was changed to ‘generate’; in the abstract, ‘develop’ was changed to ‘inform’.
2. We added a referenced sentence to the first paragraph of the introduction, to acknowledge that behavioural interventions may also be digitally delivered.
3. In the first paragraph of the methods section, we added a reference to back up our claim that the Delphi approach has been widely used in health research.
4. We added the following sentence at the end of the description in the methods section of the Delphi, phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’
5. We provided a reference for Limesurvey software in the methods section.
6. We added the following sentence to clarify what was asked of participants within the e-Delphi survey: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’
7. We added new text to the methods section to clarify the procedure of including new items in survey 2 of items generated in the free-text comment box in survey 1.
8. In the dissemination section, we removed ‘at the IBTN Conference 2018’ and added detail about how data will be presented as follows: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported. The findings will be written up following recent reporting guidelines..’.

See the authors' detailed response to the review by Stephan Dombrowski
See the authors' detailed response to the review by Claire Pentecost

Introduction

There is now compelling evidence that behaviour change is central to global reduction of chronic disease prevalence, mortality, and burden of disease¹. However, despite the significant potential for interventions that target behaviour to improve health and clinical outcomes, the reach and impact of behavioural interventions remains limited². Behavioural interventions are “interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”³. Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors⁴. Experts in the area have concluded that the uptake and impact of behavioural interventions have been limited by methodological challenges specific to the design and conduct of behavioural trials³ and a lack of investment from funders to conduct research to tackle these issues⁵.

To address these challenges, the International Behavioural Trial Network (IBTN) was established by a team of behavioural researchers in June 2013 with three main goals: first, to facilitate the global improvement of the quality of behavioural trials; second, to create networks and capacity to undertake more and higher quality trials; and third, to develop a repository of resources of existing recommendations, tools, and methodology papers on behavioural trials and intervention development.

Improving the quality and potential of behavioural trials requires methodological issues in this area to be identified and research conducted with the specific aim of addressing these issues. Previously identified methodological issues include intervention development and piloting, intervention reporting, identifying suitable comparison groups, selection of appropriate outcome measures and intervention fidelity³. A formal process to identify and specify methodological priorities is now needed to facilitate the development of an international and cohesive behavioural trials research agenda.

Research prioritisation provides a process for key stakeholders to generate ideas and reach consensus on important research topics. The prioritisation process has been used to identify priorities across conditions and populations^6,7. In the area of trials, research prioritisation has been conducted with Directors of UK Clinical Research Collaboration Clinical Trials Units to inform the broader trials methodological research agenda⁸. In addition, a priority setting exercise has recently been reported to inform the global health trials methodology research agenda⁹.

Our aim is to identify priorities for methodology research specific to trials of behavioural interventions and to inform and guide the direction of the behavioural trials research agenda internationally. We will use a Delphi priority setting consensus approach, inviting all members of the IBTN to participate.

Methods

The Delphi process

An electronic Delphi (e-Delphi), with online administration of questionnaires, will be used for this research prioritisation to facilitate international participation¹⁰. The Delphi process is a structured group facilitation technique to obtain consensus among anonymous respondents through iterative rounds with feedback¹¹. The Delphi approach has been widely used in health research¹². The features of the Delphi process that make it suitable for gaining consensus include: anonymity to facilitate balanced participation and iterative rounds to allow participants to change their opinion in response to controlled feedback where participants are provided with information on the distribution of overall group responses from previous rounds¹². The Delphi process has been identified as valuable for large groups or when participants are separated geographically¹².

Participants

Three levels of participants will be included in the process.

The IBTN Research Prioritisation Team (Molly Byrne, Jenny Mc Sharry, Oonagh Meade, Simon Bacon and Kim Lavoie) will develop the research prioritisation tasks, pilot test tasks and manage the research prioritisation process.
Core IBTN members (n = 15), selected by the research team as individuals within the Network with extensive expertise in behavioural trials, will be invited to participate in the expert topic generation process.
Current IBTN members (n = 295) will be invited to take part in the two-stage e-Delphi survey.

Delphi phases

Delphi Phase 1: Expert topic generation. A member of the research team (MB) will email core IBTN members and experts in behavioural trials (n = 15) to invite them to generate a list of all possible topics or research questions that they think are important for behavioural trials methodology research. Demographic details of participants including job title and years of experience will also be requested.

Two members of the research team (MB and JMS) will review generated items initially and propose a ‘long-list’ of items. These will be reviewed, discussed and approved by the full ITBN Research Prioritisation Team. This list will be sent by email to the full list of core IBTN members for comment and approval in advance of Phase 2. The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.

Delphi Phase 2: E-Delphi survey. All members of the IBTN will be invited by email to participate in two online surveys, using LimeSurvey online survey software¹³. In survey 1, participants rate their agreement with the importance of the items identified in Phase 1. Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions. They will be asked to assign a score between 1 and 9 to indicate the priority they place on each topic. The cut-off scales proposed by the RAND group and used in similar research prioritisation exercises⁸ will be used, with scores of 1–3 indicating a topic is not important, scores of 4 to 6 indicating that the topic is important but not critical, and scores of 7 to 9 indicating that the topic is critical. In addition, participants will be asked to select and rank their ‘top-five’ overall priorities. Participants will also be invited to add any items that they believe are important and missing from the list in a free text comment box at the end of the survey. Participants will be asked to provide demographic information including: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions.

In the second survey (administered 3 weeks after the closing of survey 1), participants will receive information about how others rated and ranked the items in survey 1 (a bar chart plotting total ratings for each item, the mean rating score for each item and the percentage of all respondents placing each item in their ‘top-five’ priority selection). They will also receive a reminder of how they rated each item and what items they selected and ranked as their ‘top-five’. Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item. Respondents will be asked to re-rate each item and re-select and rank items to indicate their ‘top-five’ priorities. New items added to survey 2 will be rated only once. Ratings and levels of consensus from survey 1 and 2, as well as endorsement of items as ‘top-five’, will be collated and presented.

Ethical approval

Ethical approval was granted by the National University of Ireland Galway Research Ethics Committee (reference: 17-Jun-13). Participation in the surveys is taken to indicate participant consent; this is explicitly stated on the introductory page of the surveys.

Dissemination of research

Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network. All data will be stored anonymously on password protected computers to which only members of the research team have access. Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported. The findings will be written up following recent reporting guidelines¹⁴ for publication in a high quality, open-access international journal to promote visibility of the findings among behavioural trialists internationally, enabling them to focus research on high priority methodological questions, which will improve the quality of behavioural trials into the future.

Current study status

The first e-Delphi survey was launched in late January 2018.

Data availability

No raw data are associated with this article.

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board Ireland [RL-2013–8], Research Leadership Award 2013 to MB.

Ireland Canada University Foundation James M. Flaherty Visiting Professor Award 2016–17 to MB.

The IBTN was developed with funding from the Canadian Institutes of Health Research [MPE 309504] and is supported by SLB’s CIHR SPOR Chair [SMC 383472] and KLL’s UQAM Behavioral Medicine Chair and Salary Award from the Fonds de la recherche du Québec – Santé (FRQS).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgements

We acknowledge the assistance of Dr Geneviève Szczepanik and Ms Thalie Labonté, of the Montreal Behavioural Medicine Centre, who helped with online survey support.

F1000 recommended

References

1. Lim SS, Vos T, Flaxman AD, et al.: A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012; 380(9859): 2224–60. PubMed Abstract | Publisher Full Text | Free Full Text
2. Michie S, Thomas J, Johnston M, et al.: The Human Behaviour-Change Project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Implement Sci. 2017; 12(1): 121. PubMed Abstract | Publisher Full Text | Free Full Text
3. Bacon SL, Lavoie KL, Ninot G, et al.: An international perspective on improving the quality and potential of behavioral clinical trials. Curr Cardiovasc Risk Rep. 2015; 9(1): 427. Publisher Full Text
4. Kraft P, Yardley L: Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychol Health. 2009; 24(6): 615–8. PubMed Abstract | Publisher Full Text
5. Michie S: Designing and implementing behaviour change interventions to improve population health. J Health Serv Res Policy. 2008; 13 Suppl 3: 64–9. PubMed Abstract | Publisher Full Text
6. Petit-Zeman S, Firkins L, Scadding JW: The James Lind Alliance: tackling research mismatches. Lancet. 2010; 376(9742): 667–9. PubMed Abstract | Publisher Full Text
7. Mc Sharry J, Fredrix M, Hynes L, et al.: Prioritising target behaviours for research in diabetes: Using the nominal group technique to achieve consensus from key stakeholders. Res Involv Engagem. 2016; 2(1): 14. PubMed Abstract | Publisher Full Text | Free Full Text
8. Tudur Smith C, Hickey H, Clarke M, et al.: The trials methodological research agenda: results from a priority setting exercise. Trials. 2014; 15(1): 32. PubMed Abstract | Publisher Full Text | Free Full Text
9. Rosala-Hallas A, Bhangu A, Blazeby J, et al.: Global health trials methodological research agenda: results from a priority setting exercise. Trials. 2018; 19(1): 48. PubMed Abstract | Publisher Full Text | Free Full Text
10. Donohoe H, Stellefson M, Tennant B: Advantages and limitations of the e-Delphi technique: Implications for health education researchers. Am J Health Educ. 2012; 43(1): 38–46. Publisher Full Text
11. McMillan SS, King M, Tully MP: How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3): 655–62. PubMed Abstract | Publisher Full Text | Free Full Text
12. Jones J, Hunter D: Consensus methods for medical and health services research. BMJ. 1995; 311(7001): 376–80. PubMed Abstract | Publisher Full Text | Free Full Text
13. Limesurvey GmbH: LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. Reference Source
14. Jünger S, Payne SA, Brine J, et al.: Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med. 2017; 31(8): 684–706. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 12 Mar 2018

Author details Author details

Molly Byrne
Roles: Conceptualization, Data Curation, Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jenny McSharry
Roles: Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Oonagh Meade
Roles: Methodology, Writing – Review & Editing

Kim L. Lavoie
Roles: Funding Acquisition, Methodology, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Simon L. Bacon
Roles: Investigation, Methodology, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board Ireland [RL-2013-8], Research Leadership Award 2013 to MB.
Ireland Canada University Foundation James M. Flaherty Visiting Professor Award 2016-17 to MB.
The IBTN was developed with funding from the Canadian Institutes of Health Research [MPE 309504] and is supported by SLB's CIHR SPOR Chair [SMC 383472] and KLL's UQAM Behavioral Medicine Chair and Salary Award from the Fonds de la recherche du Québec – Santé (FRQS).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 23 Jul 2018, 1:11

https://doi.org/10.12688/hrbopenres.12795.2

version 1

Published: 12 Mar 2018, 1:11

https://doi.org/10.12688/hrbopenres.12795.1

© 2018 Byrne M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

1082

downloads

128

Citations

SEE MORE DETAILS

CITE

how to cite this article

Byrne M, McSharry J, Meade O et al. An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol [version 2; peer review: 2 approved]. HRB Open Res 2018, 1:11 (https://doi.org/10.12688/hrbopenres.12795.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 23 Jul 2018

Revised

Views

Reviewer Report 01 Aug 2018

Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

Approved

https://doi.org/10.21956/hrbopenres.13921.r26327

I am satisfied with the ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 12 Mar 2018

Views

Reviewer Report 28 Jun 2018

Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

Approved

https://doi.org/10.21956/hrbopenres.13855.r26134

This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have ... Continue reading

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.
The methods should state that the generated lists will be summarised and approved by the research team.
Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?
Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?
Will you be following any Delphi quality reporting guidelines?
How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Behaviour change and other complex interventions

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

23 Jul 2018

Author Response
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and ... Continue reading
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 2
DOI: 10.21956/hrbopenres.13855.r26134
This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have some minor suggestions to improve the paper.

Abstract

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.

Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The methods should state that the generated lists will be summarised and approved by the research team.

Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Authors’ response: The sentence in the abstract has been changed as follows, using the word ‘inform’ rather than ‘develop’: ‘Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.’
Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Authors’ response: We acknowledge that the terms ‘behavioural trials’ has significant overlap with the term ‘complex intervention trials’. Indeed, it could be argued that the terms could be used interchangeably, that all behavioural trials are complex, and that all complex intervention trials involve behaviour change. In this section of the protocol, we reference a paper specifically discussing challenges to ‘behaviour change interventions’, so we have used this terminology here. In the final reporting of this study, we will incorporate a more elaborate discussion around the definition of behaviour change intervention trial, and how it relates to complex interventions. However, we consider such a discussion to be beyond the scope of this protocol paper.
Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?

Authors’ response: We have added the following citation for LimeSurvry: Limesurvey GmbH. / LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. URL http://www.limesurvey.org

Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?

Authors’ response: The sentence in the methods section has been changed as follows: ‘Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item.’

Will you be following any Delphi quality reporting guidelines?

Authors’ response: The following reference has now been included and we have added that the results will be written up in accordance with these reporting guidelines: Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine. 2017;31(8):684-706.

How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Authors’ response: The following sentences have been added to the ‘Dissemination of Research’ section: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported.’

Is the rationale for, and objectives of, the study clearly described?Yes

Is the study design appropriate for the research question?Yes

Are sufficient details of the methods provided to allow replication by others?Yes

Are the datasets clearly presented in a useable and accessible format?Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Behaviour change and other complex interventions
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 2
DOI: 10.21956/hrbopenres.13855.r26134
This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have some minor suggestions to improve the paper.

Abstract

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.

Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The methods should state that the generated lists will be summarised and approved by the research team.

Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Authors’ response: The sentence in the abstract has been changed as follows, using the word ‘inform’ rather than ‘develop’: ‘Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.’
Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Authors’ response: We acknowledge that the terms ‘behavioural trials’ has significant overlap with the term ‘complex intervention trials’. Indeed, it could be argued that the terms could be used interchangeably, that all behavioural trials are complex, and that all complex intervention trials involve behaviour change. In this section of the protocol, we reference a paper specifically discussing challenges to ‘behaviour change interventions’, so we have used this terminology here. In the final reporting of this study, we will incorporate a more elaborate discussion around the definition of behaviour change intervention trial, and how it relates to complex interventions. However, we consider such a discussion to be beyond the scope of this protocol paper.
Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?

Authors’ response: We have added the following citation for LimeSurvry: Limesurvey GmbH. / LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. URL http://www.limesurvey.org

Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?

Authors’ response: The sentence in the methods section has been changed as follows: ‘Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item.’

Will you be following any Delphi quality reporting guidelines?

Authors’ response: The following reference has now been included and we have added that the results will be written up in accordance with these reporting guidelines: Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine. 2017;31(8):684-706.

How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Authors’ response: The following sentences have been added to the ‘Dissemination of Research’ section: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported.’

Is the rationale for, and objectives of, the study clearly described?Yes

Is the study design appropriate for the research question?Yes

Are sufficient details of the methods provided to allow replication by others?Yes

Are the datasets clearly presented in a useable and accessible format?Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Behaviour change and other complex interventions
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

23 Jul 2018

Author Response
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and ... Continue reading
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 2
DOI: 10.21956/hrbopenres.13855.r26134
This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have some minor suggestions to improve the paper.

Abstract

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.

Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The methods should state that the generated lists will be summarised and approved by the research team.

Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Authors’ response: The sentence in the abstract has been changed as follows, using the word ‘inform’ rather than ‘develop’: ‘Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.’
Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Authors’ response: We acknowledge that the terms ‘behavioural trials’ has significant overlap with the term ‘complex intervention trials’. Indeed, it could be argued that the terms could be used interchangeably, that all behavioural trials are complex, and that all complex intervention trials involve behaviour change. In this section of the protocol, we reference a paper specifically discussing challenges to ‘behaviour change interventions’, so we have used this terminology here. In the final reporting of this study, we will incorporate a more elaborate discussion around the definition of behaviour change intervention trial, and how it relates to complex interventions. However, we consider such a discussion to be beyond the scope of this protocol paper.
Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?

Authors’ response: We have added the following citation for LimeSurvry: Limesurvey GmbH. / LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. URL http://www.limesurvey.org

Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?

Authors’ response: The sentence in the methods section has been changed as follows: ‘Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item.’

Will you be following any Delphi quality reporting guidelines?

Authors’ response: The following reference has now been included and we have added that the results will be written up in accordance with these reporting guidelines: Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine. 2017;31(8):684-706.

How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Authors’ response: The following sentences have been added to the ‘Dissemination of Research’ section: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported.’

Is the rationale for, and objectives of, the study clearly described?Yes

Is the study design appropriate for the research question?Yes

Are sufficient details of the methods provided to allow replication by others?Yes

Are the datasets clearly presented in a useable and accessible format?Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Behaviour change and other complex interventions
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 2
DOI: 10.21956/hrbopenres.13855.r26134
This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have some minor suggestions to improve the paper.

Abstract

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.

Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The methods should state that the generated lists will be summarised and approved by the research team.

Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Authors’ response: The sentence in the abstract has been changed as follows, using the word ‘inform’ rather than ‘develop’: ‘Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.’
Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Authors’ response: We acknowledge that the terms ‘behavioural trials’ has significant overlap with the term ‘complex intervention trials’. Indeed, it could be argued that the terms could be used interchangeably, that all behavioural trials are complex, and that all complex intervention trials involve behaviour change. In this section of the protocol, we reference a paper specifically discussing challenges to ‘behaviour change interventions’, so we have used this terminology here. In the final reporting of this study, we will incorporate a more elaborate discussion around the definition of behaviour change intervention trial, and how it relates to complex interventions. However, we consider such a discussion to be beyond the scope of this protocol paper.
Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?

Authors’ response: We have added the following citation for LimeSurvry: Limesurvey GmbH. / LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. URL http://www.limesurvey.org

Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?

Authors’ response: The sentence in the methods section has been changed as follows: ‘Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item.’

Will you be following any Delphi quality reporting guidelines?

Authors’ response: The following reference has now been included and we have added that the results will be written up in accordance with these reporting guidelines: Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine. 2017;31(8):684-706.

How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Authors’ response: The following sentences have been added to the ‘Dissemination of Research’ section: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported.’

Is the rationale for, and objectives of, the study clearly described?Yes

Is the study design appropriate for the research question?Yes

Are sufficient details of the methods provided to allow replication by others?Yes

Are the datasets clearly presented in a useable and accessible format?Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Behaviour change and other complex interventions
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 20 Apr 2018

Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK

Approved

https://doi.org/10.21956/hrbopenres.13855.r26131

This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Partly
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Health psychology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

23 Jul 2018

Author Response
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We ... Continue reading
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 4
DOI: 10.21956/hrbopenres.13855.r26131
This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.
Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?
Authors’ response: We have added the following sentence following the definition of behaviourial interventions to address this point: ‘Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors.’ (Ref Kraft P, Yardley L. Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychology & Health. 2009;24(6):615-8.)

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.
Authors’ response: The following reference has been added: Jones J, Hunter D. Consensus methods for medical and health services research. BMJ : British Medical Journal. 1995;311(7001):376-80.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.
Authors’ response: The following demographic data will be collected from participants at time 1: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions. As these are not going to change between survey time 1 and time 2, we do not plan to collect this data again for survey 2.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.
Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?
Authors’ response: The following sentence has been added: ‘New items added to survey 2 will be rated only once.’

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.
Authors’ response: The following sentence has been added: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.
Authors’ response: This sentence has been changed as follows: Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network.

Is the rationale for, and objectives of, the study clearly described?

Yes

Is the study design appropriate for the research question?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Health psychology
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 4
DOI: 10.21956/hrbopenres.13855.r26131
This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.
Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?
Authors’ response: We have added the following sentence following the definition of behaviourial interventions to address this point: ‘Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors.’ (Ref Kraft P, Yardley L. Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychology & Health. 2009;24(6):615-8.)

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.
Authors’ response: The following reference has been added: Jones J, Hunter D. Consensus methods for medical and health services research. BMJ : British Medical Journal. 1995;311(7001):376-80.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.
Authors’ response: The following demographic data will be collected from participants at time 1: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions. As these are not going to change between survey time 1 and time 2, we do not plan to collect this data again for survey 2.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.
Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?
Authors’ response: The following sentence has been added: ‘New items added to survey 2 will be rated only once.’

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.
Authors’ response: The following sentence has been added: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.
Authors’ response: This sentence has been changed as follows: Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network.

Is the rationale for, and objectives of, the study clearly described?

Yes

Is the study design appropriate for the research question?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Health psychology
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

23 Jul 2018

Author Response
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We ... Continue reading
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 4
DOI: 10.21956/hrbopenres.13855.r26131
This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.
Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?
Authors’ response: We have added the following sentence following the definition of behaviourial interventions to address this point: ‘Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors.’ (Ref Kraft P, Yardley L. Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychology & Health. 2009;24(6):615-8.)

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.
Authors’ response: The following reference has been added: Jones J, Hunter D. Consensus methods for medical and health services research. BMJ : British Medical Journal. 1995;311(7001):376-80.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.
Authors’ response: The following demographic data will be collected from participants at time 1: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions. As these are not going to change between survey time 1 and time 2, we do not plan to collect this data again for survey 2.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.
Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?
Authors’ response: The following sentence has been added: ‘New items added to survey 2 will be rated only once.’

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.
Authors’ response: The following sentence has been added: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.
Authors’ response: This sentence has been changed as follows: Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network.

Is the rationale for, and objectives of, the study clearly described?

Yes

Is the study design appropriate for the research question?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Health psychology
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 4
DOI: 10.21956/hrbopenres.13855.r26131
This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.
Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?
Authors’ response: We have added the following sentence following the definition of behaviourial interventions to address this point: ‘Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors.’ (Ref Kraft P, Yardley L. Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychology & Health. 2009;24(6):615-8.)

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.
Authors’ response: The following reference has been added: Jones J, Hunter D. Consensus methods for medical and health services research. BMJ : British Medical Journal. 1995;311(7001):376-80.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.
Authors’ response: The following demographic data will be collected from participants at time 1: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions. As these are not going to change between survey time 1 and time 2, we do not plan to collect this data again for survey 2.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.
Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?
Authors’ response: The following sentence has been added: ‘New items added to survey 2 will be rated only once.’

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.
Authors’ response: The following sentence has been added: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.
Authors’ response: This sentence has been changed as follows: Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network.

Is the rationale for, and objectives of, the study clearly described?

Yes

Is the study design appropriate for the research question?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Health psychology
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 12 Mar 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 23 Jul 18		read
Version 1 12 Mar 18	read	read

Stephan Dombrowski, University of Stirling, Stirling, UK
Claire Pentecost, University of Exeter Medical School, Exeter, UK

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

10 Views

01 Aug 2018 | for Version 2

Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

10 Views Cite this report Responses(0)

Approved

I am satisfied with the changes that have been made.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

13 Views

28 Jun 2018 | for Version 1

Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

13 Views Cite this report Responses(1)

Approved

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.
The methods should state that the generated lists will be summarised and approved by the research team.
Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?
Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?
Will you be following any Delphi quality reporting guidelines?
How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Behaviour change and other complex interventions

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

Referee Report 28 Jun 2018
Claire Pentecost, Exeter Clinical Trials Unit, University of Exeter Medical School, Exeter, UK

The authors would like to thank Dr Pentecost for her review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 2
DOI: 10.21956/hrbopenres.13855.r26134
This is a clearly written summary of a study protocol. The findings will help health behaviour change researchers to prioritise areas of research methodology to improve the quality and impact of health behaviour change interventions.

I have some minor suggestions to improve the paper.

Abstract

‘experts in the field of behavioural intervention research, will be invited to brainstorm a complete list of all items they consider…’

This could be interpreted as they will compile a list or they will be given a list to consider. The Full methods explain that the list will be generated by the invited experts but it is not clear here.

Also the term ‘brainstorm’ Is not used in the main methods section. For consistency and clarity the term should be used in both places and or removed from the abstract.

Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The methods should state that the generated lists will be summarised and approved by the research team.

Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Should the discussion say ‘Findings from the project will be used to ‘inform’ rather than to ‘develop’ the research agenda?

Authors’ response: The sentence in the abstract has been changed as follows, using the word ‘inform’ rather than ‘develop’: ‘Findings from the project will be used to inform the research agenda of the IBTN and to make recommendations for future research.’
Introduction

The methodological issues examples given in the 3^rd paragraph are common issues for any complex interventions. To highlight the issue of methodological problems within the behaviour change field, and the importance of this Delphi consensus study, I would like to see more evidence of specific issues with behaviour change interventions that we already know about, within disease areas or types of interventions. For example fidelity in weight loss interventions.

Authors’ response: We acknowledge that the terms ‘behavioural trials’ has significant overlap with the term ‘complex intervention trials’. Indeed, it could be argued that the terms could be used interchangeably, that all behavioural trials are complex, and that all complex intervention trials involve behaviour change. In this section of the protocol, we reference a paper specifically discussing challenges to ‘behaviour change interventions’, so we have used this terminology here. In the final reporting of this study, we will incorporate a more elaborate discussion around the definition of behaviour change intervention trial, and how it relates to complex interventions. However, we consider such a discussion to be beyond the scope of this protocol paper.
Methods

5^th paragraph – does LimeSurvey software need to be cited with LTD or a web address?

Authors’ response: We have added the following citation for LimeSurvry: Limesurvey GmbH. / LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. URL http://www.limesurvey.org

Last paragraph - how will the team agree that free text items are novel? Will everyone agree or a majority? How will any disagreements be resolved?

Authors’ response: The sentence in the methods section has been changed as follows: ‘Any additional items proposed in the free-text comment box in survey 1 will be discussed by the research team and included for rating in survey 2 if the majority of team members agrees that the item is a unique, novel, previously excluded item.’

Will you be following any Delphi quality reporting guidelines?

Authors’ response: The following reference has now been included and we have added that the results will be written up in accordance with these reporting guidelines: Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine. 2017;31(8):684-706.

How do you envisage the results will be presented? A simple list or a list linked to particular disease areas, or something else?

Authors’ response: The following sentences have been added to the ‘Dissemination of Research’ section: ‘Items will be reported according to importance ratings for individual research items in surveys 1 and 2. The numbers and percentages of participants who rated each item as their top priority in surveys 1 and 2 will also be reported.’

Is the rationale for, and objectives of, the study clearly described?Yes
Is the study design appropriate for the research question?Yes
Are sufficient details of the methods provided to allow replication by others?Yes
Are the datasets clearly presented in a useable and accessible format?Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Behaviour change and other complex interventions
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

17 Views

20 Apr 2018 | for Version 1

Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK

17 Views Cite this report Responses(1)

Approved

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Partly
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Health psychology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

23 Jul 2018

Molly Byrne, Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, H91 TK33, Ireland

Authors’ responses to review by Stephan Dombrowski, Division of Psychology, University of Stirling, Stirling, UK
The authors would like to thank Dr Dombrowski for his review and helpful comments. We have addressed these as outlined in bold font below.
Approved
VIEWS 4
DOI: 10.21956/hrbopenres.13855.r26131
This is a well written protocol of a timely study, the authors are conducting important and vital work to further advance behaviour change science. I have only minor comments for consideration by the authors.

Abstract:
The authors use the term brainstorm in the abstract, but not the methods. It might be good to use the same terms for the proposed activities.
Authors’ response: The sentence in the abstract has been changed to: ‘Fifteen core members of IBTN, who are experts in the field of behavioural intervention research, will be invited to generate a list of all items they consider priority areas for methodological research in trials of behavioural interventions.’

The definition of behavioural intervention on p. 3 is very useful (“interventions that require the active participation of a target group (e.g., the patient/individual, health professional, health care systems) in a program delivered by a trained interventionist with the proximal or ultimate goal of changing health-related behaviour”). However, might this definition omit interventions which are delivered remotely and electronically, such as ehealth and mhealth interventions which are typically delivered automatically and not by a trained interventionist?
Authors’ response: We have added the following sentence following the definition of behaviourial interventions to address this point: ‘Behavioural interventions may be delivered in person or digitally, employing digital technologies such as the Internet, telephones, and mobile and environmental sensors.’ (Ref Kraft P, Yardley L. Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychology & Health. 2009;24(6):615-8.)

P. 3 – “The Delphi approach has been widely used in health research.” – please provide a reference.
Authors’ response: The following reference has been added: Jones J, Hunter D. Consensus methods for medical and health services research. BMJ : British Medical Journal. 1995;311(7001):376-80.

I wonder if the authors might want to collect the same demographic information between Phase 1 and Phase 2 to allow comparison.
Authors’ response: The following demographic data will be collected from participants at time 1: sex, current professional position, country of residence, age group and number of years of experience of working in the area of trials of behavioural interventions. As these are not going to change between survey time 1 and time 2, we do not plan to collect this data again for survey 2.

It might be useful to add a sentence on how additional comments from the 15 experts at the end of Phase 1 will be handled.
Authors’ response: The following sentence has been added at the of the methods section Phase 1: ‘The final list of items will be approved and agreed by the IBTN Research Prioritisation Team.’

Similarly, how will new items integrated into Survey 2 following Survey 1 be handled. Will these be rated only once in Survey 2?
Authors’ response: The following sentence has been added: ‘New items added to survey 2 will be rated only once.’

When the participants are asked to rate the importance, it might be useful to specify what importance relates to. Someone who is interested in theory development might rate importance different to someone primarily interested in clinical implementation of evidence. The exact phrasing of the question is thus crucial to avoid introducing unwanted variation in the provided ratings and to be specific as to what the IBTN members are asked to judge.
Authors’ response: The following sentence has been added: ‘Participants are asked to rate the perceived importance, from their own individual point of view, of the items for methodological research in trials of behavioural interventions.’

The authors might want to consider removing the year 2018 from the dissemination section (p.4) in case timelines slip and the results are not fully ready in time for this particular conference.
Authors’ response: This sentence has been changed as follows: Findings from this research prioritisation project will be shared with members of the IBTN to guide the development of a research agenda for the Network.

Is the rationale for, and objectives of, the study clearly described?

Yes

Is the study design appropriate for the research question?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Referee Expertise: Health psychology
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Lim SS, Vos T, Flaxman AD, et al.: A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012; 380(9859): 2224–60. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Michie S, Thomas J, Johnston M, et al.: The Human Behaviour-Change Project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Implement Sci. 2017; 12(1): 121. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Bacon SL, Lavoie KL, Ninot G, et al.: An international perspective on improving the quality and potential of behavioral clinical trials. Curr Cardiovasc Risk Rep. 2015; 9(1): 427. Publisher Full Text

[4] 4. Kraft P, Yardley L: Current issues and new directions in Psychology and Health: What is the future of digital interventions for health behaviour change? Psychol Health. 2009; 24(6): 615–8. PubMed Abstract | Publisher Full Text

[5] 5. Michie S: Designing and implementing behaviour change interventions to improve population health. J Health Serv Res Policy. 2008; 13 Suppl 3: 64–9. PubMed Abstract | Publisher Full Text

[6] 6. Petit-Zeman S, Firkins L, Scadding JW: The James Lind Alliance: tackling research mismatches. Lancet. 2010; 376(9742): 667–9. PubMed Abstract | Publisher Full Text

[7] 7. Mc Sharry J, Fredrix M, Hynes L, et al.: Prioritising target behaviours for research in diabetes: Using the nominal group technique to achieve consensus from key stakeholders. Res Involv Engagem. 2016; 2(1): 14. PubMed Abstract | Publisher Full Text | Free Full Text

[8] 8. Tudur Smith C, Hickey H, Clarke M, et al.: The trials methodological research agenda: results from a priority setting exercise. Trials. 2014; 15(1): 32. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Rosala-Hallas A, Bhangu A, Blazeby J, et al.: Global health trials methodological research agenda: results from a priority setting exercise. Trials. 2018; 19(1): 48. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Donohoe H, Stellefson M, Tennant B: Advantages and limitations of the e-Delphi technique: Implications for health education researchers. Am J Health Educ. 2012; 43(1): 38–46. Publisher Full Text

[11] 11. McMillan SS, King M, Tully MP: How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3): 655–62. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Jones J, Hunter D: Consensus methods for medical and health services research. BMJ. 1995; 311(7001): 376–80. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Limesurvey GmbH: LimeSurvey: An Open Source survey tool /LimeSurvey GmbH, Hamburg, Germany. Reference Source

[14] 14. Jünger S, Payne SA, Brine J, et al.: Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med. 2017; 31(8): 684–706. PubMed Abstract | Publisher Full Text

An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

The Delphi process

Participants

Delphi phases

Ethical approval

Dissemination of research

Current study status

Data availability

Competing interests

Grant information

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

An international, Delphi consensus study to identify priorities for methodological research in behavioural trials: A study protocol

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

The Delphi process

Participants

Delphi phases

Ethical approval

Dissemination of research

Current study status

Data availability

Competing interests

Grant information

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!