A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials

Marina Zaki; Marie Galligan; Lydia O'Sullivan; Declan Devane; Eilish McAuliffe

doi:10.12688/hrbopenres.13068.2

Home Browse A protocol for a systematic review investigating the factors influencing...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Revised

A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials

[version 2; peer review: 2 approved]

Marina Zaki ^1,2, Marie Galligan³, Lydia O'Sullivan^2,3, Declan Devane^2,4,5, Eilish McAuliffe¹

Marina Zaki ^1,2, Marie Galligan³, [...] Lydia O'Sullivan^2,3, Declan Devane^2,4,5, Eilish McAuliffe¹

PUBLISHED 24 Sep 2020

Author details Author details

¹ University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland
² Health Research Board – Trials Methodology Research Network (HRB-TMRN), School of Nursing and Midwifery,, National University of Ireland, Galway, Galway, Ireland
³ School of Medicine and Medical Science, University College Dublin, Dublin 4, Ireland
⁴ School of Nursing and Midwifery, National University of Ireland, Galway, Galway, Ireland
⁵ Evidence Synthesis Ireland, National University of Ireland, Galway, Galway, Ireland

Marina Zaki
Roles: Conceptualization, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Marie Galligan
Roles: Conceptualization, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Lydia O'Sullivan
Roles: Writing – Review & Editing

Declan Devane
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Eilish McAuliffe
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the HRB-TMRN gateway.

Abstract

Trials can be defined as prospective human research studies to test the effectiveness and safety of interventions, such as medications, surgeries, medical devices and other interventions for the management of patient care. Statistics is an important and powerful tool in trials. Inappropriately designed trials and/or inappropriate statistical analysis produce unreliable results, with limited clinical use. The aim of this systematic literature review is to identify, describe and synthesise factors contributing to or influencing the statistical planning, design, conduct, analysis and reporting of trials. This protocol will describe the methodological approach taken for the following: conducting a systematic and comprehensive search for relevant articles, applying eligibility criteria for the inclusion of such articles, extracting data and information, appraising the quality of the articles, and thematically synthesizing the data to illuminate the key factors influencing statistical aspects of trials.

Keywords

Clinical trials, statistics, systematic review, protocol

Corresponding author: Marina Zaki

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board Ireland [TMRN-2014-1]. This work is also funded and supported by the School of Nursing, Midwifery and Health Systems, University College Dublin, Ireland. The funders of this doctoral research had no role in developing this protocol.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 Zaki M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Zaki M, Galligan M, O'Sullivan L et al. A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials [version 2; peer review: 2 approved]. HRB Open Res 2020, 3:36 (https://doi.org/10.12688/hrbopenres.13068.2) First published: 08 Jun 2020, 3:36 (https://doi.org/10.12688/hrbopenres.13068.1) Latest published: 24 Sep 2020, 3:36 (https://doi.org/10.12688/hrbopenres.13068.2)

Revised Amendments from Version 1

Protocol version 2 includes clarification on the included sources, in light of constructive comments from the reviewers. The term 'trial' has been used throughout the protocol, and replaces 'clinical trial' and 'randomised trial', to capture the breadth of articles included in the review. The title of the protocol has been amended to reflect this also. The use of 'backwards citation screening' was also included and briefly explained under 'information sources'. The originally proposed timeline for collecting sources has been removed.

See the authors' detailed response to the review by Nadeem Gadelseyed and Basil Elnazir
See the authors' detailed response to the review by Dawid Pieper

Abbreviations

FDA: Food and Drug Association; ICH GCP: International Conference on Harmonisation Good Clinical Practice; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; WHO: World Health Organization

Introduction

Clinical trials are research studies that test the safety and efficacy of novel treatments and interventions. Their effects on the health outcomes of humans are evaluated prospectively (WHO, 2020). Such medical interventions include drugs, cells (and other biological products), surgical procedures, medical devices, behavioural treatments, radiological procedures and measures for preventative care (WHO, 2020). Findings from trials have the potential to change clinical practice. For this reason and to minimise harm to patients, trials must be conducted to a very high standard.

The Merriam-Webster Dictionary describes statistics as "a branch of mathematics dealing with the collection, analysis, interpretation, and presentation of masses of numerical data" (Merriam-Webster Dictionary, 2020). To put this definition into context, it implies: refining the study design in order to efficiently address the study’s research hypothesis while minimising bias, defining data to be collected, appropriately analysing data collected, and interpreting results in such a way as to facilitate clinical decision making.

Clinical data management (CDM) facilitates the statistical aspects in a trial. CDM activity in a trial involves the appropriate collection, management, access to and cleaning of clinical research data. The integrity of statistical analysis in a trial ultimately depends on the quality of the data that is available for analysis (Adams-Huet & Ahn, 2009; Krishnankutty et al., 2012; Nesbitt, 2004, p.135). The validity of a clinical research study is therefore not solely judged on its results, but also on how the study itself was designed and conducted (Adams-Huet & Ahn, 2009). The key team members responsible for the quality of statistical outputs from a trial are statisticians (/biostatisticians), clinical data managers and principal investigators (PIs). It is important to note the importance of, not only their roles and responsibilities, but their qualifications, experience, and training and how they work with other team members, and the potential impact these may have on the statistics of a trial.

There is also a relationship between statistics and ethics in trials. Ethical issues can affect not only the design of a trial, but all stages. If the statistical considerations of a trial are inadequate, the research will be unethical. This misuse of statistics in the clinical research field may have consequences for trial participants (Altman, 1980), and other resources, including researchers’ time and effort. It is also unethical to publish and disseminate statistical results that may be misleading (Altman, 1980).

Therefore, it can be deduced that statistics play an important role in all aspects of a trial, from planning, design, conduct, analysis through to reporting and publication. There is a plethora of literature discussing statistical methodologies in trials from a mathematical perspective. Less has been written about the trial team and other resources and how these may be influencing the statistical planning, design, conduct, analysis and reporting of trial data. Therefore, there remains a gap in the literature that needs to be addressed; systematically reviewing articles to fully comprehend, collate, and thematically synthesise the factors influencing the statistical aspects of trials is required.

This protocol will describe the methodological details for:

conducting a comprehensive search for relevant articles,
applying justified inclusion and exclusion (eligibility) criteria,
evaluating and appraising the included studies,
thematically synthesising the findings of the included studies.

Research question

What are the factors influencing the statistical planning, design, conduct, analysis and reporting in trials?

Rationale and aim of this review

The aim of this systematic review is to identify and describe factors contributing to and influencing the planning, design, conduct, analysis and reporting of trials, from a statistical perspective.

Objectives

To construct a systematic search strategy incorporating the relevant elements at the centre of the research question,
To identify, evaluate and critically appraise peer-reviewed literature (and relevant non-peer reviewed sources) that discuss factors contributing to the statistical aspects of a trial,
To develop an explanatory framework through a narrative thematic synthesis, for how these factors may influence the statistical planning, design, conduct, analysis and reporting of trials,
To present recommendations (based on the findings of this review) for how the statistical aspects of trials can be improved.

Methods

The protocol for this systematic review is not deemed eligible for registration in the PROSPERO database, as the outcome of this review is not a health-related one. This systematic review will investigate aspects of a trial methodological nature and will not investigate specific outcomes of a clinical relevance.

This protocol follows the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols; Moher et al., 2015) guidelines. The PRISMA-P checklist can be found in Figshare (Zaki et al., 2020c).

Criteria for considering studies for this review

Phenomenon of interest. The phenomenon of interest are the factors influencing the statistical planning, design, conduct, analysis or reporting of trials.

Papers will be included if they discuss any one or more of the following factors:

Roles, responsibilities, and tasks of key team members responsible for the statistical elements of the planning, design, conduct, analysis and reporting of trials
Qualifications, training, knowledge, experience and professional development of key team members responsible for the statistical planning, design, conduct, analysis and reporting of trials
Processes of communication and collaboration between key team members, and with others, responsible for the statistical planning, design, conduct, analysis and reporting of trials

Sources that do not meet the eligibility criteria will be excluded. If the key term 'trial' is not included in the full text, the article is not included in the final stage of the screening process.

While novel statistical methodologies, for example novel trial designs, can contribute to the statistical aspects of a trial, the focus of this review is on other factors influencing the planning, design, conduct, analysis and reporting of a trial.

Articles will be excluded if they focus solely on:

Pre-clinical research studies (animal/non-human studies)
Evaluation of effectiveness of an intervention for the prevention or treatment or clinical diseases/conditions
Cost-effectiveness articles or ones of a health economics perspective, not influencing the statistics of a trial
Regulatory articles, with no reference to factors influencing the statistical planning, design, conduct, analysis or reporting of randomised trials
Statistical theory

Types of studies. We will include all primary studies of any study design including studies that collect or analyse primary data through means of a secondary analysis. A preliminary scope of the literature revealed a lack of empirical literature in this field and therefore we will also include discussion/commentary papers.

Search methodology

Keyword search terms are informed by conducting a preliminary scope of the literature. The search strategy for this review is created and refined with the assistance of an information specialist (health sciences liaison librarian), experienced in systematic reviews. Key words are first identified, followed using controlled vocabulary search terms in individual databases.

The SPICE (Setting – Perspectives – Intervention – Comparison – Evaluation) framework is considered to be the best fit for the search strategy required in this systematic review, as the individual search features are suitable for the type of search in this review (see below). Other frameworks such as PICO (Population-Intervention-Comparison-Outcome), SPIDER (Sample-Phenomenon of Interest-Design-Evaluation-Research Type) and CIMO (Context-Intervention-Mechanism-Outcome) (Booth, 2016) were considered but not deemed appropriate. The search strings (for S, P, I and E) can be found in Table 1 below. An optional search string, Comparison (C), is not included, as the aim is not to compare interventions or study designs but to extract information from each article.

Table 1. Keyword search terms.

Feature	Search Terms
Setting (S)	"clinical research"
Perspective (P)	Statistician* OR biostatistician* OR "data manager" OR investigator OR "trial team" OR "clinical research team" OR "statistics team" OR "data management team"
Intervention (I)	"clinical trial*" OR trial OR trials
Evaluation (E)	Statistics OR biostatistics OR data OR statistical OR planning OR design OR conduct OR analysis OR analyses OR reporting

While the Setting (S) of the search terms is "clinical research", the Intervention (I) and focus of the review is on trials, not other clinical studies such as observational research. With regards to the Evaluation (E), the key elements from the research question (planning, design, conduct, analysis (or the plural ‘analyses’) and reporting) are included as these are the aspects to be evaluated and from where the relevant factors can be extracted. As statistics is the central phenomenon of this systematic review, the terms ‘statistics’ and ‘statistical’ are crucial. An alternative term for statistics, ‘biostatistics’ is included to ensure articles that mention this term instead of ‘statistics’ are not missed. Similarly, the search term ‘data’ is included to retrieve articles discussing data aspects that may influence the statistics of a trial.

Within each of the search term features, the keywords are combined using the ‘OR’ Boolean operators between them. This allows for broadening the search. Methods used to narrow the search while also using the Boolean operator includes the concept ‘AND’ between the features (see Table 1). Before the search is run in its entirety, search terms are individually assessed and run through the databases.

A systematic literature search is performed on the following electronic databases: PubMed NCBI, Web of Science, PsycINFO and CINAHL. This search in the bibliographic databases retrieves peer reviewed literature.

A pre-defined set of controlled vocabulary terms are used in the individual databases. These terms index, describe and categorise sources of information and ensures a comprehensive search in the databases. In the CINAHL database, such controlled vocabulary terms are known as 'Subject Headings'. In PubMed, the Advanced Search Builder finds alternative terms used by that database to describe a term. Such terms are known as MeSH (Medical Subject Headings). The list of controlled vocabulary terms in the PyscINFO database are known as the 'Thesaurus of Psychological Index Terms'. Web of Science is the only database included in this search strategy that does not have controlled vocabulary terms. If controlled vocabulary terms are appropriate and relevant, they are included for that database. Articles of interest retrieved in the preliminary scope of the literature are also used to locate MeSH terms of relevance in PubMed. These controlled vocabulary terms, while often the same as the original keyword search terms, are included to maximise the retrieval of potentially relevant articles. Individual MeSH/controlled vocabulary/thesauri terms from the individual databases can be found in Extended Data (Zaki et al., 2020a).

To ensure a comprehensive search is conducted, the review is not limited geographically and so there are no restrictions on the locations/countries where articles are published.

The searches are also not limited by publication date. No database filters or restrictions are placed on gender or age of participants discussed in the articles. Due to limited translation resources, searches are limited to only include articles written in English. This systematic review is limited to peer-reviewed articles retrieved in the bibliographic databases and non-peer reviewed sources from the grey literature.

Information sources

While it is expected that most of the articles relevant for this systematic review can be obtained from the systematic bibliographic database search discussed, important sources may also be retrieved from grey searches. This search will therefore retrieve unpublished or non-peer reviewed literature sources. Open Grey EU is the grey literature database that will be used in this review. Backwards citation screening will also be conducted in this review. Relevant articles from the reference list of included articles, that meet the eligibility criteria, are included.

Guidelines relevant to the scope of the review are also important sources of information. Guidelines that will be included in this systematic review are:

The FDA, PSI, ICH GCP, ASA and CONSORT websites will be listed as professional organisations.

If additional information (not explicitly stated in the articles) is required, direct contact will be made with the study authors of the articles.

Data collection and analysis

Selection of studies. Two reviewers (MZ and LOS) will independently screen title and abstracts against the inclusion criteria. Disagreements will be resolved through discussion with a third reviewer (EM). The full text of papers included at title and abstract stage will also be assessed by two reviewers with recourse to a third (EM) should disagreements occur. A summarised synthesis will be provided of reasons all papers at full text stage were excluded.

Data extraction and management. Citations from database searches will be imported into EndNote and then merged and de-duplicated. Microsoft Excel will be used for data extraction.

Key characteristics and important features of the included articles will be extracted using a previously piloted structured data extraction form, in Microsoft Excel (template provided in Extended Data (Zaki et al., 2020b)). In brief, this file presents the key features (‘data items’, in accordance with PRISMA-P) that will be extracted from the full text of the included articles. 10–20% of the data extraction will be checked for accuracy and thoroughness by a second researcher (EM).

Quality assessment

Once the researcher is aware of the types of articles to be included in the systematic review, it will then be decided whether the Critical Appraisal Skills Programme (CASP, 2020) checklists or Mixed Methods Appraisal Tools (MMAT, 2018) will be used to assess the quality of included articles. This will determine the strength of the body of evidence. The results of this appraisal will then be used to inform the synthesis and the interpretation of the results as well as the discussion of the findings of this review.

Data analysis and synthesis

As this review is not aiming to assess the risk of bias in any particular intervention or appraise the methodological quality of trials, the data analysis and synthesis will be limited to a thematic/narrative synthesis approach. While quantitative articles will be included in the review, their quantitative outcomes will not be assessed for heterogeneity or quantitatively synthesised in the form of a meta-analysis. There is also no planned assessment for meta-biases in the studies that are included in this review.

The aim of the data extraction, analysis and synthesis process is to extract factors influencing statistics, statistical processes, statistical interactions, and other statistical aspects in all stages of a trial. For this review, ‘factors’ can be defined as any element impacting, influencing, or contributing to any aspect of statistical planning, design, conduct, analysis and reporting in trials. A narrative thematic synthesis approach will be used to collate and describe the key information from all included sources. This approach has been used to analyse data and information extracted from both quantitative and qualitative studies (Ryan et al., 2018).

Each stage of the analysis will be documented, as it is important to maintain transparency about how patterns within the data are identified, and to note any similarities or differences between sources. A coding frame will be presented. Similarities between codes will be identified, labelled and categorised into themes. Narrative descriptions of each theme will be provided. The final outcome of the thematic synthesis will be an overall account of the factors influencing the statistical planning, design, conduct, analysis and reporting in trials. Findings across articles will be synthesised and their meanings interpreted to answer the research question (Ryan et al., 2018). Recommendations, where provided in the articles, will be emphasised. This review will also illuminate areas that are under-researched and where there is scope for future work. One reviewer (MZ) will undertake each of the steps of data extraction, synthesis and comparison of findings across information sources, and the work will be checked by a second researcher (EM). Feedback from a Research Studies Panel (MG, DD, EM) will be included in the final version of the systematic review.

Potential limitations

This review will only include studies that are published in English, due to limitations in translation resources. This could mean excluding other relevant information based on language barriers. Secondly, unpublished literature will not be included, possibly leaning towards an increased risk of publication bias in the research that is included.

Tracked and dated amendments

Any amendments to this protocol, including the dates of the amendments and justifications, will be documented and presented in a table in the systematic review publication.

Dissemination

The findings of the systematic review will be published in a peer-reviewed journal upon completion. This systematic review will be of interest not only to statisticians (biostatisticians) and data managers, but also PIs and other researchers and healthcare professionals working in the trials field.

Conclusion

The aim of this systematic review is to identify, describe and synthesise factors contributing to or influencing the statistical planning, design, conduct, analysis and reporting of trials. This will be conducted through a systematic search of the literature in four bibliographic databases: PubMed NCBI, Web of Science, PsycINFO and CINAHL. The eligibility criteria for including relevant articles is discussed along with the plans for critical appraisal, data extraction and data synthesis. The systematic review and thematic synthesis will be written in accordance with the PRISMA statement for systematic reviews. The data extraction and findings will be tabulated to present key features. Findings will then be synthesised narratively. The findings will inform healthcare professionals and researchers of key factors revolving around trial statistics and whether such factors can be addressed to better understand the methodology of trials.

Data availability

Underlying data

No data is associated with this article.

Extended data

Figshare Repository: Controlled Vocabulary Search Terms Bibliographic Databases, https://doi.org/10.6084/m9.figshare.12403421.v1, (Zaki et al., 2020a).

Figshare Repository: Data Extraction Template for Systematic Review, https://doi.org/10.6084/m9.figshare.12403562.v1, (Zaki et al., 2020b).

Reporting guidelines

Figshare: PRISMA-P checklist for “A protocol for a systematic review investigating factors influencing statistical planning, design, conduct, analysis and reporting of clinical trials”, https://doi.org/10.6084/m9.figshare.12264938.v1 (Zaki et al., 2020c).

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

Acknowledgements

The authors acknowledge the support of the Health Research Board and the Health Research Board – Trials Methodology Research Network and the School of Nursing, Midwifery & Health Systems, University College Dublin for providing the scholarship funding to the corresponding author for a doctoral study.

The authors also wish to thank Mr. Diarmuid Stokes, health sciences liaison librarian at the University College Dublin, for his assistance in creating and refining the search strategy for this systematic review.

Faculty Opinions recommended

References

Adams-Huet B, Ahn C: Bridging clinical investigators and statisticians: writing the statistical methodology for a research proposal. J Investig Med. 2009; 57(8): 818–824. PubMed Abstract | Publisher Full Text | Free Full Text
Altman DG: 'Statistics and ethics in medical research: Misuse of statistics is unethical' BMJ. 1980; 281(6249): 1182–4. PubMed Abstract | Publisher Full Text | Free Full Text
Booth A: Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016; 5: 74. PubMed Abstract | Publisher Full Text | Free Full Text
Krishnankutty B, Bellary S, Kumar NBR, et al.: 'Data management in clinical research: An overview', Indian J Pharmacol. 2012; 44(2): 168–172. PubMed Abstract | Publisher Full Text | Free Full Text
Moher D, Liberati A, Tetzlaff J, et al.: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text
Nesbitt LA: Clinical Research: What It Is and How It Works. MA, USA: Jones and Bartlett Publishers, 2004; xvi–xviii. pp.135–136. Reference Source
Ryan C, Hesselgreaves H, Wu O, et al.: Protocol for a systematic review and thematic synthesis of patient experiences of central venous access devices in anti-cancer treatment. Syst Rev. 2018; 7(1): 61. PubMed Abstract | Publisher Full Text | Free Full Text
Zaki M, Galligan M, O'Sullivan L, et al.: Controlled Vocabulary Search Terms Bibliographic Databases. figshare. Figure. 2020a. http://www.doi.org/10.6084/m9.figshare.12403421.v1
Zaki M, Galligan M, O'Sullivan L, et al.: Data Extraction Template for Systematic Review. figshare. Figure. 2020b. http://www.doi.org/10.6084/m9.figshare.12403562.v1
Zaki M, Galligan M, O'Sullivan L, et al.: PRISMA-P-checklist Systematic Review Protocol - Statistical Factors in Clinical Trials.doc. figshare. Figure. 2020c. http://www.doi.org/10.6084/m9.figshare.12264938.v1

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 08 Jun 2020

Author details Author details

Marina Zaki
Roles: Conceptualization, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Marie Galligan
Roles: Conceptualization, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Lydia O'Sullivan
Roles: Writing – Review & Editing

Declan Devane
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Eilish McAuliffe
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board Ireland [TMRN-2014-1]. This work is also funded and supported by the School of Nursing, Midwifery and Health Systems, University College Dublin, Ireland. The funders of this doctoral research had no role in developing this protocol.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 24 Sep 2020, 3:36

https://doi.org/10.12688/hrbopenres.13068.2

version 1

Published: 08 Jun 2020, 3:36

https://doi.org/10.12688/hrbopenres.13068.1

© 2020 Zaki M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

1171

downloads

Citations

SEE MORE DETAILS

CITE

how to cite this article

Zaki M, Galligan M, O'Sullivan L et al. A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials [version 2; peer review: 2 approved]. HRB Open Res 2020, 3:36 (https://doi.org/10.12688/hrbopenres.13068.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 08 Jun 2020

Views

Reviewer Report 24 Aug 2020

Nadeem Gadelseyed, Trinity College Dublin, Dublin, Ireland

Basil Elnazir, Trinity College Dublin, Dublin, Ireland; Tallaght University Hospital, Dublin, Ireland

Approved

https://doi.org/10.21956/hrbopenres.14167.r27784

I have asked Dr Nadeem Gadeleseyd to independently review this article with me and we both reached the below conclusion.

This study sets out to identify, describe and synthesise factors contributing to or influencing the statistical planning, design, conduct, analysis and reporting of clinical trials.

The article is clearly written and easy to follow. The search methodology is clear and using standard tools.

Below are few pointers the authors should attend to:

The inclusion criteria are a bit vague and confusing and I suggest rewriting this section with clear determinants and tight criteria to avoid selection bias.
The authors should describe the reason for selecting this timeline. 'The timeline for gathering sources to be included in this systematic review is from April to June 2020'.
It will be helpful if the authors would eventually consider a plan to construct a protocol template to better inform researchers and healthcare professionals around statistical aspects, planning, design, conduct, analysis and reporting of clinical trials.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Childhood Asthma, Cystic fibrosis and sleep disordered breathing.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

24 Sep 2020

Author Response

The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and ... Continue reading The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and avoiding selection bias. The new version of the protocol has been amended to reflect this. As this systematic review was part of a wider doctoral study, the timeline for gathering sources was described. However this statement has been removed in the revised version.
The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and avoiding selection bias. The new version of the protocol has been amended to reflect this. As this systematic review was part of a wider doctoral study, the timeline for gathering sources was described. However this statement has been removed in the revised version.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

24 Sep 2020

Author Response

The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and ... Continue reading The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and avoiding selection bias. The new version of the protocol has been amended to reflect this. As this systematic review was part of a wider doctoral study, the timeline for gathering sources was described. However this statement has been removed in the revised version.
The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and avoiding selection bias. The new version of the protocol has been amended to reflect this. As this systematic review was part of a wider doctoral study, the timeline for gathering sources was described. However this statement has been removed in the revised version.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 10 Aug 2020

Dawid Pieper, Institute for Research in Operative Medicine, Faculty of Health, School of Medicine, Witten/Herdecke University, Cologne, Germany

Approved

https://doi.org/10.21956/hrbopenres.14167.r27785

This is a well written protocol. I have only a few comments.

In your objectives you state that you are also interested in factors related to reporting of trials. What do you mean by that? Is that the reporting of the whole trials or does this only relate to statistical aspects. If the latter is the case, I doubt that this can be differentiated clearly as this would not only be about the methods section but also about the results and its interpretation.

In your title you state that your study is about clinical trials. This is not stated clearly in your eligibility criteria where you refer to randomised trials which is not exactly the same. Are you intending to exclude trials in psychology? Psychology might be a completely different field as psychologists are well educated in methods and statistics (in contrast to clinicians). What I am trying to say is that an average psychologist will be able to run a better methodological trial on your own than an average clinician. The latter does need the help of a statistician.

Did you involve an information specialist in drafting your search strategy? Was the search strategy checked using the PRESS guideline? I feel that searching for "clinical research" is too narrow. The term "clinical trial*" can be omitted as you are searching for "trial" and "trials" anyway. At other instances I would suggest to search for "conducting*" instead of "conduct" (the same does apply to other terms).

All in all, identifying these articles might be challenging. What about forward and backward citation screening?

I wonder about the statement that the literature will be gathered between April and June 2020 only. This would mean that the search did already take place. While I understand that researchers cannot wait to go on with their study waiting for the protocol to be published for several reasons, I would still advise the authors to check their search strategy and information sources.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Methodologist, clinical epidemiologist, meta-researcher.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

24 Sep 2020

Author Response

The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree ... Continue reading The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree that it is difficult to differentiate between the reporting of whole trials and statistical reporting (methods, results and interpretation). Therefore, we would like to clarify that we have included articles discussing the reporting of trials, in the context of statisticians/investigators roles and responsibilities, and therefore this relates back to the research question of factors influencing statistics.

2) The new version of the protocol has been revised, to address the comment regarding the eligibility criteria. The term 'trial' has now replaced 'randomised trial' and 'clinical trial', to capture the breadth of articles included in this review.

3) An information specialist was involved when drafting the search strategy. The authors acknowledge that the PRESS guideline and truncation could have been used. However, a search with the suggested truncated terms did not reveal additional articles.

4) The authors acknowledge the importance of backwards citation screening and so this will be conducted. The protocol has been amended to reflect this.

5) As this systematic review is part of a doctoral study, the aim was to have sources gathered by the stated timeline. However, this statement has now been removed and the authors acknowledge the comments about checking the information sources and search strategy.
The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree that it is difficult to differentiate between the reporting of whole trials and statistical reporting (methods, results and interpretation). Therefore, we would like to clarify that we have included articles discussing the reporting of trials, in the context of statisticians/investigators roles and responsibilities, and therefore this relates back to the research question of factors influencing statistics.

2) The new version of the protocol has been revised, to address the comment regarding the eligibility criteria. The term 'trial' has now replaced 'randomised trial' and 'clinical trial', to capture the breadth of articles included in this review.

3) An information specialist was involved when drafting the search strategy. The authors acknowledge that the PRESS guideline and truncation could have been used. However, a search with the suggested truncated terms did not reveal additional articles.

4) The authors acknowledge the importance of backwards citation screening and so this will be conducted. The protocol has been amended to reflect this.

5) As this systematic review is part of a doctoral study, the aim was to have sources gathered by the stated timeline. However, this statement has now been removed and the authors acknowledge the comments about checking the information sources and search strategy.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

24 Sep 2020

Author Response

The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree ... Continue reading The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree that it is difficult to differentiate between the reporting of whole trials and statistical reporting (methods, results and interpretation). Therefore, we would like to clarify that we have included articles discussing the reporting of trials, in the context of statisticians/investigators roles and responsibilities, and therefore this relates back to the research question of factors influencing statistics.

2) The new version of the protocol has been revised, to address the comment regarding the eligibility criteria. The term 'trial' has now replaced 'randomised trial' and 'clinical trial', to capture the breadth of articles included in this review.

3) An information specialist was involved when drafting the search strategy. The authors acknowledge that the PRESS guideline and truncation could have been used. However, a search with the suggested truncated terms did not reveal additional articles.

4) The authors acknowledge the importance of backwards citation screening and so this will be conducted. The protocol has been amended to reflect this.

5) As this systematic review is part of a doctoral study, the aim was to have sources gathered by the stated timeline. However, this statement has now been removed and the authors acknowledge the comments about checking the information sources and search strategy.
The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree that it is difficult to differentiate between the reporting of whole trials and statistical reporting (methods, results and interpretation). Therefore, we would like to clarify that we have included articles discussing the reporting of trials, in the context of statisticians/investigators roles and responsibilities, and therefore this relates back to the research question of factors influencing statistics.

2) The new version of the protocol has been revised, to address the comment regarding the eligibility criteria. The term 'trial' has now replaced 'randomised trial' and 'clinical trial', to capture the breadth of articles included in this review.

3) An information specialist was involved when drafting the search strategy. The authors acknowledge that the PRESS guideline and truncation could have been used. However, a search with the suggested truncated terms did not reveal additional articles.

4) The authors acknowledge the importance of backwards citation screening and so this will be conducted. The protocol has been amended to reflect this.

5) As this systematic review is part of a doctoral study, the aim was to have sources gathered by the stated timeline. However, this statement has now been removed and the authors acknowledge the comments about checking the information sources and search strategy.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 08 Jun 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 24 Sep 20
Version 1 08 Jun 20	read	read

Dawid Pieper, Witten/Herdecke University, Cologne, Germany
Nadeem Gadelseyed, Trinity College Dublin, Dublin, Ireland

Basil Elnazir, Trinity College Dublin, Dublin, Ireland; Tallaght University Hospital, Dublin, Ireland

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

23 Views

24 Aug 2020 | for Version 1

Nadeem Gadelseyed, Trinity College Dublin, Dublin, Ireland

Basil Elnazir, Trinity College Dublin, Dublin, Ireland; Tallaght University Hospital, Dublin, Ireland

23 Views Cite this report Responses(1)

Approved

The inclusion criteria are a bit vague and confusing and I suggest rewriting this section with clear determinants and tight criteria to avoid selection bias.
The authors should describe the reason for selecting this timeline. 'The timeline for gathering sources to be included in this systematic review is from April to June 2020'.
It will be helpful if the authors would eventually consider a plan to construct a protocol template to better inform researchers and healthcare professionals around statistical aspects, planning, design, conduct, analysis and reporting of clinical trials.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Childhood Asthma, Cystic fibrosis and sleep disordered breathing.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

The authors would like to greatly thank Professor Elnazir and Dr. Gadelseyed for their constructive comments and suggestions on this protocol. We acknowledge the reviewer's comments regarding the eligibility criteria and avoiding selection bias. The new version of the protocol has been amended to reflect this. As this systematic review was part of a wider doctoral study, the timeline for gathering sources was described. However this statement has been removed in the revised version.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

36 Views

10 Aug 2020 | for Version 1

Dawid Pieper, Institute for Research in Operative Medicine, Faculty of Health, School of Medicine, Witten/Herdecke University, Cologne, Germany

36 Views Cite this report Responses(1)

Approved

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Methodologist, clinical epidemiologist, meta-researcher.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

24 Sep 2020

Marina Zaki, University College Dublin, Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS Centre), School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin 4, Ireland

The authors would like to greatly thank Dr. Pieper for the constructive comments on this protocol. The authors would like to clarify and address a number of points:

1) We agree that it is difficult to differentiate between the reporting of whole trials and statistical reporting (methods, results and interpretation). Therefore, we would like to clarify that we have included articles discussing the reporting of trials, in the context of statisticians/investigators roles and responsibilities, and therefore this relates back to the research question of factors influencing statistics.

2) The new version of the protocol has been revised, to address the comment regarding the eligibility criteria. The term 'trial' has now replaced 'randomised trial' and 'clinical trial', to capture the breadth of articles included in this review.

3) An information specialist was involved when drafting the search strategy. The authors acknowledge that the PRESS guideline and truncation could have been used. However, a search with the suggested truncated terms did not reveal additional articles.

4) The authors acknowledge the importance of backwards citation screening and so this will be conducted. The protocol has been amended to reflect this.

5) As this systematic review is part of a doctoral study, the aim was to have sources gathered by the stated timeline. However, this statement has now been removed and the authors acknowledge the comments about checking the information sources and search strategy.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] Adams-Huet B, Ahn C: Bridging clinical investigators and statisticians: writing the statistical methodology for a research proposal. J Investig Med. 2009; 57(8): 818–824. PubMed Abstract | Publisher Full Text | Free Full Text

[2] Altman DG: 'Statistics and ethics in medical research: Misuse of statistics is unethical' BMJ. 1980; 281(6249): 1182–4. PubMed Abstract | Publisher Full Text | Free Full Text

[3] Booth A: Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016; 5: 74. PubMed Abstract | Publisher Full Text | Free Full Text

[4] Krishnankutty B, Bellary S, Kumar NBR, et al.: 'Data management in clinical research: An overview', Indian J Pharmacol. 2012; 44(2): 168–172. PubMed Abstract | Publisher Full Text | Free Full Text

[5] Moher D, Liberati A, Tetzlaff J, et al.: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text

[6] Nesbitt LA: Clinical Research: What It Is and How It Works. MA, USA: Jones and Bartlett Publishers, 2004; xvi–xviii. pp.135–136. Reference Source

[7] Ryan C, Hesselgreaves H, Wu O, et al.: Protocol for a systematic review and thematic synthesis of patient experiences of central venous access devices in anti-cancer treatment. Syst Rev. 2018; 7(1): 61. PubMed Abstract | Publisher Full Text | Free Full Text

[8] Zaki M, Galligan M, O'Sullivan L, et al.: Controlled Vocabulary Search Terms Bibliographic Databases. figshare. Figure. 2020a. http://www.doi.org/10.6084/m9.figshare.12403421.v1

[9] Zaki M, Galligan M, O'Sullivan L, et al.: Data Extraction Template for Systematic Review. figshare. Figure. 2020b. http://www.doi.org/10.6084/m9.figshare.12403562.v1

[10] Zaki M, Galligan M, O'Sullivan L, et al.: PRISMA-P-checklist Systematic Review Protocol - Statistical Factors in Clinical Trials.doc. figshare. Figure. 2020c. http://www.doi.org/10.6084/m9.figshare.12264938.v1

A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials

Abstract

Keywords

Revised Amendments from Version 1

Abbreviations

Introduction

Research question

Rationale and aim of this review

Objectives

Methods

Criteria for considering studies for this review

Search methodology

Table 1. Keyword search terms.

Information sources

Data collection and analysis

Quality assessment

Data analysis and synthesis

Potential limitations

Tracked and dated amendments

Dissemination

Conclusion

Data availability

Underlying data

Extended data

Reporting guidelines

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

A protocol for a systematic review investigating the factors influencing the statistical planning, design, conduct, analysis and reporting of trials

Abstract

Keywords

Revised Amendments from Version 1

Abbreviations

Introduction

Research question

Rationale and aim of this review

Objectives

Methods

Criteria for considering studies for this review

Search methodology

Table 1. Keyword search terms.

Information sources

Data collection and analysis

Quality assessment

Data analysis and synthesis

Potential limitations

Tracked and dated amendments

Dissemination

Conclusion

Data availability

Underlying data

Extended data

Reporting guidelines

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!