The Use of Sequential Multiple Assignment Randomized Trials (SMARTs) in Physical Activity Interventions: Protocol for a Systematic Review

Aoife Whiston; Kelley M Kidwell; Siobhán O'Rielly; Cathal Walsh; John Forbes; Jane Walsh; Glynn Liam; Katie Robinson; Sara Hayes

doi:10.12688/hrbopenres.13955.1

Home Browse The Use of Sequential Multiple Assignment Randomized Trials (SMARTs)...

ALL Metrics

-

Views

12

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

The Use of Sequential Multiple Assignment Randomized Trials (SMARTs) in Physical Activity Interventions: Protocol for a Systematic Review

[version 1; peer review: 1 approved]

Aoife Whiston ¹, Kelley M Kidwell², Siobhán O'Rielly¹, [...] Cathal Walsh³, John Forbes⁴, Jane Walsh⁵, Glynn Liam⁴, Katie Robinson¹, Sara Hayes¹

Aoife Whiston ¹, Kelley M Kidwell², [...] Siobhán O'Rielly¹, Cathal Walsh³, John Forbes⁴, Jane Walsh⁵, Glynn Liam⁴, Katie Robinson¹, Sara Hayes¹

PUBLISHED 29 Jan 2025

Author details Author details

¹ Ageing Research Centre, School of Allied Health, Health Research Institute, University of Limerick, Limerick, Ireland
² School of Public Health, University of Michigan, Michigan, USA
³ TCD Biostatistics Unit, Trinity College Dublin, Dublin, Ireland
⁴ School of Medicine, Health Research Institute, University of Limerick, Limerick, Ireland
⁵ School of Psychology, University of Galway, Galway, Ireland

Aoife Whiston
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Kelley M Kidwell
Roles: Methodology, Writing – Review & Editing

Siobhán O'Rielly
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Cathal Walsh
Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing

John Forbes
Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing

Jane Walsh
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Glynn Liam
Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing

Katie Robinson
Roles: Methodology, Writing – Review & Editing

Sara Hayes
Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Introduction

Adaptive interventions involve a sequence of treatments tailored to individual responses, requiring multiple treatment decisions throughout an individual’s treatment pathway. Effective management of many chronic health conditions requires interventions adapted to individual performance. Physical activity (PA) is central to risk reduction therapies for chronic conditions. PA interventions are complex, multidimensional, and tailored to individual responses over time, requiring flexible study designs. However, PA intervention evidence is dominated by standard trial designs for non-adaptive interventions. A sequential, multiple assignment, randomised trial (SMART) design was developed to build adaptive interventions. SMARTs are factorial designs with sequential settings. Despite their potential for developing flexible interventions, SMARTs are relatively new in PA research. This review examines the state of SMART designs in PA interventions, focusing on study characteristics, design, and analysis methods.

Methods and analysis

A systematic review of SMARTs, wherein the intervention consisted of a PA intervention, will be conducted (June 2023). The following electronic databases were searched: PubMed, Embase, PsychINFO, CENTRAL, and CINAHL. The reference lists of all the identified studies will be reviewed to identify additional studies for inclusion. The titles and abstracts were independently screened by two review authors for selection. Any disagreement regarding inclusion was resolved by discussion or by referral to a third assessor. The methodological quality was assessed using the Cochrane Risk of Bias 2 tool. This review will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

Ethics and dissemination

As this systematic review will only collect secondary data, ethical approval is not required. These findings will be disseminated through academic conferences and peer-reviewed journals.

Keywords

Sequential Multiple Assignment Randomised Trial, physical activity, systematic review

Corresponding author: Aoife Whiston

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board [DIFA-023-2018].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2025 Whiston A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Whiston A, Kidwell KM, O'Rielly S et al. The Use of Sequential Multiple Assignment Randomized Trials (SMARTs) in Physical Activity Interventions: Protocol for a Systematic Review [version 1; peer review: 1 approved]. HRB Open Res 2025, 8:16 (https://doi.org/10.12688/hrbopenres.13955.1) First published: 29 Jan 2025, 8:16 (https://doi.org/10.12688/hrbopenres.13955.1) Latest published: 29 Jan 2025, 8:16 (https://doi.org/10.12688/hrbopenres.13955.1)

List of abbreviations

NCD: Non-communicable disease

PA: Physical activity

RCT: Randomised control trial

AI: Adaptive intervention

SMART: Sequential, Multiple Assignment, Randomised Trial

PRISMA: Preferred Items for Systematic Reviews and Meta-Analysis

MI: Motivational interviewing

Introduction

Health behaviors, such as physical inactivity, unhealthy eating, smoking, and excessive alcohol consumption, are leading contributors to morbidity and mortality in Europe owing to the development of non-communicable diseases (NCDs). The World Health Organization Global Action Plan urges national governments to develop non-communicable disease targets and public health strategies to improve people’s health¹. Due to the heterogeneous nature of clinical presentations among people with non-communicable diseases, such as stroke and myocardial infarction, interventions that are personalized in nature, tailored to individual needs, and adaptive to individuals’ response to treatment over time are required. Physical activity (PA) is defined as any bodily movement produced by skeletal muscles that results in energy expenditure². Cohort studies consistently support the association between PA and primary NCD prevention, such as stroke³, coronary artery disease⁴, and diabetes⁵, and PA is the cornerstone in risk-reduction therapies for the prevention and treatment of many non-communicable diseases.

PA interventions have been described as complex, multidimensional, and adaptive in nature; hence, the study designs used in their evaluation require such flexibility. Adaptive interventions involve a sequence of treatments, each adapted to individual responses, and hence, multiple treatment decisions throughout the course of an individual’s treatment pathway. Adaptive interventions have key four elements⁶: (i) decision stages, each beginning with a decision concerning treatment, and, at each stage, (ii) treatment options, (iii) tailoring variables, and (iv) decision rules. The treatment options correspond to different treatment types, dosages, or delivery options. The tailoring variables capture information about the individual used in making treatment decisions. At the beginning of each decision stage, a decision rule links the tailoring variables to specific treatment options⁶. There is a clear need for the treatment of heterogeneous lifelong NCDs to be adaptive in nature and to allow for and maximize the potential of each individual undergoing treatment. However, there is a lack of empirical data on the optimal sequence of these treatments. The evidence is dominated by standard trial designs used to evaluate non-adaptive interventions. Randomized controlled trials (RCTs) are the gold standard for examining causal relationships between interventions and outcomes in health research. Owing to the once-off randomization of participants at baseline, a standard RCT design does not enable the required flexibility to assess multiple questions regarding adaptive interventions or multiple questions regarding multiple components of an adaptive intervention.

Sequential, multiple assignment, and randomized trial (SMART) designs⁷ were developed to build and provide robust evidence for adaptive interventions. SMARTs are factorial designs in a sequential setting^6,7 and can be described as multistage RCT designs. Each stage corresponds to a decision point during the course of care. In a SMART design, individuals are randomized to an initial intervention and then may change or augment the intervention over time based on their response to the initial treatment. Randomization occurred at the beginning of the decision stages for all or some subset of participants. By enabling repeated randomization of participants to treatments, the use of SMART designs allows for causal inference across decision points in an adaptive intervention. Recently, Chow and Hampton⁸ (2022) conducted a systematic review of SMARTs (n=13) in education research. While the authors outlined some challenges in using SMARTs, such as the cost and resources required for re-randomization, they discussed the promise of the SMART design, given the dynamic nature of education research, which is dominated by diverse student populations and individualized learning interventions.

While the SMART design has the potential to address shortcomings associated with standard trial designs in the evaluation of PA interventions, questions remain about the uptake of the design, rigor, and the quality of reporting this trial design. A recent systematic review by Lorenzoni et al. examined the current evidence using SMART designs in oncology, focusing on the discrepancy between the available methodological approaches in the statistical literature and the procedures applied in cancer clinical trials⁹. Results from the 33 included records outlined that SMART designs in oncology are still limited, and that study powering and analysis is mainly based on statistical approaches traditionally used in single-stage parallel trial designs. There was substantial heterogeneity in the types of adaptive interventions designed using the SMART design, tailoring variables used, decision rules applied, and number of treatment stages. The authors concluded that formal reporting guidelines for SMART designs are needed⁹. Another systematic review (12 SMART result papers, 24 SMART protocols, and 91 methodological papers) that aimed to report the quality of SMART reporting¹⁰ demonstrated that most of the included trials did not provide the parameters required for sample size calculations and were mainly powered using stage-specific aims. Very few of the included SMARTs considered analyses involving embedded adaptive interventions¹⁰. SMART designs have received increased interest from researchers in recent years and have gained popularity owing to their real-world clinical appeal¹¹. However, despite their potential to fit into research aimed at developing flexible and adaptive interventions, SMARTs represent a relatively new trial design for the development of PA interventions.

The aim of this systematic review is to examine state-of-the-art SMART designs in PA interventions, with a focus on study characteristics, study design, and study analysis methods. This systematic review will inform the design and delivery of an SMART, wherein a PA intervention for people post-stroke will be developed.

Methods

Preferred Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines will be followed¹².

Search strategy and selection criteria

The electronic databases PubMed, Embase, PsychINFO, CENTRAL, and CINAHL were searched. For details of the search strategy, see Table 1. Reference lists of the included studies will be searched to identify potentially eligible studies, and the authors of key texts may be contacted as appropriate. The forward citations of the included studies were checked. Studies including trial registries, published protocols, and published results will be considered eligible if: (1) A SMART design is employed or planned – presence of at least 2-stages where participants could be re-randomized to subsequent interventions according to a set of predefined decision rules related to intervention or patient characteristics; and (2) a PA-based intervention is proposed or evaluated, inclusive of any intervention aimed at increasing PA or exercise including but not limited to general exercises, physical therapy, education, coaching, motivational interviewing (MI), cognitive behavioral therapy, or feedback using electronic devices (e.g., step-count). Interventions of any duration, conducted at any time, or in any setting were included. Pharmacological interventions (including over-the-counter medications) will be excluded unless PA treatments are included, wherein data can be extracted. (3) Studies proposing recruiting adults (over 18 years). We will exclude clinical trials with designs other than SMARTs. No selection criteria for the population, bar age, or outcome measures were applied.

Table 1. Search Strategy.

Search Terms- PubMed

“Sequential Multiple Assignment” OR “smart trial” OR "smart trials" OR “smart design” OR “smart designs” OR “dynamic treatment” OR “dynamic treatments” OR “sequential treatment” OR “sequential treatments” OR “adaptive treatment” OR “adaptive treatments” OR “adaptive intervention” OR “adaptive interventions” OR “multiple lines” OR “multiple line” OR “multiple assignment” OR “multiple assignments” OR “multistage treatment” OR “multi stage treatment” OR “multi stage treatments” OR “multiple treatment” OR “multiple treatments” OR “multiple therapy” OR “multiple therapies” OR “adaptive therapy” OR “adaptive therapies” OR “second randomization” OR “second randomisation” OR “re-randomization” OR “re-randomisation” OR “rerandomization” OR “rerandomisation” OR “double randomization” OR “second random assignment” OR “stepped care strategy” OR “stepped care strategies” OR “treatment sequence” OR “individualized treatment rules” OR “individualized treatment rule”

PA + Behavior Therapy
Exercise OR “physical activity” OR “exercise prescription” OR “exercise therapy” OR training OR fitness OR aerobic OR strength OR resist OR “ambulatory activity” OR walk OR workout OR “physical fitness” OR accelerometer OR sport* OR dancing OR bicycle OR “circuit based exercise” OR “behaviour therapy” OR “behavior therapy” OR “cognitive therapy” OR “Cognitive Behavioural Therapy “ OR “Cognitive Behavioral Therapy” OR “CBT” OR “behavior control” OR “behaviour control” OR “behavior intervention” OR “behaviour intervention” OR “behavior technique” OR “behaviour technique” OR “behavior training” OR “behaviour training” OR coaching OR education OR “motivational interviewing” OR “health coaching” OR “psychical activity monitoring” OR lifestyle OR “life style” OR “activity schedule” OR “activity scheduling” OR psychotherapy OR “Health Education” OR “circuit training” OR “endurance exercise” OR “endurance training” OR “exercise training” OR “exercise program” OR “exercise programme” OR “activity tracking” OR pedometer OR pedometry OR accelerometry OR “muscle exercise” OR “fitness training” OR “functional activity” OR “physical education” OR gait OR “health training” OR “health promotion” OR “activities of daily living” OR jog OR jogging OR run OR running OR kinesiotherapy OR “physical therapy” OR physiotherapy OR “resistive exercise” OR “resistive training” OR rowing OR swim OR swimming OR “strength training” OR “resistive exercise training” OR “weight training” OR “training program” OR “training programme” OR treadmill OR yoga

Data collection and extraction

References returned from each database will be imported directly into the reference managing software Rayyan, and duplicate references will be removed. Two review authors (SH and AW) will independently screen the titles and abstracts of articles for potential inclusion. The full texts of potentially eligible articles will be independently assessed against the inclusion criteria by the authors (SH and AW). Disagreements were resolved through consensus discussions with the third review author (SO’R). The results of the screening process are detailed in the PRISMA flow diagram.

Data related to study characteristics, study design, and study analysis were extracted. Study characteristics will include publication year, setting, funding, trial registration, and definition of the study design as SMART. The study design information will include the number of treatment stages, number and types of interventions delivered at each stage, tailoring variables and definitions of response used, groups that are randomized, study objectives and endpoints, and sample size reporting and calculation information. The study analysis domain will include the aims of the SMARTs, methods used for data analysis, including sample size reporting, stage-specific aims, non-responder comparisons, embedded adaptive intervention comparisons, optimal adaptive intervention testing, use of multiple testing, and analysis methods used. The data extraction tool is based on an Excel file. The extracted data were presented using a narrative summary and a table.

Risk of bias

Two review authors (S.H. and A.W.) independently assessed the internal validity of the studies using The Cochrane Risk of Bias Tool 2¹³. Disagreements were resolved by group consensus.

Discussion

This systematic review will use rigorous methodology to provide up-to-date evidence regarding the design and delivery of SMART interventions aimed at increasing PA. Interest in SMART design is growing, as evidenced by the increasing number of protocols and trial registrations in recent years. The use of SMART designs in health research remains limited, with attractive features of the design continuing to be under-deployed^8–11.

In the context of PA interventions, the heterogeneity of individual capacity and response to the intervention are important factors within and between people that necessitate the ability to account for differences in the context of intervention choice and decision-making. A “one-size-fits-all” approach, with no recognition of individual response to PA, is no longer reasonable given the dynamic nature of clinical and rehabilitation settings, the multiple factors, both within-patient characteristics and contextual and environmental factors, and the need for individualization. Given that randomized experiments provide the highest level of evidence for effectiveness¹⁴, designs that can consider individual differences as well as responses to an intervention within a randomized trial are important⁸. The findings of this systematic review will inform the future development of adaptive interventions and methods used in the development of SMART protocols for PA research. SMART designs are motivated to examine embedded adaptive interventions, and the current review will examine data analytical approaches currently being used by researchers in the field. In the absence of conduct and reporting guidelines for SMART design, the current systematic review will form the basis for formal guidelines in the field. To our knowledge, this is the first systematic review of SMARTs in the field of PA and will contribute to the evidence base by outlining the existing approaches to SMART design and analysis methods. Future SMARTs aimed at designing adaptive PA interventions among clinical populations will benefit from a systematic synopsis of current state-of-the-art research.

Dissemination

The findings will be disseminated through publications in peer-reviewed journals and conferences. The rigorous scrutiny of primary studies will identify the strengths and limitations of the current research and will provide recommendations for future research in this area.

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Declarations

Data availability statement

No data are associated with this article.

Reporting guidelines

OSF :: PRISMA-P checklist for The Use of Sequential Multiple Assignment Randomized Trials (SMARTs) in Physical Activity Interventions: Protocol for a Systematic Review. https://doi.org/10.17605/OSF.IO/HNZ7W¹⁵.

The project contains the below mentioned reporting guidelines:

▪ PRISMA-P-SystRev-checklist.docx

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication). 

Authors' contributions

Conceptualization: Sara Hayes, Jane Walsh, Cathal Walsh, John Forbes, Liam Glynn
Funding acquisition: Sara Hayes, Jane Walsh, Cathal Walsh, John Forbes, Liam Glynn, Health Research Board, Ireland.
Methodology: Sara Hayes, Aoife Whiston, Kelley M. Kidwell, Jane Walsh, Cathal Walsh, John Forbes, Liam Glynn, and Katie Robinson.
Writing – original draft: Sara Hayes, Aoife Whiston, Siobhan O’Reilly.
Writing – review & editing: Sara Hayes, Aoife Whiston, Kelley M. Kidwell, Jane Walsh, Cathal Walsh, John Forbes, Liam Glynn, Katie Robinson, and Siobhan O’Reilly.

Acknowledgements

nil

Faculty Opinions recommended

References

1. WHO: Development of an implementation roadmap 2023–2030 for the global action plan for the prevention and control of NCDS 2013–2030. 2021; 6 July 2023. Reference Source
2. Caspersen CJ, Powell KE, Christenson GM: Physical Activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep. 1985; 100(2): 126–31. PubMed Abstract | Free Full Text
3. Kubota Y, Evenson KR, Maclehose RF, et al.: Physical Activity and lifetime risk of cardiovascular disease and cancer. Med Sci Sports Exerc. 2017; 49(8): 1599–605. PubMed Abstract | Publisher Full Text | Free Full Text
4. Sattelmair J, Pertman J, Ding EL, et al.: Dose response between Physical Activity and risk of coronary heart disease: a meta-analysis. Circulation. 2011; 124(7): 789–95. PubMed Abstract | Publisher Full Text | Free Full Text
5. Alves AJ, Viana JL, Cavalcante SL, et al.: Physical Activity in primary and secondary prevention of cardiovascular disease: overview updated. World J Cardiol. 2016; 8(10): 575–83. PubMed Abstract | Publisher Full Text | Free Full Text
6. Almirall D, Nahum-Shani I, Sherwood NE, et al.: Introduction to SMART designs for the development of Adaptive Interventions: with application to weight loss research. Transl Behav Med. 2014; 4(3): 260–74. PubMed Abstract | Publisher Full Text | Free Full Text
7. Murphy SA: An experimental design for the development of adaptive treatment strategies. Stat Med. 2005; 24(10): 1455–81. PubMed Abstract | Publisher Full Text
8. Hampton LH, Chow JC: Deeply tailoring Adaptive Interventions: enhancing knowledge generation of SMARTs in special education. Remedial Spec Educ. 2022; 43(3): 195–205. Publisher Full Text
9. Lorenzoni G, Petracci E, Scarpi E, et al.: Use of Sequential Multiple Assignment Randomized Trials (SMARTs) in oncology: systematic review of published studies. Br J Cancer. 2023; 128(7): 1177–1188. PubMed Abstract | Publisher Full Text | Free Full Text
10. Bigirumurame T, Uwimpuhwe G, Wason J: Sequential Multiple Assignment Randomized Trial studies should report all key components: a systematic review. J Clin Epidemiol. 2022; 142: 152–60. PubMed Abstract | Publisher Full Text | Free Full Text
11. Kidwell KM, Almirall D: Sequential, Multiple Assignment, Randomized Trial designs. JAMA. 2023; 329(4): 336–337. PubMed Abstract | Publisher Full Text | Free Full Text
12. David M, Alessandro L, Jennifer T, et al.: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. Ann Intern Med. 2009; 151(4): 264–9. PubMed Abstract | Publisher Full Text
13. Sterne JAC, Savović J, Page MJ, et al.: RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019; 366: l4898. PubMed Abstract | Publisher Full Text
14. Hariton E, Locascio JJ: Randomised controlled trials - the gold standard for effectiveness research: study design: randomised controlled trials. BJOG. 2018; 125(13): 1716. PubMed Abstract | Publisher Full Text | Free Full Text
15. Whiston A: The use of Sequential Multiple Assignment Randomized Trials (SMARTs) in Physical Activity interventions: protocol for a systematic review. OSF, 22 Jan 2025. http://www.doi.org/10.17605/OSF.IO/HNZ7W

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 29 Jan 2025