Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review

Fiona Riordan; Geoffrey M. Curran; Cara C. Lewis; Byron J. Powell; Justin Presseau; Luke Wolfenden; Sheena M. McHugh

doi:10.12688/hrbopenres.13507.1

Home Browse Characterising processes and outcomes of tailoring implementation...

ALL Metrics

Views

144

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review

[version 1; peer review: 2 approved with reservations]

Fiona Riordan ¹, Geoffrey M. Curran², Cara C. Lewis³, [...] Byron J. Powell⁴, Justin Presseau⁵, Luke Wolfenden⁶, Sheena M. McHugh¹

Fiona Riordan ¹, Geoffrey M. Curran², [...] Cara C. Lewis³, Byron J. Powell⁴, Justin Presseau⁵, Luke Wolfenden⁶, Sheena M. McHugh¹

PUBLISHED 03 Mar 2022

Author details Author details

¹ School of Public Health, University College Cork, Cork, Cork, T12 XF62, Ireland
² Department of Psychiatry, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
³ Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
⁴ Brown School, Washington University in St. Louis, St. Louis, Missouri, USA
⁵ Clinical Epidemiology, Ottawa Hospital Research Institute, Ottawa, Canada
⁶ School of Medicine and Public Health, Faculty of Health and Medicine, University of Newcastle,, New South Wales, Australia

Fiona Riordan
Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Geoffrey M. Curran
Roles: Conceptualization, Methodology, Writing – Review & Editing

Cara C. Lewis
Roles: Conceptualization, Methodology, Writing – Review & Editing

Byron J. Powell
Roles: Conceptualization, Methodology, Writing – Review & Editing

Justin Presseau
Roles: Conceptualization, Methodology, Writing – Review & Editing

Luke Wolfenden
Roles: Conceptualization, Methodology, Writing – Review & Editing

Sheena M. McHugh
Roles: Conceptualization, Methodology, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Tailoring strategies to target the salient barriers to and enablers of implementation is considered a critical step in supporting successful delivery of evidence based interventions in healthcare. Theory, evidence, and stakeholder engagement are considered key ingredients in the process however, these ingredients can be combined in different ways. There is no consensus on the definition of tailoring or single method for tailoring strategies to optimize impact, ensure transparency, and facilitate replication.
Aim: The purpose of this scoping review is to describe how tailoring has been undertaken within healthcare to answer questions about how it has been conceptualised, described, and conducted in practice, and to identify research gaps.
Methods: The review will be conducted in accordance with best practice guidelines and the Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for scoping reviews (PRISMA-ScR) will be used to guide the reporting. Searches will be conducted of MEDLINE, Embase, Web of Science, Scopus, from 2005 to present. Reference lists of included articles will be searched. Grey literature will be searched on Google Scholar. Screening and data extraction will be conducted by two or more members of the research team, with any discrepancies resolved by consensus discussion with a third reviewer. Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies and the tailoring process. Qualitative content analysis aligned to the research questions will also be conducted, and data managed using NVivo where applicable. This scoping review is pre-registered with the Open Science Framework.
Conclusions: The findings will serve a resource for implementation researchers and practitioners to guide future research in this field and facilitate systematic, transparent, and replicable development of tailored implementation strategies.

Keywords

implementation science, implementation strategies, tailoring, scoping review, protocol

Corresponding author: Fiona Riordan

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board [HRB RL-2020-004].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2022 Riordan F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Riordan F, Curran GM, Lewis CC et al. Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review [version 1; peer review: 2 approved with reservations]. HRB Open Res 2022, 5:17 (https://doi.org/10.12688/hrbopenres.13507.1) First published: 03 Mar 2022, 5:17 (https://doi.org/10.12688/hrbopenres.13507.1) Latest published: 20 Sep 2022, 5:17 (https://doi.org/10.12688/hrbopenres.13507.2)

Introduction

Adoption and subsequent health impact of evidence-based interventions (EBIs) is facilitated by the careful use of implementation strategies which are “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”¹. It is recommended that such strategies are purposefully selected and designed to better address the specific needs within a given implementation context, be these at the level of the patient, provider, organisation, and/or system. Numerous possible implementation strategies have been proposed. For example, 73 discrete strategies were compiled by the Expert Recommendations on Implementing Change (ERIC) study². However, a process is necessary to choose between them and operationalise them for a given setting. Tailoring is one process which involves selecting and/or modifying strategies to address contextual factors that potentially influence implementation³. It is considered a critical step in supporting successful delivery of EBIs in healthcare⁴.

However, there are varying definitions and conceptualisations of tailoring and it has been suggested that tailoring is a “poorly developed concept” within implementation science⁵. The term ‘tailoring’ has most commonly⁴ been used to describe a proactive approach to customize the method of implementation (i.e. the strategy), however, the term has also be used to describe the process of modifying or personalising strategies to fit with different population subgroups^6–8, or adapting strategies during delivery⁹.

Tailoring as a process is also somewhat ambiguous as the timing, level, scope and methods of application varies and there is no generally accepted guiding principles for tailoring to ensure transparency and facilitate replication. There are two levels to how the tailoring process is operationalised: (1) the individual steps to take, by whom, when and how often (timing), and (2) the overall approach, or way in which those steps are taken, including the research methods and data collection techniques used. The steps included in the tailoring process vary, with studies including some or all of the following: (a) the identification of (barriers and enablers) implementation determinants, (b) prioritisation of determinants, and (c) selection of strategies by matching to determinants^6–8,10 and modifying or adjusting these strategies³. It is also unclear whether application and testing of strategies forms part of the tailoring process^3,9,11. It is generally recommended that there should be some identification of implementation determinants, but in isolation that is insufficient; there should be a corresponding process to select strategies to address those factors⁴. In terms of the timing, it is unclear whether tailoring is an initial design process that should take place before deployment of an implementation strategy, as specified by Baker et al.,⁴ or whether it can be an iterative process that continues during strategy deployment as implementation challenges arise^12,13. The Template for Intervention Description and Replication (TIDieR) checklist distinguishes between tailoring (personalisation and planned adaptations) and modification (unplanned modifications over the course of the study)¹⁴. This contrasts with studies that have used the Knowledge-To-Action (KTA)¹⁵ translation cycle to engage in iterative tailoring to modify strategies, or the Model of Implementation which has been used to undertake more than one round of tailoring, informing further changes to the original strategy in response to emerging barriers^16,17. The Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) refers to the modification of strategies after deployment as ongoing tailoring.¹⁸, while efforts to identify and develop practical tools for tracking strategies, distinguish between tailoring as prospective design of strategies, and adaptations as ‘deliberate modifications’ to the treatment or how it is delivered¹⁹.

The overall approach to tailoring, that is, how tailoring has been applied, the way in which the steps have been delineated and the research methods (e.g. intervention mapping) and data collection techniques used (e.g., surveys, focus groups), has not been consistent or well described³. A Cochrane review of the effectiveness of tailored implementation strategies, found tailoring was undertaken in different ways, the methods used varied substantially, as did the rationale for their selection, they lacked detail, and trials often did not outline the underpinning rationale for tailoring⁴. Powell et al.,³ described four approaches that have been used for prospective tailoring ranging from multiphase approaches (intervention mapping) to specific analytic tools (concept mapping, conjoint analysis and group model building). These approaches, which often involve several steps and types of data, were selected on the basis that they had been used to develop interventions in contexts outside of implementation and behavioural science, there is extensive literature to guide their use, and they are not proprietary. While existing approaches to tailoring, like those outlined by Powell et al., share common elements (theory, evidence, and stakeholder engagement) and steps (determinant identification, determinant prioritisation, selection of strategies to address determinants, and application of the selected strategy), there is no consensus on how to combine these elements of tailoring and what each element should involve^3,20. Work has been conducted as part of the Tailored Implementation in Chronic Diseases (TICD) study to investigate the methods used to identify determinants, concluding that brainstorming is a feasible approach which can yield a high number of plausibly important determinants²¹. However, there have been calls for greater clarity on how determinants are prioritised, and how strategies are selected²².

Purpose of conducting the scoping review

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is recommended to support the successful delivery of EBIs. Research to date has been systematically reviewed to establish the effectiveness of tailored strategies, the outcome of the tailoring process. The literature has not been reviewed to map the description and application of this process, to establish a clear picture of how tailoring has been conceptualised and conducted in practice, including the contexts in which tailoring has been used, when it has been used, and who has been involved in the process³. Our aim is to address this gap by conducting a scoping review to: 1) explore how tailoring has been defined and conceptualised in the healthcare literature; 2) examine how tailoring has been operationalised within the healthcare context (timing, level and scope); 3) determine how tailoring has been evaluated in the context of healthcare; and 4) identify knowledge gaps and future research priorities. As the aim of this review is to explore how tailoring has been defined and conceptualised, we adopt a broad definition of tailoring as a proactive process to develop the method of implementation, and/or modifications of a strategy to fit with different population subgroups or emerging barriers. Scoping reviews are defined by Colquhoun et al., as “a form of knowledge synthesis that addresses an exploratory research question aimed at mapping key concepts, types of evidence, and gaps in research related to a defined area or field by systematically searching, selecting, and synthesizing existing knowledge”²³ providing essentially a ‘map of the evidence’²⁴. By clarifying how tailoring is conceptualised and summarising approaches to tailoring, this review will serve as a resource for the implementation research and practice community who are designing implementation strategies to address a particular implementation gap in healthcare. It will identify research priorities on tailoring and may serve as a starting point from which to develop and agree a consolidated definition of tailoring.

Protocol

Design

The review will be conducted in line with the framework and principles of Arksey and O’Malley²⁵ and the refinements proposed by Levac et al.,²⁴ We will also draw on updated guidance developed by the Joanna Briggs Institute (JBI) institute²⁶. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA- ScR) guidelines will be followed. There will be six stages in the review process: 1) Identifying the research questions; 2) Identifying relevant studies; 3) Study selection; 4) Data extraction/charting the data; 5) Collating, summarising, and reporting results, and 6) Stakeholder consultation.

Stage 1: identifying the research questions

A preliminary search of relevant literature was undertaken in one database (searching MEDLINE) using “tailoring” and “implementation” and “strategy” or “intervention” to generate an initial understanding of how tailoring has been operationalised. These findings (e.g., existing editorials and reviews of tailoring, examples of tailoring approaches) were used to inform the background for the review, refine the scope of the review, generate eligibility criteria, and to develop the search strategy.

The following objectives and research questions were identified through this iterative process:

Objective 1: Conceptualisation

1. How has tailoring been defined and conceptualised in healthcare contexts?
2. Is there a rationale for tailoring and what is the rationale?
3. Is tailoring based on a theory which explains why it should work?

Objective 2: Operationalisation

1. What steps comprise the tailoring process?
2. Which research methods (e.g., intervention mapping, conjoint analysis) and data collection techniques have been used?
3. Which theoretical frameworks have been drawn on and at which steps(s)?
4. When was tailoring carried out (stage of implementation)?
5. Where was tailoring been carried out (in which healthcare settings)
6. Who was involved, when (what step(s)), for what purpose, and how?
7. How was the prioritisation of determinants and selection of strategies conducted?
- a. What methods have been used and what is the rationale?
- b. What are the inputs during strategy selection – evidence, stakeholders, theory?
- c. What is the balance and timing of each input?

Objective 3: Evaluation

1. How has the tailoring process been evaluated in healthcare?
2. Which study designs, including comparison conditions, have been used to evaluate tailoring?
3. What proximal outcomes of tailoring “success” have been used?

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring.

Stage 2: identifying relevant studies

Eligibility criteria

The ‘‘PCC’’ mnemonic (population, concept, and context) is recommended by the Joanna Briggs Institute to construct clear inclusion criteria for scoping review and identify the focus and context of the review^26,27.

Population

All participants who are involved in a tailoring process will be considered, including healthcare professionals, researchers, and people who use healthcare services.

Concept

In defining the concept for this review, we made several decisions to make the process more feasible (Table 1). Studies which use a tailoring process to develop implementation strategies in healthcare will be included. As one of our objectives is to examine how tailoring has been conceptualised, we will adopt a broad definition of the process for the purpose of identifying relevant studies. To address objective 1 (conceptualisation), we will consider primary studies and commentaries or editorials which discuss how tailoring should be done and/or why. To address objectives 2 (operationalisation) and 3 (application), we will only consider primary studies which use tailoring as a proactive process to develop the method of implementation, or modifications of a strategy to fit with different population subgroups (e.g., socio-demographic characteristics; age, gender, culture, socio-economic status) or individual sites. For primary studies to be considered tailoring, approaches must at least involve the selection of strategies. While there is variation in the scope of tailoring (i.e., which steps are involved, whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing), strategy selection is a core aspect. Authors must explicitly describe the process as tailoring. In line with guidance specified by Levac et al.,²⁴ who advise the need to balance breadth and comprehensiveness with feasibility, we place these bounds on the concept to manage the scope of the review. For example, there is risk of unearthing a large volume of literature focused only on determinant identification without subsequent processes to tailor strategies.

Table 1. Eligibility criteria for the review.

Criterion	Inclusion	Exclusion	Justification
Population	All participants who are involved in a tailoring process will be considered, including healthcare professionals, implementation researchers, and people who use healthcare services.	Participants not involved in any stage of the tailoring process	Focus of the research programme is healthcare interventions. Extending beyond healthcare would be unmanageable with review resources.
Concept	Commentaries or editorials/opinion pieces which discuss how tailoring should be done and/or why (objective 1) Primary studies that use a tailoring process to prospectively develop implementation strategies prior to deployment in healthcare or tailoring of a strategy to fit with different population subgroups (objectives 1-3)	Studies adapting implementation strategies to emergent barriers after deployment.	The aim of the review is to inform priorities for a research programme on prospective tailoring to develop strategies to support the adoption and implementation of healthcare interventions.
Context	Studies which use a tailoring process to develop strategies for application in healthcare settings*	Studies using tailoring outside of the healthcare context.	Focus of the research programme is healthcare interventions. Extending the review beyond healthcare would be unmanageable with the resources available for the review.
Language	English	Non-English	Reviewers are only English speaking and do not have the resources to translate articles to other languages.
Time period	Year 2005 – 2022	Before 2005	Given the breadth of the search, and to manage resources, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006.
Types of articles	1. Peer reviewed journal articles of primary studies 2. Unpublished (grey literature) including theses/ dissertations, and book chapters. 3. Study protocols 4. Editorial/opinion pieces will be used as source of additional articles and to address objective 1	• Conference abstracts • Non-peer reviewed sources (i.e., websites, blogs, studies in progress)	Aim to capture more than peer reviewed literature to fully understand how tailoring has been conducted in practice.
Geographic	Any location	None	Tailoring implementation strategies has application globally.

*state-funded and private organisations providing services in the following areas: disability, older persons, nursing homes, acute and non-acute hospitals, community hospitals, mental health, social inclusion, palliative care, chronic illness, primary care (GP, dental, pharmacies, physiotherapy clinics), health and wellbeing, hospice, rehabilitation, home care, paramedics, and community services (e.g. youth, substance abuse, suicide prevention, community development).”²⁸

If studies are identified which operationalise tailoring differently to our broad definition, this will also be documented. For example, studies describing tailoring conducted after deployment of a strategy, adapting delivery to target emerging barriers will be excluded. Studies which describe an approach to tailor an EBI as distinct from a strategy to support implementation of an EBI will be excluded. The difference between the two can be ambiguous²⁹. Further confusion is caused by labels; ‘implementation intervention’ is sometimes used interchangeably with ‘implementation strategy’ . We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guidelines or innovation) being implemented. The focus won’t be on the nature or type intervention to be supported by the tailored strategies, or whether it should be implemented. If we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Studies will only be included if they (a) describe the tailoring approach in some detail; the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

Context

Studies which use a tailoring process to develop strategies for application in healthcare settings will be included. Healthcare settings is defined as “state-funded and private organisations providing services in the following areas: disability, older persons, nursing homes, acute and non-acute hospitals, community hospitals, mental health, social inclusion, palliative care, chronic illness, primary care (GP, dental, pharmacies, physiotherapy clinics), health and wellbeing, hospice, rehabilitation, home care, paramedics, and community services (e.g. youth, substance abuse, suicide prevention, community development).”²⁸

Both qualitative and quantitative primary research studies will be eligible. Similarly, commentaries/editorials/opinion pieces will be used as a source of studies and to address objective 1 on conceptualisation. Unpublished (grey literature) will be included e.g., toolkits, guidance, reports. Conference abstracts will be excluded. Only reports and articles in the English language and between the years 2005 and 2022 will be included. Table 1 outlines the eligibility criteria.

Search strategy

An information scientist (university librarian) will aid the authors to design and refine the search strategy. The search strategy will follow a three step strategy in line with JBI guidance³⁰. First, MEDLINE and Embase will be searched. Text words and index terms contained in the title and abstract of retrieved papers will be compiled to inform subsequent more comprehensive searches. This list will be developed further by considering appropriate thesaurus terms and synonyms. A number of exemplar studies^9,21,31–35 will be used to check the search strategy.

Second, a search using all identified keywords (e.g., ‘tailor’ ‘tailoring’) and index (MeSH) terms will be undertaken across four databases, MEDLINE, Embase, Web of Science Core Collection, and Scopus³⁶. Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. Complete search strategies for each database will be reported as appendices. Third, the reference lists of identified articles and reports will be searched for additional sources and the authors of primary articles or reviews will be contacted for further information if required. For the full MEDLINE search strategy, see Extended data³⁷.

Searching other resources

Handsearching of the journal Implementation Science (2006-2022), BMJ Quality and Safety (2005-2022) and two new journals, Implementation Science Communication (2020, 2021, 2022) and Implementation Research and Practice (2020, 2021, 2022) will also be completed using keywords and Boolean operators. Grey literature will be searched using Google Scholar, using the title-only function, and screening the first 1000 records in accordance with guidance from Haddaway et al.,³⁸ Seminal reviews⁴, commentaries³ and articles (The Tailored Implementation in Chronic Diseases (TICD) study^9,39 and ImpleMentAll) will be included and their reference lists searched. Lastly, implementation process frameworks (e.g., KTA, EPIS) will be identified and included to understand how tailoring has been conceptualised and inform Objective 1.

Articles retrieved from each database will be imported as .ris files into Covidence where duplicates will be removed. Covidence has been shown to overperform reference managers in de-duplication⁴⁰.

Stage 3: study selection

The screening process will be carried out using Covidence. Titles/abstract screening will be split between two or more reviewers. The screening process will be piloted on 10% of retrieved articles and inter-rater reliability calculated for title and abstract screening using the kappa (κ) statistic. Screening will begin once an agreement rate of 75% or greater is reached. Articles identified by either reviewer as meeting the inclusion/exclusion criteria will be included in the full-text review. If it is unclear whether to include an article based on the first stage of the reviewing process, it will be included for full text review. Full text screening will be split between two or more reviewers. Any uncertainties will be discussed and if consensus is not reached then an additional reviewer will be consulted. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

Stage 4: charting the data

Data will be extracted by at least two members of the research team into Microsoft Excel software following guidelines from JBI²⁶. Each member will extract data for a proportion of articles. The data extraction form will be pilot tested by the reviewers on 10% of full text articles to ensure that data is consistently extracted. After pilot testing, any discrepancies that arise will be discussed with the rest of the team, and any refinements to the form will be made, if required.

Data to be extracted will include study characteristics and characteristics of tailoring relating to our research questions. Arksey and O’Malley suggests charting the data according to research themes²⁵. The information to be extracted (Table 2) aligns with the review objectives and is intended to capture key findings that relate to the scoping review question(s).

Table 2. Data charting elements.

Publication details	Associated question(s)	Research question
Study title
Author(s)	What year was the study/document published?	-
Year of publication	What year was the study/document published?	-
Publication type	Is the document an empirical study or grey literature?	-
Origin/country of origin	Which country is the study/document focusing on?	-
General details
Aims/purpose	What are the study/document aims?	-
Study design	What is the study/document design?	-
Study setting/healthcare context	What is the described healthcare setting?	9
Study population	Which is the target population of the tailoring approach?	-
Content
Details of the evidence-based intervention whose implementation is being tailored	What intervention is being implemented? What is the aim of the intervention? What is the health-related focus of the study/document (health condition focused on)? What is the study population?	- - -
Details of the tailoring approach	Is tailoring used prospectively to develop implementation strategy?	8
	Is there a rationale for tailoring? What is the rationale? Is tailoring based on a theory? What is the theory?	2 2 3 3
	Was a theory or framework drawn on during tailoring process? Which framework/theory? At what stage?	7 7 7
	What steps are involved?	6
	When is tailoring carried out (stage of implementation)?	8
	Who is involved: health care professionals, researchers and/or people who used healthcare services? When (what stage(s)) are they involved), for what purpose (what is their role), and how are they involved?	10 10
	Has determinant prioritisation been conducted?	11
	Has strategy selection been conducted?	11
	How has the prioritisation and selection step of tailoring been conducted? What data collection or broader methods have been used? Is there a rationale for the methods used for this step? What is the rationale? What are the inputs during strategy selection – evidence, stakeholders, theory? What is the balance and timing of each input?	11 11a 11a 11a 11b 11c
Outcome(s) of tailoring process and how measured.	Was tailoring evaluated? How is tailoring evaluated? What proximal outcomes of tailoring “success” were used?	12 12 12

Assessment of methodological quality

Critical appraisal of studies included in scoping review is not consistently performed but is increasingly recommended to facilitate the interpretation of the results and support uptake into to policy and practice²⁴. However, as the aim of this review is to describe and understand how tailoring has been conducted, undertaking a quality assessment of included studies is less relevant. The tailoring literature is voluminous and, as indicated by the different definitions, is potentially heterogenous, therefore we considered quality assessment beyond the review scope. Lastly, as flagged by Levac et al.,²⁴, it is unclear how grey literature and the additional evidence from the stakeholder consultation step may be evaluated (Stage 6). A tailoring process may involve several studies and different methods (i.e., surveys, focus groups, interviews) and so it would be challenging to provide an overall assessment of a tailoring process, and such assessment would be unlikely to inform review conclusions regarding the evidence gaps and future research.

Stage 5: collating, summarising, and reporting the results

Levac et al.,²⁴ advise scoping review authors to divide Stage 5 into three distinct steps: analysing the data (collating and summarising), reporting results, and applying meaning to the results.

Step 1: collating and summarising the results (analysis)

Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to research gaps. NVivo will used to manage qualitative analysis where applicable.

Step 2: reporting

Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised within an overarching structure based on the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. The results will be reported with a view to achieving the following goals:

An improved understanding of how tailoring has been defined, and carried out (i.e., what is commonly done and why?)
Enhanced understanding of what information is used to guide determinant or strategy prioritisation
‘Menu’ of tailoring approaches used to date.
Clarity on the core ingredients of tailoring and the balance of these elements.
Highlight gaps in the evidence.

Step 3: Implications

The review findings will help develop an understanding how tailoring is conceptualised, operationalised, and evaluated within the literature. The broader implications of the findings for policy, practice and research will be highlighted.

Stage 6: Consultation with stakeholders

This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, suggest further resources, and identify different strategies for dissemination. Project stakeholders who represent health service organisations will also be consulted. This will help ensure the findings have practical relevance. Further consultation will be considered based on recommendations from the scientific advisory panel. If we proceed with further consultation, an invitation will be issued to stakeholders via co-author networks and through relevant professional societies and organisations. Overall, the consultation process will ask experts, based on the evidence gaps, to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings.

Discussion

Careful use of strategies tailored to address important barriers to, and enablers of, implementation is essential to support the successful adoption and delivery of EBIs. Despite tailoring becoming more common, we lack clarity on definitions, methods, and principles of tailoring. Currently no synthesis of the existing evidence on the tailoring process exists. While also systematic, scoping reviews are distinguished from systematic reviews by having broader research question(s), with the aim of identifying the scope of literature on a subject, gaps in the evidence, rather than that formally assessing the quality of evidence and generating a conclusion in response to a focused research question⁴². This scoping review will describe how tailoring has been undertaken within the healthcare context, identify research gaps, and inform priorities for a research programme on tailoring. This review can be used as a resource by the implementation science community and healthcare practitioners involved in implementation, to guide future research in this field and facilitate systematic, transparent, and replicable development of tailored implementation strategies.

Study status

The initial searches are currently underway at the time of this article publication, and the MEDLINE search strategy has been developed. We intend to disseminate the results through publication in a peer-reviewed journals, conference presentations, and research summaries.

Data availability

Underlying data

All data underlying the results are available as part of the article.

Extended data

Open Science Framework: Characterising processes and outcomes of tailoring implementation strategies in healthcare: A scoping review

https://doi.org/10.17605/OSF.IO/ER9HY³⁷

This project contains the following extended data:

MEDLINE search strategy.pdf (MEDLINE search strategy)
Eligibility criteria.pdf (Study eligibility criteria)
Data charting elements.pdf (Elements to be extracted from included studies)
PRISMA-P 2015 checklist.pdf

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Software availability

Covidence is a proprietary software. An alternative software which can be used for review management is Rayyan.

Acknowledgements

We would like to thank Virginia Conrick, Academic Success Librarian, UCC Library, UCC, Cork, Ireland, for her advice on the development of the search strategy and database searches.

Faculty Opinions recommended

References

1. Proctor EK, Powell BJ, McMillen JC: Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013; 8(1): 139. PubMed Abstract | Publisher Full Text | Free Full Text
2. Powell BJ, Waltz TJ, Chinman MJ, et al.: A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015; 10: 21. PubMed Abstract | Publisher Full Text | Free Full Text
3. Powell BJ, Beidas RS, Lewis CC, et al.: Methods to Improve the Selection and Tailoring of Implementation Strategies. J Behav Heal Serv Res. 2017; 44(2): 177–94. PubMed Abstract | Publisher Full Text | Free Full Text
4. Baker R, Camosso-Stefinovic J, Gillies C, et al.: Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015; 2015(4): CD005470. PubMed Abstract | Publisher Full Text | Free Full Text
5. Wright J: Tailoring Stroke Best Practice Recommendations to the Care Home Context: Identifying a Balance Between Theory and Real-World Practice. PQDT - Glob. 2019. Reference Source
6. Jensen JD, King AJ, Carcioppolo N, et al.: Comparing tailored and narrative worksite interventions at increasing colonoscopy adherence in adults 50-75: a randomized controlled trial. Soc Sci Med. 2014; 104: 31–40. PubMed Abstract | Publisher Full Text
7. Gatwood J, Balkrishnan R, Erickson SR, et al.: The impact of tailored text messages on health beliefs and medication adherence in adults with diabetes: A randomized pilot study. Res Soc Adm Pharm. 2016; 12(1): 130–140. PubMed Abstract | Publisher Full Text
8. Lake AJ, Browne JL, Abraham C, et al.: A tailored intervention to promote uptake of retinal screening among young adults with type 2 diabetes - an intervention mapping approach. BMC Health Serv Res. 2018; 18(1): 396. PubMed Abstract | Publisher Full Text | Free Full Text
9. Bührmann L, Schuurmans J, Ruwaard J, et al.: Tailored implementation of internet-based cognitive behavioural therapy in the multinational context of the ImpleMentAll project: a study protocol for a stepped wedge cluster randomized trial. Trials. 2020; 21(1): 893. PubMed Abstract | Publisher Full Text | Free Full Text
10. Kwok EYL, Moodie STF, Cunningham BJ, et al.: Selecting and tailoring implementation interventions: A concept mapping approach. BMC Health Serv Res. 2020; 20(1): 385. PubMed Abstract | Publisher Full Text | Free Full Text
11. Flottorp SA, Oxman AD, Krause J, et al.: A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013; 8(1): 35. PubMed Abstract | Publisher Full Text | Free Full Text
12. Wensing M, Oxman A, Baker R, et al.: Tailored implementation for chronic diseases (TICD): A project protocol. Implement Sci. 2011; 6(1): 103. PubMed Abstract | Publisher Full Text | Free Full Text
13. Moussa L, Kocbek S: Data-Driven Approach for Tailoring Facilitation Strategies to Overcome Implementation Barriers in Community Pharmacy. 1805; 1–25.
14. Hoffmann TC, Glasziou PP, Boutron I, et al.: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014; 348: g1687. PubMed Abstract | Publisher Full Text
15. Wilson KM, Brady TJ, Lesesne C, et al.: An organizing framework for translation in public health: The knowledge to action framework. Prev Chronic Dis. 2011; 8(2): A46. PubMed Abstract | Free Full Text
16. Shrubsole K, Lin TJ, Burton C, et al.: Delivering an iterative Communication Partner Training programme to multidisciplinary healthcare professionals: A pilot implementation study and process evaluation. Int J Lang Commun Disord. 2021; 56(3): 620–36. PubMed Abstract | Publisher Full Text
17. Reynolds SS, Woltz P, Keating E, et al.: Results of the CHlorhexidine Gluconate Bathing implementation intervention to improve evidence-based nursing practices for prevention of central line associated bloodstream infections Study (CHanGing BathS): a stepped wedge cluster randomized trial. Implement Sci. 2021; 16(1): 45. PubMed Abstract | Publisher Full Text | Free Full Text
18. Miller CJ, Barnett ML, Baumann AA, et al.: The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021; 16(1): 36. PubMed Abstract | Publisher Full Text | Free Full Text
19. Walsh-Bailey C, Palazzo LG, Jones SM, et al.: A pilot study comparing tools for tracking implementation strategies and treatment adaptations. Implement Res Pract. 2021; 2: 263348952110160. Publisher Full Text
20. Colquhoun HL, Squires JE, Kolehmainen N, et al.: Methods for designing interventions to change healthcare professionals' behaviour: A systematic review. Implement Sci. 2017; 12(1): 30. PubMed Abstract | Publisher Full Text | Free Full Text
21. Krause J, Van Lieshout J, Klomp R, et al.: Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implement Sci. 2014; 9(1): 102. PubMed Abstract | Publisher Full Text | Free Full Text
22. Powell BJ, Fernandez ME, Williams NJ, et al.: Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health. 2019; 7(3): 3. PubMed Abstract | Publisher Full Text | Free Full Text
23. Colquhoun HL, Levac D, O’Brien KK, et al.: Scoping reviews: Time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014; 67(12): 1291–4. PubMed Abstract | Publisher Full Text
24. Levac D, Colquhoun H, O’Brien KK: Scoping studies: advancing the methodology. Implement Sci. 2010; 5: 69. PubMed Abstract | Publisher Full Text
25. Arksey H, O’malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1): 19–32. Publisher Full Text
26. Peters MDJ, Marnie C, Tricco AC, et al.: Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020; 18(10): 2119–26. PubMed Abstract | Publisher Full Text
27. Peters MDJ, Godfrey CM, Khalil H, et al.: Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015; 13(3): 141–6. PubMed Abstract | Publisher Full Text
28. Health Protection Surveilliance Centre: Definition of a healthcare worker. Reference Source
29. Eldh AC, Almost J, DeCorby-Watson K, et al.: Clinical interventions, implementation interventions, and the potential greyness in between -a discussion paper. BMC Health Serv Res. 2017; 17(1): 16. PubMed Abstract | Publisher Full Text | Free Full Text
30. Joanna Briggs Institute (JBI): JBI Manual for Evidence Synthesis 4.2.6 Search strategy. 2019. Reference Source
31. Lewis CC, Scott K, Marriott BR: A methodology for generating a tailored implementation blueprint: An exemplar from a youth residential setting. Implement Sci. 2018; 13(1): 68. PubMed Abstract | Publisher Full Text | Free Full Text
32. Powell BJ, Haley AD, Patel SV, et al.: Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring I. Implement Sci Commun. 2020; 1: 9. PubMed Abstract | Publisher Full Text | Free Full Text
33. Huntink E, van Lieshout J, Aakhus E, et al.: Stakeholders' contributions to tailored implementation programs: an observational study of group interview methods. Implement Sci. 2014; 9: 185. PubMed Abstract | Publisher Full Text | Free Full Text
34. Jäger C, Steinhäuser J, Freund T, et al.: Process evaluation of five tailored programs to improve the implementation of evidence-based recommendations for chronic conditions in primary care. Implement Sci. 2016; 11(1): 123. PubMed Abstract | Publisher Full Text | Free Full Text
35. Wensing M, Huntink E, van Lieshout J, et al.: Tailored implementation of evidence-based practice for patients with chronic diseases. PLoS One. 2014; 9(7): e101981. PubMed Abstract | Publisher Full Text | Free Full Text
36. Bramer WM, Rethlefsen ML, Kleijnen J, et al.: Optimal database combinations for literature searches in systematic reviews: A prospective exploratory study. Syst Rev. 2017; 6(1): 245. PubMed Abstract | Publisher Full Text | Free Full Text
37. Riordan F, McHugh S, Lewis CC, et al.: Characterising processes and outcomes of tailoring implementation strategies in healthcare: A scoping review. 2022. http://www.doi.org/10.17605/OSF.IO/ER9HY
38. Haddaway NR, Collins AM, Coughlin D, et al.: The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS One. 2015; 10(9): e0138237. PubMed Abstract | Publisher Full Text | Free Full Text
39. Wensing M: The Tailored Implementation in Chronic Diseases (TICD) project: introduction and main findings. Implement Sci. 2017; 12(1): 5. PubMed Abstract | Publisher Full Text | Free Full Text
40. McKeown S, Mir ZM: Considerations for conducting systematic reviews: evaluating the performance of different methods for de-duplicating references. Syst Rev. 2021; 10(1): 38. PubMed Abstract | Publisher Full Text | Free Full Text
41. Braun V, Clarke V: Thematic analysis a reflexive approach. Reference Source
42. Brien SE, Lorenzetti DL, Lewis S, et al.: Overview of a formal scoping review on health system report cards. Implement Sci. 2010; 5(1): 2. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 03 Mar 2022

Author details Author details

Fiona Riordan
Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Geoffrey M. Curran
Roles: Conceptualization, Methodology, Writing – Review & Editing

Cara C. Lewis
Roles: Conceptualization, Methodology, Writing – Review & Editing

Byron J. Powell
Roles: Conceptualization, Methodology, Writing – Review & Editing

Justin Presseau
Roles: Conceptualization, Methodology, Writing – Review & Editing

Luke Wolfenden
Roles: Conceptualization, Methodology, Writing – Review & Editing

Sheena M. McHugh
Roles: Conceptualization, Methodology, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board [HRB RL-2020-004].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 20 Sep 2022, 5:17

https://doi.org/10.12688/hrbopenres.13507.2

version 1

Published: 03 Mar 2022, 5:17

https://doi.org/10.12688/hrbopenres.13507.1

© 2022 Riordan F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

1834

downloads

144

Citations

SEE MORE DETAILS

CITE

how to cite this article

Riordan F, Curran GM, Lewis CC et al. Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review [version 1; peer review: 2 approved with reservations]. HRB Open Res 2022, 5:17 (https://doi.org/10.12688/hrbopenres.13507.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 03 Mar 2022

Views

Reviewer Report 07 Jun 2022

Craig S. Rosen, Dissemination & Training Division, National Center for PTSD, VA Palo Alto Health Care System, Menlo Park, CA, USA

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14736.r31980

This submission outlines a protocol for a scoping review of tailoring of implementation strategies in health care. The rationale and review of prior literature are clear and compelling. The methods are clearly described and appear appropriate. Some small modifications might ... Continue reading

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.
p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
Table 2 – the listed study question for ‘authors’ appears incorrect.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Implementation science; mental health services reseach; posttraumatic stress

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 21 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

21 Sep 2022

Author Response
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring ... Continue reading
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.

We have amended the text (pg 6):

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is commonly recommended to support the successful delivery of EBIs.

p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
Thank you for this suggested clarification. We have amended the conceptualisation objective (pg 7) and inclusion criteria in Table 1 and pg 10 as below:

Objective 1: Conceptualisation

1. How has tailoring of implementation strategies (i.e. methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) been defined and conceptualised in healthcare contexts?

2. Is there a rationale for tailoring and what is the rationale?

3. Is tailoring based on a theory which explains why it should work?

Context
Studies which use a tailoring process to develop strategies (methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) for application in healthcare settings will be included.

p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
We recognise the reviewers point and have amended this on pg. 8 and in Table 1.

Table 1:
All populations involved in health care research are relevant. This could include healthcare professionals, implementation researchers, and other people who plan, provide and/or use health services
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.

We have amended the text (pg 6):

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is commonly recommended to support the successful delivery of EBIs.

p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
Thank you for this suggested clarification. We have amended the conceptualisation objective (pg 7) and inclusion criteria in Table 1 and pg 10 as below:

Objective 1: Conceptualisation

1. How has tailoring of implementation strategies (i.e. methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) been defined and conceptualised in healthcare contexts?

2. Is there a rationale for tailoring and what is the rationale?

3. Is tailoring based on a theory which explains why it should work?

Context
Studies which use a tailoring process to develop strategies (methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) for application in healthcare settings will be included.

p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
We recognise the reviewers point and have amended this on pg. 8 and in Table 1.

Table 1:
All populations involved in health care research are relevant. This could include healthcare professionals, implementation researchers, and other people who plan, provide and/or use health services
Competing Interests: None Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 21 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

21 Sep 2022

Author Response
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring ... Continue reading
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.

We have amended the text (pg 6):

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is commonly recommended to support the successful delivery of EBIs.

p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
Thank you for this suggested clarification. We have amended the conceptualisation objective (pg 7) and inclusion criteria in Table 1 and pg 10 as below:

Objective 1: Conceptualisation

1. How has tailoring of implementation strategies (i.e. methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) been defined and conceptualised in healthcare contexts?

2. Is there a rationale for tailoring and what is the rationale?

3. Is tailoring based on a theory which explains why it should work?

Context
Studies which use a tailoring process to develop strategies (methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) for application in healthcare settings will be included.

p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
We recognise the reviewers point and have amended this on pg. 8 and in Table 1.

Table 1:
All populations involved in health care research are relevant. This could include healthcare professionals, implementation researchers, and other people who plan, provide and/or use health services
Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.

We have amended the text (pg 6):

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is commonly recommended to support the successful delivery of EBIs.

p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
Thank you for this suggested clarification. We have amended the conceptualisation objective (pg 7) and inclusion criteria in Table 1 and pg 10 as below:

Objective 1: Conceptualisation

1. How has tailoring of implementation strategies (i.e. methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) been defined and conceptualised in healthcare contexts?

2. Is there a rationale for tailoring and what is the rationale?

3. Is tailoring based on a theory which explains why it should work?

Context
Studies which use a tailoring process to develop strategies (methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) for application in healthcare settings will be included.

p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
We recognise the reviewers point and have amended this on pg. 8 and in Table 1.

Table 1:
All populations involved in health care research are relevant. This could include healthcare professionals, implementation researchers, and other people who plan, provide and/or use health services
Competing Interests: None Close
Report a concern

Views

Reviewer Report 23 Mar 2022

Elaine Yuen Ling Kwok, CanChild, Institute of Applied Health Sciences, McMaster University, Hamilton, ON, Canada; Northwestern University, Evanston, IL, USA

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14736.r31612

Thank you for the opportunity to review this study protocol. Overall, I am pleased with the research question and think that an appropriate plan has been developed for the review. I left several comments below which I hope will be ... Continue reading

Yes, the objective & rationale of the study are clearly described and I agree with the importance of conducting this scoping review.

Is the study design appropriate for the research question (including statistical power analysis, where appropriate)?

Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019)¹ or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”? Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?
A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
- Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005. Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?
- Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).
- Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.
- Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.
- Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?
- Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?). Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description. It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

Are sufficient details of the methods provided to allow replication by others?

Yes, see comments above.

Have the authors pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks?

No major concerns here. One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

Are the datasets clearly presented in a useable and accessible format?

N/A.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Partly
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

1. Nilsen P, Bernhardsson S: Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes.BMC Health Serv Res. 2019; 19 (1): 189 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Rehabilitation/Clinical Services Research

CITE

Report a concern

Author Response 22 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

22 Sep 2022

Author Response
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare ... Continue reading
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019) or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”?

Thank you for reviewing our paper and your suggestions, these are greatly appreciated.
We agree there is value in exploring tailoring in other settings beyond healthcare settings. However, we have limited the scope of review to healthcare settings given the resources and time available. Given the ambiguity in terms of what constitutes tailoring and that the types of studies we may identify for this review are quite broad, we needed to introduce some boundaries on other criteria to make it more manageable.

Within the healthcare setting, we will examine the different levels of setting, that is the local level, wider organizational level, and external health system level. This has been added as part of data extraction. We have amended the text in Table 2 to capture this:

What is the described healthcare setting? (Where this information is available, describe the setting in terms of local/team level, organisational level, and external health system level)

Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?

We will describe the prioritisation of determinants in greater detail. We have amended Table 2 to reflect this:

How has the prioritisation and selection step of tailoring been conducted?

What criteria have been used in prioritisation?

Who selected these criteria?

What data collection or broader methods have been used

Is there a rationale for the methods used for this step?

What is the rationale?

What are the inputs during strategy selection – evidence, stakeholders, theory?

What is the balance and timing of each input?

A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005.

We have expanded on our justification in Table 1:

Given the breadth of the search and resources available, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006, signalling the formal establishment of the field in 2006 and subsequently, dedicated journals, including Implementation Science Communications (2020), Implementation Research and Practice (2020), and Global Implementation Research and Applications (2021) have been established.

We have also added the following to the main text (’Context’) (pg 11):

A preliminary electronic search for key terms (“tailor*” and terms for strategy or intervention and implementation and healthcare) in title and abstracts along with relevant subject headings in MEDLINE (14) indicates that in the last 20 years, there has been an upward trend in articles referring to tailoring, with 1722 articles published 2012-2021 compared to 443 articles in the 10 years (2002-2011) previous. In total there were 152 articles published 1972 – 2005 and 2148 from 2006 to 2022. Hence our decision regarding search date limitations.

To note, we also include additional clarification on why we completed dedicated searches for a specific tailoring approach, ‘implementation mapping’ (pg.11):

Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. This specific approach was chosen as it is unique to tailoring implementation strategies. Other methods (intervention mapping concept mapping, conjoint analysis, and group model building) while suggested as possible tailoring approaches, can also be used in other fields and for other purposes. This lack of specificity was confirmed during preliminary searches conducted to inform search development.

We also wished to take this opportunity to be more explicit in terms of our eligibility, namely that studies which use ‘implementation mapping’ term will be included as this is taken to be an approach to tailoring (pg. 8):

While there is variation in the scope of tailoring (i.e., whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing are included), strategy selection is a core aspect of tailoring. Authors must explicitly describe the process as tailoring or the strategy as tailored. Studies will also be included if they refer to the intervention/strategy development process as ‘implementation mapping’. This term was adapted from ‘intervention mapping’ and first used by Fernandez et al to specifically describe an approach to tailor implementation strategies.

We also provided further clarification on dealing with ambiguity between evidence-based interventions and implementation strategies (pg. 11):

We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice” ¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guideline, policy or innovation) being implemented. The focus will not be on the nature or type of intervention being implemented, or whether it should be implemented, rather the focus will be on the tailored strategy supporting implementation. Where the delineation between intervention and strategy is unclear, we will refer to the ERIC taxonomy to identify implementation strategies. At the full text screening stage, if we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?

We agree. This is something we have discussed when piloting our screening approach.

We expect that the search strategy will capture multiple articles from the same study. As suggested, some may have separate articles which cover the evaluation of a tailored intervention or strategy. During title and abstract screening, we will be inclusive. We will include cost-effectiveness studies and evaluations of tailored strategies. During full text screening these ‘results’ papers will be checked for reference to the tailoring phase. We will exclude them at full text screening stage if they do not provide further details on the tailoring process (in line with our eligibility criteria) or refer to a related publication which describes this process. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Under our description of study selection (pg. 12), we include some additional details on this point:

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).

Grey literature will be searched, albeit in a limited way, using Google Scholar, using the title-only function, and screening the first 1000 records. We will include some unpublished (grey literature) including theses/dissertations, book chapters, toolkits, guidance, and reports but exclude conference abstracts and other non-peer reviewed sources (i.e., presentations, websites, blogs, studies in progress) (pg. 9 and Table 1). We have amended Table 1 to clarify the nature of the grey literature.

Originally, we limited the grey literature search due to resource constraints and our uncertainty as to whether it would be worth the yield, for example, searching organisational websites to identify reports. We suggest that future work could look more extensively at the application of tailoring in non-academic literature.

Please note that we have also amended the related question in Table 2 Data charting elements (pg 13):
Is the document a peer-reviewed, published article or grey literature?

Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.

We have expanded our review team and piloted our screening approach to ensure we consistently apply the definition and inclusion criteria, holding a number of alignment meetings to discuss uncertainties. We limited the pilot to 250 articles, so that we could dedicate more time to discussion and reflect on any discrepancies. We held a number of meetings during the piloting process to discuss the application of eligibility criteria. As it is difficult to apply the concept of tailoring, we have agreed to be inclusive at the title and abstract screening stage, recognising that this will lead to a larger number of full texts to screen. We have also decided to apply double screening. All reviewers have an opportunity to practice applying the screening questions.

We have amended the text (pg 12) as follows:

The screening process will be carried out using Covidence. Titles/abstract and full text screening will be split between two or more reviewers. The title/abstract screening process will be piloted initially – 250 titles and abstracts will be screened by the review team using the predefined eligibility criteria. If any discrepancies are identified, these will be discussed within the team. Inter-rater reliability will be calculated for title and abstract screening using the kappa (κ) statistic to indicate alignment during the piloting stage. The screening process will involve double-screening. Two reviewers will screen each title and abstract, with conflicts resolved by consensus or a third reviewer. Full texts will also be screened in duplicate with conflicts resolved by consensus or a third reviewer. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.

We have reflected on data extraction in study context above and made amendments to capture different levels within the healthcare setting.

Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?

Our process to identify and highlight evidence gaps will involve:

Thematic analysis

Visual depiction (where appropriate)

Stakeholder consultation

As outlined in the protocol, evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies, and where there is uncertainty about the way to approach an aspect of tailoring. They will be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation).

In terms of visualisation, there is little consensus on the exact definition of what constitutes an evidence gap map but broadly speaking it is considered a systematic search to identify gaps in knowledge and/or future research needs, presenting these in a user-friendly format e.g., a visual figure or graph, or a searchable database (Miake-Lye et al., 2016).

Miake-Lye, I.M., Hempel, S., Shanman, R. et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 5, 28 (2016).

We have added some text to elaborate on the approach in our ‘Analysis’ section (pg 15):

Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to evidence gaps. Evidence gaps will broadly be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation). For example, under the research question ‘How was the prioritisation of determinants and selection of strategies conducted?’ a theme could be ‘variable criteria used for prioritisation’. We would propose the gaps are potentially (1) lack of evidence on which criteria to use to prioritise determinants and (2) lack of evidence on the impact of the criteria chosen. Proposed evidence gaps will be reviewed as part of the stakeholder consultation (Stage 6). Where appropriate we will develop visual depictions (e.g., geographic or topical visualisation) to support thematic summaries of the evidence gaps.

We have amended the text under ‘Stage 6: consultation with stakeholders’ (pg 16):

Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation.

Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?).
Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description.

In the first instance stakeholders will involve co-authors (an international group of experts in the implementation science field) and project stakeholders. They will represent academic and healthcare settings. They may not all have direct experience of tailoring but will have experience in and be familiar with implementation of evidence-based practices (science and practice). Further stakeholders to consult may be suggested by this initial group (pg 16).

The purpose of the stakeholder consultation is to advise on the content and priorities of a tailoring research agenda and on the dissemination of review findings. This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted, along with project stakeholders who represent health service organisations, during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, and identify strategies for disseminating review findings. This will help ensure the findings have practical relevance.

We will not ask stakeholders to suggest additional resources as it is potentially extending or re-opening the review and given the search is systematic, we expect these resources should be picked up.

It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

The purpose of the stakeholder consultation is to inform a research agenda for the CUSTOMISE project and advise on dissemination plans for the scoping review results. We have made this more explicit in the objectives (pg 6) and the stakeholder engagement section (pg 16).

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring and inform a research agenda focused on tailored approaches to implementation (Objective 5).

One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

We will capture and synthesize the level and completeness of information available on various aspects of tailoring as part of the results of this review. Studies will be included even if they provide minimal data as described below (pg 10):

Studies will only be included if they (a) describe the tailoring approach in some detail, that is, the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

In cases where data are lacking, we will report this in the results section and use this to inform our findings on research gaps. For example, our findings may point towards the need for systematic reporting of certain aspects of tailoring process. If there is insufficient data for inductive thematic analysis, the deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’. We have amended text to reflect how minimal data will be analysed (pg. 15) will reported (pg 16):

Pg. 15:
Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke ⁴¹, will be conducted to develop themes related to evidence gaps. If there is insufficient data for inductive thematic analysis, deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’.

Pg. 16:
Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised according to the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. We will record and summarise where there is insufficient data on aspects of tailoring to inform our findings on the tailoring evidence gap
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019) or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”?

Thank you for reviewing our paper and your suggestions, these are greatly appreciated.
We agree there is value in exploring tailoring in other settings beyond healthcare settings. However, we have limited the scope of review to healthcare settings given the resources and time available. Given the ambiguity in terms of what constitutes tailoring and that the types of studies we may identify for this review are quite broad, we needed to introduce some boundaries on other criteria to make it more manageable.

Within the healthcare setting, we will examine the different levels of setting, that is the local level, wider organizational level, and external health system level. This has been added as part of data extraction. We have amended the text in Table 2 to capture this:

What is the described healthcare setting? (Where this information is available, describe the setting in terms of local/team level, organisational level, and external health system level)

Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?

We will describe the prioritisation of determinants in greater detail. We have amended Table 2 to reflect this:

How has the prioritisation and selection step of tailoring been conducted?

What criteria have been used in prioritisation?

Who selected these criteria?

What data collection or broader methods have been used

Is there a rationale for the methods used for this step?

What is the rationale?

What are the inputs during strategy selection – evidence, stakeholders, theory?

What is the balance and timing of each input?

A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005.

We have expanded on our justification in Table 1:

Given the breadth of the search and resources available, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006, signalling the formal establishment of the field in 2006 and subsequently, dedicated journals, including Implementation Science Communications (2020), Implementation Research and Practice (2020), and Global Implementation Research and Applications (2021) have been established.

We have also added the following to the main text (’Context’) (pg 11):

A preliminary electronic search for key terms (“tailor*” and terms for strategy or intervention and implementation and healthcare) in title and abstracts along with relevant subject headings in MEDLINE (14) indicates that in the last 20 years, there has been an upward trend in articles referring to tailoring, with 1722 articles published 2012-2021 compared to 443 articles in the 10 years (2002-2011) previous. In total there were 152 articles published 1972 – 2005 and 2148 from 2006 to 2022. Hence our decision regarding search date limitations.

To note, we also include additional clarification on why we completed dedicated searches for a specific tailoring approach, ‘implementation mapping’ (pg.11):

Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. This specific approach was chosen as it is unique to tailoring implementation strategies. Other methods (intervention mapping concept mapping, conjoint analysis, and group model building) while suggested as possible tailoring approaches, can also be used in other fields and for other purposes. This lack of specificity was confirmed during preliminary searches conducted to inform search development.

We also wished to take this opportunity to be more explicit in terms of our eligibility, namely that studies which use ‘implementation mapping’ term will be included as this is taken to be an approach to tailoring (pg. 8):

While there is variation in the scope of tailoring (i.e., whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing are included), strategy selection is a core aspect of tailoring. Authors must explicitly describe the process as tailoring or the strategy as tailored. Studies will also be included if they refer to the intervention/strategy development process as ‘implementation mapping’. This term was adapted from ‘intervention mapping’ and first used by Fernandez et al to specifically describe an approach to tailor implementation strategies.

We also provided further clarification on dealing with ambiguity between evidence-based interventions and implementation strategies (pg. 11):

We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice” ¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guideline, policy or innovation) being implemented. The focus will not be on the nature or type of intervention being implemented, or whether it should be implemented, rather the focus will be on the tailored strategy supporting implementation. Where the delineation between intervention and strategy is unclear, we will refer to the ERIC taxonomy to identify implementation strategies. At the full text screening stage, if we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?

We agree. This is something we have discussed when piloting our screening approach.

We expect that the search strategy will capture multiple articles from the same study. As suggested, some may have separate articles which cover the evaluation of a tailored intervention or strategy. During title and abstract screening, we will be inclusive. We will include cost-effectiveness studies and evaluations of tailored strategies. During full text screening these ‘results’ papers will be checked for reference to the tailoring phase. We will exclude them at full text screening stage if they do not provide further details on the tailoring process (in line with our eligibility criteria) or refer to a related publication which describes this process. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Under our description of study selection (pg. 12), we include some additional details on this point:

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).

Grey literature will be searched, albeit in a limited way, using Google Scholar, using the title-only function, and screening the first 1000 records. We will include some unpublished (grey literature) including theses/dissertations, book chapters, toolkits, guidance, and reports but exclude conference abstracts and other non-peer reviewed sources (i.e., presentations, websites, blogs, studies in progress) (pg. 9 and Table 1). We have amended Table 1 to clarify the nature of the grey literature.

Originally, we limited the grey literature search due to resource constraints and our uncertainty as to whether it would be worth the yield, for example, searching organisational websites to identify reports. We suggest that future work could look more extensively at the application of tailoring in non-academic literature.

Please note that we have also amended the related question in Table 2 Data charting elements (pg 13):
Is the document a peer-reviewed, published article or grey literature?

Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.

We have expanded our review team and piloted our screening approach to ensure we consistently apply the definition and inclusion criteria, holding a number of alignment meetings to discuss uncertainties. We limited the pilot to 250 articles, so that we could dedicate more time to discussion and reflect on any discrepancies. We held a number of meetings during the piloting process to discuss the application of eligibility criteria. As it is difficult to apply the concept of tailoring, we have agreed to be inclusive at the title and abstract screening stage, recognising that this will lead to a larger number of full texts to screen. We have also decided to apply double screening. All reviewers have an opportunity to practice applying the screening questions.

We have amended the text (pg 12) as follows:

The screening process will be carried out using Covidence. Titles/abstract and full text screening will be split between two or more reviewers. The title/abstract screening process will be piloted initially – 250 titles and abstracts will be screened by the review team using the predefined eligibility criteria. If any discrepancies are identified, these will be discussed within the team. Inter-rater reliability will be calculated for title and abstract screening using the kappa (κ) statistic to indicate alignment during the piloting stage. The screening process will involve double-screening. Two reviewers will screen each title and abstract, with conflicts resolved by consensus or a third reviewer. Full texts will also be screened in duplicate with conflicts resolved by consensus or a third reviewer. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.

We have reflected on data extraction in study context above and made amendments to capture different levels within the healthcare setting.

Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?

Our process to identify and highlight evidence gaps will involve:

Thematic analysis

Visual depiction (where appropriate)

Stakeholder consultation

As outlined in the protocol, evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies, and where there is uncertainty about the way to approach an aspect of tailoring. They will be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation).

In terms of visualisation, there is little consensus on the exact definition of what constitutes an evidence gap map but broadly speaking it is considered a systematic search to identify gaps in knowledge and/or future research needs, presenting these in a user-friendly format e.g., a visual figure or graph, or a searchable database (Miake-Lye et al., 2016).

Miake-Lye, I.M., Hempel, S., Shanman, R. et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 5, 28 (2016).

We have added some text to elaborate on the approach in our ‘Analysis’ section (pg 15):

Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to evidence gaps. Evidence gaps will broadly be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation). For example, under the research question ‘How was the prioritisation of determinants and selection of strategies conducted?’ a theme could be ‘variable criteria used for prioritisation’. We would propose the gaps are potentially (1) lack of evidence on which criteria to use to prioritise determinants and (2) lack of evidence on the impact of the criteria chosen. Proposed evidence gaps will be reviewed as part of the stakeholder consultation (Stage 6). Where appropriate we will develop visual depictions (e.g., geographic or topical visualisation) to support thematic summaries of the evidence gaps.

We have amended the text under ‘Stage 6: consultation with stakeholders’ (pg 16):

Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation.

Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?).
Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description.

In the first instance stakeholders will involve co-authors (an international group of experts in the implementation science field) and project stakeholders. They will represent academic and healthcare settings. They may not all have direct experience of tailoring but will have experience in and be familiar with implementation of evidence-based practices (science and practice). Further stakeholders to consult may be suggested by this initial group (pg 16).

The purpose of the stakeholder consultation is to advise on the content and priorities of a tailoring research agenda and on the dissemination of review findings. This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted, along with project stakeholders who represent health service organisations, during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, and identify strategies for disseminating review findings. This will help ensure the findings have practical relevance.

We will not ask stakeholders to suggest additional resources as it is potentially extending or re-opening the review and given the search is systematic, we expect these resources should be picked up.

It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

The purpose of the stakeholder consultation is to inform a research agenda for the CUSTOMISE project and advise on dissemination plans for the scoping review results. We have made this more explicit in the objectives (pg 6) and the stakeholder engagement section (pg 16).

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring and inform a research agenda focused on tailored approaches to implementation (Objective 5).

One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

We will capture and synthesize the level and completeness of information available on various aspects of tailoring as part of the results of this review. Studies will be included even if they provide minimal data as described below (pg 10):

Studies will only be included if they (a) describe the tailoring approach in some detail, that is, the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

In cases where data are lacking, we will report this in the results section and use this to inform our findings on research gaps. For example, our findings may point towards the need for systematic reporting of certain aspects of tailoring process. If there is insufficient data for inductive thematic analysis, the deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’. We have amended text to reflect how minimal data will be analysed (pg. 15) will reported (pg 16):

Pg. 15:
Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke ⁴¹, will be conducted to develop themes related to evidence gaps. If there is insufficient data for inductive thematic analysis, deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’.

Pg. 16:
Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised according to the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. We will record and summarise where there is insufficient data on aspects of tailoring to inform our findings on the tailoring evidence gap
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

22 Sep 2022

Author Response
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare ... Continue reading
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019) or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”?

Thank you for reviewing our paper and your suggestions, these are greatly appreciated.
We agree there is value in exploring tailoring in other settings beyond healthcare settings. However, we have limited the scope of review to healthcare settings given the resources and time available. Given the ambiguity in terms of what constitutes tailoring and that the types of studies we may identify for this review are quite broad, we needed to introduce some boundaries on other criteria to make it more manageable.

Within the healthcare setting, we will examine the different levels of setting, that is the local level, wider organizational level, and external health system level. This has been added as part of data extraction. We have amended the text in Table 2 to capture this:

What is the described healthcare setting? (Where this information is available, describe the setting in terms of local/team level, organisational level, and external health system level)

Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?

We will describe the prioritisation of determinants in greater detail. We have amended Table 2 to reflect this:

How has the prioritisation and selection step of tailoring been conducted?

What criteria have been used in prioritisation?

Who selected these criteria?

What data collection or broader methods have been used

Is there a rationale for the methods used for this step?

What is the rationale?

What are the inputs during strategy selection – evidence, stakeholders, theory?

What is the balance and timing of each input?

A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005.

We have expanded on our justification in Table 1:

Given the breadth of the search and resources available, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006, signalling the formal establishment of the field in 2006 and subsequently, dedicated journals, including Implementation Science Communications (2020), Implementation Research and Practice (2020), and Global Implementation Research and Applications (2021) have been established.

We have also added the following to the main text (’Context’) (pg 11):

A preliminary electronic search for key terms (“tailor*” and terms for strategy or intervention and implementation and healthcare) in title and abstracts along with relevant subject headings in MEDLINE (14) indicates that in the last 20 years, there has been an upward trend in articles referring to tailoring, with 1722 articles published 2012-2021 compared to 443 articles in the 10 years (2002-2011) previous. In total there were 152 articles published 1972 – 2005 and 2148 from 2006 to 2022. Hence our decision regarding search date limitations.

To note, we also include additional clarification on why we completed dedicated searches for a specific tailoring approach, ‘implementation mapping’ (pg.11):

Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. This specific approach was chosen as it is unique to tailoring implementation strategies. Other methods (intervention mapping concept mapping, conjoint analysis, and group model building) while suggested as possible tailoring approaches, can also be used in other fields and for other purposes. This lack of specificity was confirmed during preliminary searches conducted to inform search development.

We also wished to take this opportunity to be more explicit in terms of our eligibility, namely that studies which use ‘implementation mapping’ term will be included as this is taken to be an approach to tailoring (pg. 8):

While there is variation in the scope of tailoring (i.e., whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing are included), strategy selection is a core aspect of tailoring. Authors must explicitly describe the process as tailoring or the strategy as tailored. Studies will also be included if they refer to the intervention/strategy development process as ‘implementation mapping’. This term was adapted from ‘intervention mapping’ and first used by Fernandez et al to specifically describe an approach to tailor implementation strategies.

We also provided further clarification on dealing with ambiguity between evidence-based interventions and implementation strategies (pg. 11):

We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice” ¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guideline, policy or innovation) being implemented. The focus will not be on the nature or type of intervention being implemented, or whether it should be implemented, rather the focus will be on the tailored strategy supporting implementation. Where the delineation between intervention and strategy is unclear, we will refer to the ERIC taxonomy to identify implementation strategies. At the full text screening stage, if we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?

We agree. This is something we have discussed when piloting our screening approach.

We expect that the search strategy will capture multiple articles from the same study. As suggested, some may have separate articles which cover the evaluation of a tailored intervention or strategy. During title and abstract screening, we will be inclusive. We will include cost-effectiveness studies and evaluations of tailored strategies. During full text screening these ‘results’ papers will be checked for reference to the tailoring phase. We will exclude them at full text screening stage if they do not provide further details on the tailoring process (in line with our eligibility criteria) or refer to a related publication which describes this process. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Under our description of study selection (pg. 12), we include some additional details on this point:

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).

Grey literature will be searched, albeit in a limited way, using Google Scholar, using the title-only function, and screening the first 1000 records. We will include some unpublished (grey literature) including theses/dissertations, book chapters, toolkits, guidance, and reports but exclude conference abstracts and other non-peer reviewed sources (i.e., presentations, websites, blogs, studies in progress) (pg. 9 and Table 1). We have amended Table 1 to clarify the nature of the grey literature.

Originally, we limited the grey literature search due to resource constraints and our uncertainty as to whether it would be worth the yield, for example, searching organisational websites to identify reports. We suggest that future work could look more extensively at the application of tailoring in non-academic literature.

Please note that we have also amended the related question in Table 2 Data charting elements (pg 13):
Is the document a peer-reviewed, published article or grey literature?

Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.

We have expanded our review team and piloted our screening approach to ensure we consistently apply the definition and inclusion criteria, holding a number of alignment meetings to discuss uncertainties. We limited the pilot to 250 articles, so that we could dedicate more time to discussion and reflect on any discrepancies. We held a number of meetings during the piloting process to discuss the application of eligibility criteria. As it is difficult to apply the concept of tailoring, we have agreed to be inclusive at the title and abstract screening stage, recognising that this will lead to a larger number of full texts to screen. We have also decided to apply double screening. All reviewers have an opportunity to practice applying the screening questions.

We have amended the text (pg 12) as follows:

The screening process will be carried out using Covidence. Titles/abstract and full text screening will be split between two or more reviewers. The title/abstract screening process will be piloted initially – 250 titles and abstracts will be screened by the review team using the predefined eligibility criteria. If any discrepancies are identified, these will be discussed within the team. Inter-rater reliability will be calculated for title and abstract screening using the kappa (κ) statistic to indicate alignment during the piloting stage. The screening process will involve double-screening. Two reviewers will screen each title and abstract, with conflicts resolved by consensus or a third reviewer. Full texts will also be screened in duplicate with conflicts resolved by consensus or a third reviewer. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.

We have reflected on data extraction in study context above and made amendments to capture different levels within the healthcare setting.

Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?

Our process to identify and highlight evidence gaps will involve:

Thematic analysis

Visual depiction (where appropriate)

Stakeholder consultation

As outlined in the protocol, evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies, and where there is uncertainty about the way to approach an aspect of tailoring. They will be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation).

In terms of visualisation, there is little consensus on the exact definition of what constitutes an evidence gap map but broadly speaking it is considered a systematic search to identify gaps in knowledge and/or future research needs, presenting these in a user-friendly format e.g., a visual figure or graph, or a searchable database (Miake-Lye et al., 2016).

Miake-Lye, I.M., Hempel, S., Shanman, R. et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 5, 28 (2016).

We have added some text to elaborate on the approach in our ‘Analysis’ section (pg 15):

Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to evidence gaps. Evidence gaps will broadly be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation). For example, under the research question ‘How was the prioritisation of determinants and selection of strategies conducted?’ a theme could be ‘variable criteria used for prioritisation’. We would propose the gaps are potentially (1) lack of evidence on which criteria to use to prioritise determinants and (2) lack of evidence on the impact of the criteria chosen. Proposed evidence gaps will be reviewed as part of the stakeholder consultation (Stage 6). Where appropriate we will develop visual depictions (e.g., geographic or topical visualisation) to support thematic summaries of the evidence gaps.

We have amended the text under ‘Stage 6: consultation with stakeholders’ (pg 16):

Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation.

Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?).
Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description.

In the first instance stakeholders will involve co-authors (an international group of experts in the implementation science field) and project stakeholders. They will represent academic and healthcare settings. They may not all have direct experience of tailoring but will have experience in and be familiar with implementation of evidence-based practices (science and practice). Further stakeholders to consult may be suggested by this initial group (pg 16).

The purpose of the stakeholder consultation is to advise on the content and priorities of a tailoring research agenda and on the dissemination of review findings. This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted, along with project stakeholders who represent health service organisations, during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, and identify strategies for disseminating review findings. This will help ensure the findings have practical relevance.

We will not ask stakeholders to suggest additional resources as it is potentially extending or re-opening the review and given the search is systematic, we expect these resources should be picked up.

It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

The purpose of the stakeholder consultation is to inform a research agenda for the CUSTOMISE project and advise on dissemination plans for the scoping review results. We have made this more explicit in the objectives (pg 6) and the stakeholder engagement section (pg 16).

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring and inform a research agenda focused on tailored approaches to implementation (Objective 5).

One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

We will capture and synthesize the level and completeness of information available on various aspects of tailoring as part of the results of this review. Studies will be included even if they provide minimal data as described below (pg 10):

Studies will only be included if they (a) describe the tailoring approach in some detail, that is, the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

In cases where data are lacking, we will report this in the results section and use this to inform our findings on research gaps. For example, our findings may point towards the need for systematic reporting of certain aspects of tailoring process. If there is insufficient data for inductive thematic analysis, the deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’. We have amended text to reflect how minimal data will be analysed (pg. 15) will reported (pg 16):

Pg. 15:
Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke ⁴¹, will be conducted to develop themes related to evidence gaps. If there is insufficient data for inductive thematic analysis, deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’.

Pg. 16:
Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised according to the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. We will record and summarise where there is insufficient data on aspects of tailoring to inform our findings on the tailoring evidence gap
Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019) or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”?

Thank you for reviewing our paper and your suggestions, these are greatly appreciated.
We agree there is value in exploring tailoring in other settings beyond healthcare settings. However, we have limited the scope of review to healthcare settings given the resources and time available. Given the ambiguity in terms of what constitutes tailoring and that the types of studies we may identify for this review are quite broad, we needed to introduce some boundaries on other criteria to make it more manageable.

Within the healthcare setting, we will examine the different levels of setting, that is the local level, wider organizational level, and external health system level. This has been added as part of data extraction. We have amended the text in Table 2 to capture this:

What is the described healthcare setting? (Where this information is available, describe the setting in terms of local/team level, organisational level, and external health system level)

Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?

We will describe the prioritisation of determinants in greater detail. We have amended Table 2 to reflect this:

How has the prioritisation and selection step of tailoring been conducted?

What criteria have been used in prioritisation?

Who selected these criteria?

What data collection or broader methods have been used

Is there a rationale for the methods used for this step?

What is the rationale?

What are the inputs during strategy selection – evidence, stakeholders, theory?

What is the balance and timing of each input?

A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005.

We have expanded on our justification in Table 1:

Given the breadth of the search and resources available, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006, signalling the formal establishment of the field in 2006 and subsequently, dedicated journals, including Implementation Science Communications (2020), Implementation Research and Practice (2020), and Global Implementation Research and Applications (2021) have been established.

We have also added the following to the main text (’Context’) (pg 11):

A preliminary electronic search for key terms (“tailor*” and terms for strategy or intervention and implementation and healthcare) in title and abstracts along with relevant subject headings in MEDLINE (14) indicates that in the last 20 years, there has been an upward trend in articles referring to tailoring, with 1722 articles published 2012-2021 compared to 443 articles in the 10 years (2002-2011) previous. In total there were 152 articles published 1972 – 2005 and 2148 from 2006 to 2022. Hence our decision regarding search date limitations.

To note, we also include additional clarification on why we completed dedicated searches for a specific tailoring approach, ‘implementation mapping’ (pg.11):

Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. This specific approach was chosen as it is unique to tailoring implementation strategies. Other methods (intervention mapping concept mapping, conjoint analysis, and group model building) while suggested as possible tailoring approaches, can also be used in other fields and for other purposes. This lack of specificity was confirmed during preliminary searches conducted to inform search development.

We also wished to take this opportunity to be more explicit in terms of our eligibility, namely that studies which use ‘implementation mapping’ term will be included as this is taken to be an approach to tailoring (pg. 8):

While there is variation in the scope of tailoring (i.e., whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing are included), strategy selection is a core aspect of tailoring. Authors must explicitly describe the process as tailoring or the strategy as tailored. Studies will also be included if they refer to the intervention/strategy development process as ‘implementation mapping’. This term was adapted from ‘intervention mapping’ and first used by Fernandez et al to specifically describe an approach to tailor implementation strategies.

We also provided further clarification on dealing with ambiguity between evidence-based interventions and implementation strategies (pg. 11):

We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice” ¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guideline, policy or innovation) being implemented. The focus will not be on the nature or type of intervention being implemented, or whether it should be implemented, rather the focus will be on the tailored strategy supporting implementation. Where the delineation between intervention and strategy is unclear, we will refer to the ERIC taxonomy to identify implementation strategies. At the full text screening stage, if we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?

We agree. This is something we have discussed when piloting our screening approach.

We expect that the search strategy will capture multiple articles from the same study. As suggested, some may have separate articles which cover the evaluation of a tailored intervention or strategy. During title and abstract screening, we will be inclusive. We will include cost-effectiveness studies and evaluations of tailored strategies. During full text screening these ‘results’ papers will be checked for reference to the tailoring phase. We will exclude them at full text screening stage if they do not provide further details on the tailoring process (in line with our eligibility criteria) or refer to a related publication which describes this process. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Under our description of study selection (pg. 12), we include some additional details on this point:

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).

Grey literature will be searched, albeit in a limited way, using Google Scholar, using the title-only function, and screening the first 1000 records. We will include some unpublished (grey literature) including theses/dissertations, book chapters, toolkits, guidance, and reports but exclude conference abstracts and other non-peer reviewed sources (i.e., presentations, websites, blogs, studies in progress) (pg. 9 and Table 1). We have amended Table 1 to clarify the nature of the grey literature.

Originally, we limited the grey literature search due to resource constraints and our uncertainty as to whether it would be worth the yield, for example, searching organisational websites to identify reports. We suggest that future work could look more extensively at the application of tailoring in non-academic literature.

Please note that we have also amended the related question in Table 2 Data charting elements (pg 13):
Is the document a peer-reviewed, published article or grey literature?

Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.

We have expanded our review team and piloted our screening approach to ensure we consistently apply the definition and inclusion criteria, holding a number of alignment meetings to discuss uncertainties. We limited the pilot to 250 articles, so that we could dedicate more time to discussion and reflect on any discrepancies. We held a number of meetings during the piloting process to discuss the application of eligibility criteria. As it is difficult to apply the concept of tailoring, we have agreed to be inclusive at the title and abstract screening stage, recognising that this will lead to a larger number of full texts to screen. We have also decided to apply double screening. All reviewers have an opportunity to practice applying the screening questions.

We have amended the text (pg 12) as follows:

The screening process will be carried out using Covidence. Titles/abstract and full text screening will be split between two or more reviewers. The title/abstract screening process will be piloted initially – 250 titles and abstracts will be screened by the review team using the predefined eligibility criteria. If any discrepancies are identified, these will be discussed within the team. Inter-rater reliability will be calculated for title and abstract screening using the kappa (κ) statistic to indicate alignment during the piloting stage. The screening process will involve double-screening. Two reviewers will screen each title and abstract, with conflicts resolved by consensus or a third reviewer. Full texts will also be screened in duplicate with conflicts resolved by consensus or a third reviewer. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.

We have reflected on data extraction in study context above and made amendments to capture different levels within the healthcare setting.

Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?

Our process to identify and highlight evidence gaps will involve:

Thematic analysis

Visual depiction (where appropriate)

Stakeholder consultation

As outlined in the protocol, evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies, and where there is uncertainty about the way to approach an aspect of tailoring. They will be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation).

In terms of visualisation, there is little consensus on the exact definition of what constitutes an evidence gap map but broadly speaking it is considered a systematic search to identify gaps in knowledge and/or future research needs, presenting these in a user-friendly format e.g., a visual figure or graph, or a searchable database (Miake-Lye et al., 2016).

Miake-Lye, I.M., Hempel, S., Shanman, R. et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 5, 28 (2016).

We have added some text to elaborate on the approach in our ‘Analysis’ section (pg 15):

Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to evidence gaps. Evidence gaps will broadly be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation). For example, under the research question ‘How was the prioritisation of determinants and selection of strategies conducted?’ a theme could be ‘variable criteria used for prioritisation’. We would propose the gaps are potentially (1) lack of evidence on which criteria to use to prioritise determinants and (2) lack of evidence on the impact of the criteria chosen. Proposed evidence gaps will be reviewed as part of the stakeholder consultation (Stage 6). Where appropriate we will develop visual depictions (e.g., geographic or topical visualisation) to support thematic summaries of the evidence gaps.

We have amended the text under ‘Stage 6: consultation with stakeholders’ (pg 16):

Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation.

Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?).
Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description.

In the first instance stakeholders will involve co-authors (an international group of experts in the implementation science field) and project stakeholders. They will represent academic and healthcare settings. They may not all have direct experience of tailoring but will have experience in and be familiar with implementation of evidence-based practices (science and practice). Further stakeholders to consult may be suggested by this initial group (pg 16).

The purpose of the stakeholder consultation is to advise on the content and priorities of a tailoring research agenda and on the dissemination of review findings. This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted, along with project stakeholders who represent health service organisations, during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, and identify strategies for disseminating review findings. This will help ensure the findings have practical relevance.

We will not ask stakeholders to suggest additional resources as it is potentially extending or re-opening the review and given the search is systematic, we expect these resources should be picked up.

It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

The purpose of the stakeholder consultation is to inform a research agenda for the CUSTOMISE project and advise on dissemination plans for the scoping review results. We have made this more explicit in the objectives (pg 6) and the stakeholder engagement section (pg 16).

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring and inform a research agenda focused on tailored approaches to implementation (Objective 5).

One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

We will capture and synthesize the level and completeness of information available on various aspects of tailoring as part of the results of this review. Studies will be included even if they provide minimal data as described below (pg 10):

Studies will only be included if they (a) describe the tailoring approach in some detail, that is, the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

In cases where data are lacking, we will report this in the results section and use this to inform our findings on research gaps. For example, our findings may point towards the need for systematic reporting of certain aspects of tailoring process. If there is insufficient data for inductive thematic analysis, the deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’. We have amended text to reflect how minimal data will be analysed (pg. 15) will reported (pg 16):

Pg. 15:
Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke ⁴¹, will be conducted to develop themes related to evidence gaps. If there is insufficient data for inductive thematic analysis, deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’.

Pg. 16:
Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised according to the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. We will record and summarise where there is insufficient data on aspects of tailoring to inform our findings on the tailoring evidence gap
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 03 Mar 2022

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 20 Sep 22		read	read
Version 1 03 Mar 22	read	read

Elaine Yuen Ling Kwok, McMaster University, Hamilton, Canada; Northwestern University, Evanston, USA
Craig S. Rosen, VA Palo Alto Health Care System, Menlo Park, USA
Thomas von Lengerke, Hannover Medical School, Hannover, Germany

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

46 Views

08 Feb 2024 | for Version 2

Thomas von Lengerke, Department of Medical Psychology, Center for Public Health and Health Care, Hannover Medical School, Hannover, Germany

46 Views Cite this report Responses(1)

Approved

This article presents a protocol for a scoping review on processes and outcomes of tailoring implementation strategies in healthcare. In a nutshell, it aims to explore how tailoring had been defined/conceptualised, operationalized, and evaluated in healthcare so far. With the framework and principles of (Arksey and O'Malley , 2005) [Ref 1] and (the refinements of Levac et al, 2010) [Ref 2]., the JBI guidance(Micah DJ Peters et al,2020) [Ref 3], and the PRISMA-ScR guidelines (Tricco AC et al,2018) [Ref 4], in my view all relevant scoping review recommendations have been appropriately considered.

As the article has already been subject to one round of reviewing and revision, many issues have already been, in my view, sufficiently dealt with, among others exclusion of clinical interventions in precision medicine tailored to patient characteristics, how to deal with multiple related, but separate publications from individual projects, and the role of grey literature. Thus, I can only add a limited number of suggestions:

a.) Fourth aim (see "Purpose of conducting the scoping review"): Isn't the identification of knowledge gaps and future research priorities always one aim of (scoping) reviews? That is, doesn't this aim go without saying?

b.) Broad definition of tailoring (see "Purpose of conducting the scoping review"): Ultimately, tailoring refers to actions/activities/behaviours of proxy agents and implementation support practitioners. If this is what is meant with "proactive process" and "modifications of an" in the definition, that's fine; yet, I suggest to think about being more explicit here, e.g. by changing the second wording into "proactively modifying an".

c.) Please add information on how the scoping review will deal with implementation interventions tailored to proxy agents/implementation support practitioners characteristics (second-order implementation
interventions [1]) in contrast to implementation interventions tailored to clinicians (first-order implementation interventions [1]). For example, one can tailor implementation interventions to support clinicians hand hygiene (first-order), or one can tailor implementation interventions such as train-the-trainer workshops.to improve the quality of educational trainings on hand hygiene (which are eventually provided by infection and control teams to clinicians - second-order). Will the review include second-order implementations interventions as well (which would be of added value since they ultimately aim to enhance the adoption, implementation, and sustainment of clinical programs or practices as well)? And if so. How will the two different interventions types be handled and presented?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

1. H.Arksey L. O'Malley: Scoping studies: towards a methodological framework. International journal of social research methodology. 2005; 8 (1). Publisher Full Text
2. Levac D, Colquhoun H, O'Brien KK: Scoping studies: advancing the methodology.Implement Sci. 2010; 5: 69 PubMed Abstract | Publisher Full Text
3. Peters MDJ Godfrey C: JBI Manual for Evidence Synthesis. JBI/year. Publisher Full Text
4. Tricco AC, Lillie E, Zarin W, O'Brien KK, et al.: PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation.Ann Intern Med. 2018; 169 (7): 467-473 PubMed Abstract | Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

preventing healthcare-associated infections by promoting professional compliance with evidence-based guidelines; population hand hygiene behaviour; public health psychology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

25 May 2024

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

Thank you for reviewing our paper and providing your feedback, both the constructive suggestions and the very positive and encouraging view of the protocol overall. Your time and attention to this work are greatly appreciated.

a.) Fourth aim (see "Purpose of conducting the scoping review"): Isn't the identification of knowledge gaps and future research priorities always one aim of (scoping) reviews? That is, doesn't this aim go without saying?

Response:
We acknowledge that the statement about the fourth aim is valid and that this aim is somewhat obvious given that this is an aim of scoping reviews in general. However, as this is still a key aim and rationale for conducting this review, we believe that the statement of this aim still merits explicit inclusion in the protocol.

b.) Broad definition of tailoring (see "Purpose of conducting the scoping review"): Ultimately, tailoring refers to actions/activities/behaviours of proxy agents and implementation support practitioners. If this is what is meant with "proactive process" and "modifications of an" in the definition, that's fine; yet, I suggest to think about being more explicit here, e.g. by changing the second wording into "proactively modifying an".

Response:
We accept this as a reasonable suggestion and will amend the phrasing of the second part of the definition as suggested, in the final report.

c.) Please add information on how the scoping review will deal with implementation interventions tailored to proxy agents/implementation support practitioners characteristics (second-order implementation interventions [1]) in contrast to implementation interventions tailored to clinicians (first-order implementation interventions [1]). For example, one can tailor implementation interventions to support clinicians hand hygiene (first-order), or one can tailor implementation interventions such as train-the-trainer workshops to improve the quality of educational trainings on hand hygiene (which are eventually provided by infection and control teams to clinicians - second-order). Will the review include second-order implementations interventions as well (which would be of added value since they ultimately aim to enhance the adoption, implementation, and sustainment of clinical programs or practices as well)? And if so. How will the two different interventions types be handled and presented?

Response:
Regarding the distinction between first and second order implementation interventions, we thank the reviewer for bringing this up. We had not considered making this distinction when planning the review approach and second order implementation interventions were not excluded during the screening process. A brief question about this will be added to the data extraction form (i.e. to select whether the study contains first-order interventions, second-order interventions, or both), but the review will not focus on this in depth as this is considered to be outside of the scope of the current review and would not be feasible for us to add to the already extensive synthesis that is currently planned, at least with our existing time and resources. By clearly tagging the papers as containing first or second order interventions during the data extraction process, this could enable other researchers to explore this in more depth in future.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

38 Views

15 Nov 2022 | for Version 2

Craig S. Rosen, Dissemination & Training Division, National Center for PTSD, VA Palo Alto Health Care System, Menlo Park, CA, USA

38 Views Cite this report Responses(0)

Approved

The revisions are all appropriate and strengthen the protocol. I would propose accepting as is.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Implementation science; mental health services reseach; posttraumatic stress

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

49 Views

07 Jun 2022 | for Version 1

Craig S. Rosen, Dissemination & Training Division, National Center for PTSD, VA Palo Alto Health Care System, Menlo Park, CA, USA

49 Views Cite this report Responses(1)

Approved With Reservations

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.
p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
Table 2 – the listed study question for ‘authors’ appears incorrect.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Implementation science; mental health services reseach; posttraumatic stress

Respond to this report

Responses (1)

Author Response

21 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

Thank you for reviewing our paper and for the suggested amendments.

The first sentence under 'purpose of conducting the scoping review': "Despite varying definitions and lack of clarity... tailoring is recommended.." seems backwards to me. It might be clearer to say that tailoring is commonly recommended yet there is a lack of clarity on what tailoring is and how it should be done.

We have amended the text (pg 6):

Despite varying definitions and an apparent lack of clarity about how to best conduct it in practice, tailoring is commonly recommended to support the successful delivery of EBIs.

p.4 Inclusion criteria and conceptualization. It would help to be explicit here that the review of how has tailoring been defined and conceptualized in healthcare contexts will be limited to the tailoring of “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice”. Without this clarification, “tailoring in health care contexts” might include precision medicine tailoring of clinical interventions to patient characteristics.
Thank you for this suggested clarification. We have amended the conceptualisation objective (pg 7) and inclusion criteria in Table 1 and pg 10 as below:

Objective 1: Conceptualisation

1. How has tailoring of implementation strategies (i.e. methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) been defined and conceptualised in healthcare contexts?
2. Is there a rationale for tailoring and what is the rationale?
3. Is tailoring based on a theory which explains why it should work?

Context
Studies which use a tailoring process to develop strategies (methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice) for application in healthcare settings will be included.

p.5. The listed “justification” for Population in Table 1 does not seem to fit and seems redundant with the justification for Context. An alternative justification might be that all populations involved in health care research are relevant.
We recognise the reviewers point and have amended this on pg. 8 and in Table 1.

Table 1:
All populations involved in health care research are relevant. This could include healthcare professionals, implementation researchers, and other people who plan, provide and/or use health services

View more View less

Competing Interests

None

Back to all reports

Reviewer Report

72 Views

23 Mar 2022 | for Version 1

Elaine Yuen Ling Kwok, CanChild, Institute of Applied Health Sciences, McMaster University, Hamilton, ON, Canada; Northwestern University, Evanston, IL, USA

72 Views Cite this report Responses(1)

Approved With Reservations

Yes, the objective & rationale of the study are clearly described and I agree with the importance of conducting this scoping review.

Is the study design appropriate for the research question (including statistical power analysis, where appropriate)?

Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019)¹ or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”? Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?
A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
- Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005. Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?
- Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).
- Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.
- Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.
- Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?
- Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?). Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description. It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

Are sufficient details of the methods provided to allow replication by others?

Yes, see comments above.

Have the authors pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks?

No major concerns here. One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

Are the datasets clearly presented in a useable and accessible format?

N/A.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Partly
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Rehabilitation/Clinical Services Research

Respond to this report

Responses (1)

Author Response

22 Sep 2022

Fiona Riordan, School of Public Health, University College Cork, Cork, T12 XF62, Ireland

Research objectives/questions: This part is a strength. Clear objectives. Just a minor comment under objective 2: Consider expanding #5 to describe the context in more depth (i.e., beyond the healthcare setting), perhaps literature on implementation context (e.g., Nilsen & Bernhardsson 2019) or determinants framework (e.g., TDF, CFIR) can guide with this data extraction on “context”?

Thank you for reviewing our paper and your suggestions, these are greatly appreciated.
We agree there is value in exploring tailoring in other settings beyond healthcare settings. However, we have limited the scope of review to healthcare settings given the resources and time available. Given the ambiguity in terms of what constitutes tailoring and that the types of studies we may identify for this review are quite broad, we needed to introduce some boundaries on other criteria to make it more manageable.

Within the healthcare setting, we will examine the different levels of setting, that is the local level, wider organizational level, and external health system level. This has been added as part of data extraction. We have amended the text in Table 2 to capture this:

What is the described healthcare setting? (Where this information is available, describe the setting in terms of local/team level, organisational level, and external health system level)

Also please consider describing the prioritization of determinants in more depth. I would be very interested to know what criteria were used in prioritizing, and who selected those criteria (e.g., were determinants prioritized based on patients’ outcomes/implementation outcomes…)?

We will describe the prioritisation of determinants in greater detail. We have amended Table 2 to reflect this:

How has the prioritisation and selection step of tailoring been conducted?
What criteria have been used in prioritisation?
Who selected these criteria?
What data collection or broader methods have been used
Is there a rationale for the methods used for this step?
What is the rationale?
What are the inputs during strategy selection – evidence, stakeholders, theory?
What is the balance and timing of each input?

A scoping review methodology is timely and appropriate for the topic. I am raising a few suggestions/ comments for the authors to consider, and hope these comments will help improve the impact of your review:
Scope of the literature searched: Appropriate use of search strategies, although some justifications will help strengthen the decision to only include literature starting in year 2005.

We have expanded on our justification in Table 1:

Given the breadth of the search and resources available, the decision was made to focus on over 17 years of literature. The flagship journal in the field, Implementation Science started in 2006, signalling the formal establishment of the field in 2006 and subsequently, dedicated journals, including Implementation Science Communications (2020), Implementation Research and Practice (2020), and Global Implementation Research and Applications (2021) have been established.

We have also added the following to the main text (’Context’) (pg 11):

A preliminary electronic search for key terms (“tailor*” and terms for strategy or intervention and implementation and healthcare) in title and abstracts along with relevant subject headings in MEDLINE (14) indicates that in the last 20 years, there has been an upward trend in articles referring to tailoring, with 1722 articles published 2012-2021 compared to 443 articles in the 10 years (2002-2011) previous. In total there were 152 articles published 1972 – 2005 and 2148 from 2006 to 2022. Hence our decision regarding search date limitations.

To note, we also include additional clarification on why we completed dedicated searches for a specific tailoring approach, ‘implementation mapping’ (pg.11):

Dedicated searches for a specific tailoring approach, ‘implementation mapping’, will be completed for each database. This specific approach was chosen as it is unique to tailoring implementation strategies. Other methods (intervention mapping concept mapping, conjoint analysis, and group model building) while suggested as possible tailoring approaches, can also be used in other fields and for other purposes. This lack of specificity was confirmed during preliminary searches conducted to inform search development.

We also wished to take this opportunity to be more explicit in terms of our eligibility, namely that studies which use ‘implementation mapping’ term will be included as this is taken to be an approach to tailoring (pg. 8):

While there is variation in the scope of tailoring (i.e., whether determinant identification and/or prioritisation, and strategy selection, and/or application and testing are included), strategy selection is a core aspect of tailoring. Authors must explicitly describe the process as tailoring or the strategy as tailored. Studies will also be included if they refer to the intervention/strategy development process as ‘implementation mapping’. This term was adapted from ‘intervention mapping’ and first used by Fernandez et al to specifically describe an approach to tailor implementation strategies.

We also provided further clarification on dealing with ambiguity between evidence-based interventions and implementation strategies (pg. 11):

We will commit to the definition of strategy cited above: “methods or techniques used to enhance the adoption, implementation, and sustainment of a clinical program or practice” ¹. For this definition to apply there must be a clearly identifiable intervention (program, practice, guideline, policy or innovation) being implemented. The focus will not be on the nature or type of intervention being implemented, or whether it should be implemented, rather the focus will be on the tailored strategy supporting implementation. Where the delineation between intervention and strategy is unclear, we will refer to the ERIC taxonomy to identify implementation strategies. At the full text screening stage, if we cannot clearly delineate the intervention, we will contact the study authors for more information. If the author cannot provide more clarity, we will include the study but report the ambiguity.

Another comment is for authors to consider how they may handle large “tailoring” projects that resulted in multiple related, but separate publications e.g., some authors may choose to report their evaluation & prioritization of implementation determinants in a separate article from their “tailoring” phase, and then a separate article from their evaluation of the tailored implementation strategy phase. Just wondering how your search strategy can capture these related studies?

We agree. This is something we have discussed when piloting our screening approach.

We expect that the search strategy will capture multiple articles from the same study. As suggested, some may have separate articles which cover the evaluation of a tailored intervention or strategy. During title and abstract screening, we will be inclusive. We will include cost-effectiveness studies and evaluations of tailored strategies. During full text screening these ‘results’ papers will be checked for reference to the tailoring phase. We will exclude them at full text screening stage if they do not provide further details on the tailoring process (in line with our eligibility criteria) or refer to a related publication which describes this process. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Under our description of study selection (pg. 12), we include some additional details on this point:

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Inclusion criteria: Would be helpful to explain why grey literature is excluded in this review. Given the fact that scoping review allows for the inclusion of grey literature (e.g., conference proceedings/ presentation, government/not-for-profit project reports, grant applications/reports), I think it would be a great opportunity to capture how “tailoring” happens in the “real-world” by including non-traditional work (i.e., non-academic literature).

Grey literature will be searched, albeit in a limited way, using Google Scholar, using the title-only function, and screening the first 1000 records. We will include some unpublished (grey literature) including theses/dissertations, book chapters, toolkits, guidance, and reports but exclude conference abstracts and other non-peer reviewed sources (i.e., presentations, websites, blogs, studies in progress) (pg. 9 and Table 1). We have amended Table 1 to clarify the nature of the grey literature.

Originally, we limited the grey literature search due to resource constraints and our uncertainty as to whether it would be worth the yield, for example, searching organisational websites to identify reports. We suggest that future work could look more extensively at the application of tailoring in non-academic literature.

Please note that we have also amended the related question in Table 2 Data charting elements (pg 13):
Is the document a peer-reviewed, published article or grey literature?

Study selection: No major concerns. Just noting the individual responsible for abstract and full-text screening should probably be trained to understand and to be able consistently apply the definition of “tailoring” & inclusion criteria in this project- this may be tough as the concept of “tailoring, as the authors pointed out, is still poorly defined.

We have expanded our review team and piloted our screening approach to ensure we consistently apply the definition and inclusion criteria, holding a number of alignment meetings to discuss uncertainties. We limited the pilot to 250 articles, so that we could dedicate more time to discussion and reflect on any discrepancies. We held a number of meetings during the piloting process to discuss the application of eligibility criteria. As it is difficult to apply the concept of tailoring, we have agreed to be inclusive at the title and abstract screening stage, recognising that this will lead to a larger number of full texts to screen. We have also decided to apply double screening. All reviewers have an opportunity to practice applying the screening questions.

We have amended the text (pg 12) as follows:

The screening process will be carried out using Covidence. Titles/abstract and full text screening will be split between two or more reviewers. The title/abstract screening process will be piloted initially – 250 titles and abstracts will be screened by the review team using the predefined eligibility criteria. If any discrepancies are identified, these will be discussed within the team. Inter-rater reliability will be calculated for title and abstract screening using the kappa (κ) statistic to indicate alignment during the piloting stage. The screening process will involve double-screening. Two reviewers will screen each title and abstract, with conflicts resolved by consensus or a third reviewer. Full texts will also be screened in duplicate with conflicts resolved by consensus or a third reviewer. Reasons for exclusion of full-text articles will be documented to PCC in Covidence. The final search results will be outlined in a PRISMA flow diagram from the PRISMA-ScR statement.

We anticipate that there may be large projects which result in multiple, related but separate publications. For example, authors may report the tailoring process in one publication but an evaluation of the tailored strategy in another, or they may present identification and prioritisation in one publication but strategy selection in a separate publication.

Overall, we will aim to be inclusive when screening title and abstracts, for example, we will include cost-effectiveness studies and evaluations of tailored strategies. We will exclude these studies at the full text screening stage if they do not detail the tailoring process or refer to related publications which describe the process. We will check these papers for references to related studies. If there are multiple papers relating to the same study, not all will be included; we will only include papers from a study if they provide information on the tailoring process and the tailored intervention.

Charting the data: I really appreciate the mapping of your data extraction plan to your research objectives/questions. This part is clear to me. Please consider my comments under ‘Research objectives/ questions” above.

We have reflected on data extraction in study context above and made amendments to capture different levels within the healthcare setting.

Collating & summarizing results: The plan to summarize and described the studies is clear and I don’t have major concerns. The plan to highlight the gaps in evidence should be expanded. Can you give an example of how you will do that? Will you lean on an existing implementation framework to help identify these gaps? Or what will be the specific methodologies involved in this step?

Our process to identify and highlight evidence gaps will involve:

Thematic analysis
Visual depiction (where appropriate)
Stakeholder consultation

As outlined in the protocol, evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies, and where there is uncertainty about the way to approach an aspect of tailoring. They will be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation).

In terms of visualisation, there is little consensus on the exact definition of what constitutes an evidence gap map but broadly speaking it is considered a systematic search to identify gaps in knowledge and/or future research needs, presenting these in a user-friendly format e.g., a visual figure or graph, or a searchable database (Miake-Lye et al., 2016).

Miake-Lye, I.M., Hempel, S., Shanman, R. et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 5, 28 (2016).

We have added some text to elaborate on the approach in our ‘Analysis’ section (pg 15):

Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke⁴¹, will be conducted to develop themes related to evidence gaps. Evidence gaps will broadly be organised according to the study objectives (i.e., conceptualisation, operationalisation, evaluation). For example, under the research question ‘How was the prioritisation of determinants and selection of strategies conducted?’ a theme could be ‘variable criteria used for prioritisation’. We would propose the gaps are potentially (1) lack of evidence on which criteria to use to prioritise determinants and (2) lack of evidence on the impact of the criteria chosen. Proposed evidence gaps will be reviewed as part of the stakeholder consultation (Stage 6). Where appropriate we will develop visual depictions (e.g., geographic or topical visualisation) to support thematic summaries of the evidence gaps.

We have amended the text under ‘Stage 6: consultation with stakeholders’ (pg 16):

Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation.

Consultation with stakeholders: It would help to understand how you plan to identify and recruit the stakeholders and what inclusion criteria will you use (e.g., would you include only stakeholders with tailoring experience? Will they represent various healthcare settings?).
Also, I would recommend a more focused stakeholder engagement plan, perhaps more specific to this scoping review. As it currently reads, stakeholders are engaged in this phase to, for example, suggest further resources, in that case, will these resources suggested by stakeholders be included in the review as well? If so, please add that to the search strategy description.

In the first instance stakeholders will involve co-authors (an international group of experts in the implementation science field) and project stakeholders. They will represent academic and healthcare settings. They may not all have direct experience of tailoring but will have experience in and be familiar with implementation of evidence-based practices (science and practice). Further stakeholders to consult may be suggested by this initial group (pg 16).

The purpose of the stakeholder consultation is to advise on the content and priorities of a tailoring research agenda and on the dissemination of review findings. This review is part of a larger research project on the tailoring process in healthcare. The larger project involves an international scientific advisory panel of experts in the field of implementation science working in different academic and health service entities. This group will be consulted, along with project stakeholders who represent health service organisations, during later phases of the analysis and interpretation of the results (Stage 5). The preliminary format of consultation will be through a structured discussion/focus group. Stakeholders will be presented with the proposed evidence gaps, organised by theme; for example, ‘decision-making during determinant prioritisation’. Agreement with the gap and its categorisation will be checked. Then, based on the agreed gaps, stakeholders will be asked to identify priorities for a research agenda focused on tailored approaches to implementation. The consultation will also inform how to present the findings in a format useful for implementation researchers and practitioners, and how best to disseminate the findings. The consultation process represents a knowledge exchange opportunity, and stakeholders will be able to build on the findings, bring different perspectives, and identify strategies for disseminating review findings. This will help ensure the findings have practical relevance.

We will not ask stakeholders to suggest additional resources as it is potentially extending or re-opening the review and given the search is systematic, we expect these resources should be picked up.

It seems that stakeholders are also engaged to formulate a research agenda and to help identify dissemination plans. So is the outcome of this scoping review to inform a research agenda? If so, please add that to the objectives? Or perhaps I have mistaken the purpose of this step and that stakeholders are only engaged to help identify gaps in the literature by bringing in their on-the-ground tailoring experience?

The purpose of the stakeholder consultation is to inform a research agenda for the CUSTOMISE project and advise on dissemination plans for the scoping review results. We have made this more explicit in the objectives (pg 6) and the stakeholder engagement section (pg 16).

Through seeking to answer the above questions we will determine gaps in the literature (Objective 4) which need to be addressed as part of future research on tailoring and inform a research agenda focused on tailored approaches to implementation (Objective 5).

One possible scenario, in my mind, is the amount of missing information the author may identify in the existing “tailoring” literature. Knowing that our field lacks conceptual clarity around “tailoring” or a reporting guideline, I am, of course, not suggesting it is the fault of any authors who already published their “tailoring” studies. I would be very interested to know how the authors of this review plan to address a potentially “patchy” amount of data available to them in answering each of their research questions (e.g., what if there weren’t sufficient data for an inductive analysis for certain question)?

We will capture and synthesize the level and completeness of information available on various aspects of tailoring as part of the results of this review. Studies will be included even if they provide minimal data as described below (pg 10):

Studies will only be included if they (a) describe the tailoring approach in some detail, that is, the authors describe at least one aspect of the tailoring approach (i.e., when it was conducted, who was involved, format/method used, steps involved, inputs) and (b) at least involve the selection of strategies.

In cases where data are lacking, we will report this in the results section and use this to inform our findings on research gaps. For example, our findings may point towards the need for systematic reporting of certain aspects of tailoring process. If there is insufficient data for inductive thematic analysis, the deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’. We have amended text to reflect how minimal data will be analysed (pg. 15) will reported (pg 16):

Pg. 15:
Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies (e.g., number of studies, publication years, study populations) and of the tailoring process (e.g., types and number of stakeholders involved, timing of tailoring). Deductive content analysis aligned to the research questions will be undertaken, to prepare a narrative summary to accompany the tabular results. Inductive thematic analysis, following the guidance of Braun and Clarke ⁴¹, will be conducted to develop themes related to evidence gaps. If there is insufficient data for inductive thematic analysis, deductive analysis will be used to capture the descriptive information which is available (or not) e.g., items can be categorised as ‘missing’.

Pg. 16:
Results will be reported using the PRISMA-ScR guidelines. A PRISMA flow diagram will be presented to show the study selection and reasons for exclusion at full text review. Quantitative results will be presented in tabular form (Objective 2), and as a narrative summary organised according to the review questions: tailoring definitions and conceptualisation (Objective 1), operationalisation (Objective 2), evaluation (Objective 3) and evidence gaps (Objective 4). Evidence gaps will be identified by considering the following: what information is missing in the description of tailoring, whether there are research questions which cannot be answered or only partly answered based on the available literature, where the most variation exists across different studies and where there is uncertainty about the way to approach an aspect of tailoring. We will record and summarise where there is insufficient data on aspects of tailoring to inform our findings on the tailoring evidence gap

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Proctor EK, Powell BJ, McMillen JC: Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013; 8(1): 139. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Powell BJ, Waltz TJ, Chinman MJ, et al.: A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015; 10: 21. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Powell BJ, Beidas RS, Lewis CC, et al.: Methods to Improve the Selection and Tailoring of Implementation Strategies. J Behav Heal Serv Res. 2017; 44(2): 177–94. PubMed Abstract | Publisher Full Text | Free Full Text

[4] 4. Baker R, Camosso-Stefinovic J, Gillies C, et al.: Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015; 2015(4): CD005470. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Wright J: Tailoring Stroke Best Practice Recommendations to the Care Home Context: Identifying a Balance Between Theory and Real-World Practice. PQDT - Glob. 2019. Reference Source

[6] 6. Jensen JD, King AJ, Carcioppolo N, et al.: Comparing tailored and narrative worksite interventions at increasing colonoscopy adherence in adults 50-75: a randomized controlled trial. Soc Sci Med. 2014; 104: 31–40. PubMed Abstract | Publisher Full Text

[7] 7. Gatwood J, Balkrishnan R, Erickson SR, et al.: The impact of tailored text messages on health beliefs and medication adherence in adults with diabetes: A randomized pilot study. Res Soc Adm Pharm. 2016; 12(1): 130–140. PubMed Abstract | Publisher Full Text

[8] 8. Lake AJ, Browne JL, Abraham C, et al.: A tailored intervention to promote uptake of retinal screening among young adults with type 2 diabetes - an intervention mapping approach. BMC Health Serv Res. 2018; 18(1): 396. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Bührmann L, Schuurmans J, Ruwaard J, et al.: Tailored implementation of internet-based cognitive behavioural therapy in the multinational context of the ImpleMentAll project: a study protocol for a stepped wedge cluster randomized trial. Trials. 2020; 21(1): 893. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Kwok EYL, Moodie STF, Cunningham BJ, et al.: Selecting and tailoring implementation interventions: A concept mapping approach. BMC Health Serv Res. 2020; 20(1): 385. PubMed Abstract | Publisher Full Text | Free Full Text

[11] 11. Flottorp SA, Oxman AD, Krause J, et al.: A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013; 8(1): 35. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Wensing M, Oxman A, Baker R, et al.: Tailored implementation for chronic diseases (TICD): A project protocol. Implement Sci. 2011; 6(1): 103. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Moussa L, Kocbek S: Data-Driven Approach for Tailoring Facilitation Strategies to Overcome Implementation Barriers in Community Pharmacy. 1805; 1–25.

[14] 14. Hoffmann TC, Glasziou PP, Boutron I, et al.: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014; 348: g1687. PubMed Abstract | Publisher Full Text

[15] 15. Wilson KM, Brady TJ, Lesesne C, et al.: An organizing framework for translation in public health: The knowledge to action framework. Prev Chronic Dis. 2011; 8(2): A46. PubMed Abstract | Free Full Text

[16] 16. Shrubsole K, Lin TJ, Burton C, et al.: Delivering an iterative Communication Partner Training programme to multidisciplinary healthcare professionals: A pilot implementation study and process evaluation. Int J Lang Commun Disord. 2021; 56(3): 620–36. PubMed Abstract | Publisher Full Text

[17] 17. Reynolds SS, Woltz P, Keating E, et al.: Results of the CHlorhexidine Gluconate Bathing implementation intervention to improve evidence-based nursing practices for prevention of central line associated bloodstream infections Study (CHanGing BathS): a stepped wedge cluster randomized trial. Implement Sci. 2021; 16(1): 45. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Miller CJ, Barnett ML, Baumann AA, et al.: The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021; 16(1): 36. PubMed Abstract | Publisher Full Text | Free Full Text

[19] 19. Walsh-Bailey C, Palazzo LG, Jones SM, et al.: A pilot study comparing tools for tracking implementation strategies and treatment adaptations. Implement Res Pract. 2021; 2: 263348952110160. Publisher Full Text

[20] 20. Colquhoun HL, Squires JE, Kolehmainen N, et al.: Methods for designing interventions to change healthcare professionals' behaviour: A systematic review. Implement Sci. 2017; 12(1): 30. PubMed Abstract | Publisher Full Text | Free Full Text

[21] 21. Krause J, Van Lieshout J, Klomp R, et al.: Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implement Sci. 2014; 9(1): 102. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Powell BJ, Fernandez ME, Williams NJ, et al.: Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health. 2019; 7(3): 3. PubMed Abstract | Publisher Full Text | Free Full Text

[23] 23. Colquhoun HL, Levac D, O’Brien KK, et al.: Scoping reviews: Time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014; 67(12): 1291–4. PubMed Abstract | Publisher Full Text

[24] 24. Levac D, Colquhoun H, O’Brien KK: Scoping studies: advancing the methodology. Implement Sci. 2010; 5: 69. PubMed Abstract | Publisher Full Text

[25] 25. Arksey H, O’malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1): 19–32. Publisher Full Text

[26] 26. Peters MDJ, Marnie C, Tricco AC, et al.: Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020; 18(10): 2119–26. PubMed Abstract | Publisher Full Text

[27] 27. Peters MDJ, Godfrey CM, Khalil H, et al.: Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015; 13(3): 141–6. PubMed Abstract | Publisher Full Text

[28] 28. Health Protection Surveilliance Centre: Definition of a healthcare worker. Reference Source

[29] 29. Eldh AC, Almost J, DeCorby-Watson K, et al.: Clinical interventions, implementation interventions, and the potential greyness in between -a discussion paper. BMC Health Serv Res. 2017; 17(1): 16. PubMed Abstract | Publisher Full Text | Free Full Text

[30] 30. Joanna Briggs Institute (JBI): JBI Manual for Evidence Synthesis 4.2.6 Search strategy. 2019. Reference Source

[31] 31. Lewis CC, Scott K, Marriott BR: A methodology for generating a tailored implementation blueprint: An exemplar from a youth residential setting. Implement Sci. 2018; 13(1): 68. PubMed Abstract | Publisher Full Text | Free Full Text

[32] 32. Powell BJ, Haley AD, Patel SV, et al.: Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring I. Implement Sci Commun. 2020; 1: 9. PubMed Abstract | Publisher Full Text | Free Full Text

[33] 33. Huntink E, van Lieshout J, Aakhus E, et al.: Stakeholders' contributions to tailored implementation programs: an observational study of group interview methods. Implement Sci. 2014; 9: 185. PubMed Abstract | Publisher Full Text | Free Full Text

[34] 34. Jäger C, Steinhäuser J, Freund T, et al.: Process evaluation of five tailored programs to improve the implementation of evidence-based recommendations for chronic conditions in primary care. Implement Sci. 2016; 11(1): 123. PubMed Abstract | Publisher Full Text | Free Full Text

[35] 35. Wensing M, Huntink E, van Lieshout J, et al.: Tailored implementation of evidence-based practice for patients with chronic diseases. PLoS One. 2014; 9(7): e101981. PubMed Abstract | Publisher Full Text | Free Full Text

[36] 36. Bramer WM, Rethlefsen ML, Kleijnen J, et al.: Optimal database combinations for literature searches in systematic reviews: A prospective exploratory study. Syst Rev. 2017; 6(1): 245. PubMed Abstract | Publisher Full Text | Free Full Text

[37] 37. Riordan F, McHugh S, Lewis CC, et al.: Characterising processes and outcomes of tailoring implementation strategies in healthcare: A scoping review. 2022. http://www.doi.org/10.17605/OSF.IO/ER9HY

[38] 38. Haddaway NR, Collins AM, Coughlin D, et al.: The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS One. 2015; 10(9): e0138237. PubMed Abstract | Publisher Full Text | Free Full Text

[39] 39. Wensing M: The Tailored Implementation in Chronic Diseases (TICD) project: introduction and main findings. Implement Sci. 2017; 12(1): 5. PubMed Abstract | Publisher Full Text | Free Full Text

[40] 40. McKeown S, Mir ZM: Considerations for conducting systematic reviews: evaluating the performance of different methods for de-duplicating references. Syst Rev. 2021; 10(1): 38. PubMed Abstract | Publisher Full Text | Free Full Text

[41] 41. Braun V, Clarke V: Thematic analysis a reflexive approach. Reference Source

[42] 42. Brien SE, Lorenzetti DL, Lewis S, et al.: Overview of a formal scoping review on health system report cards. Implement Sci. 2010; 5(1): 2. PubMed Abstract | Publisher Full Text | Free Full Text

Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review

Abstract

Keywords

Introduction

Purpose of conducting the scoping review

Protocol

Design

Table 1. Eligibility criteria for the review.

Table 2. Data charting elements.

Discussion

Study status

Data availability

Underlying data

Extended data

Software availability

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

Characterising processes and outcomes of tailoring implementation strategies in healthcare: a protocol for a scoping review

Abstract

Keywords

Introduction

Purpose of conducting the scoping review

Protocol

Design

Table 1. Eligibility criteria for the review.

Table 2. Data charting elements.

Discussion

Study status

Data availability

Underlying data

Extended data

Software availability

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!