Effectiveness of quality improvement strategies for type 1 diabetes in children and adolescents: a systematic review protocol

Description of the condition

Type 1 diabetes (T1D) is primarily an autoimmune driven disease in which the β-cells of the pancreas become depleted, ultimately leading to an insulin insufficient state which is vulnerable to ketoacidosis¹ and a range of long-term complications, including retinopathy, nephropathy and neuropathy². For reasons which are not entirely understood at present, the incidence of T1D is increasing worldwide and the mean age at diagnosis is gradually becoming younger, ultimately culminating in a significantly greater burden of disease³. This is associated with increases in the health-related expenditure required to manage this complex disease and its associated morbidities. Although T1D is a difficult disease to manage even in the most motivated and informed of patients, children and adolescents represent a population with its own unique challenges in this regard^4,5. Suboptimal T1D control in youth increases the risk of acute and chronic diabetes complications, and is associated with poorer quality of life⁶. Multiple guidelines describe best practices in the management of T1D, but a significant proportion of children do not meet international targets for glycaemia control. Thus, there are significant immediate and long-term benefits to be achieved in applying evidence-based strategies for improving outcomes in children with T1D.

Description of the intervention

Quality improvement (QI) strategies are healthcare system interventions which aim to promote the application of best practices in healthcare settings. Such interventions may be comprised of organisation-, healthcare professional- and/or patient-level components, with changes to organisation or delivery of care at their core. Ultimately, the purpose of QI strategies is to use health system levers to increase the frequency of practices that should be standard of care. Several recent systematic reviews have examined the effects of patient-targeted components of such interventions in isolation, including patient education⁷ and promotion of self-management⁸. However, no systematic review to date has examined the full breadth of QI strategy components, independently or in combination, in children with T1D.

How the intervention might work

Although T1D is one of the most widely researched diseases in terms of novel technological interventions and best practices, the implementation of such practices is not universal and patient outcomes may suffer as a result. QI strategies aim to aid in the implementation of best practice through a range of methods which target the identified barriers. For example, such interventions may positively affect glycaemic management through case management or education practice⁹, healthcare engagement through outreach clinics or patient reminders¹⁰, or physician compliance with best practice guidelines through automated reminders and prompts or audit and feedback.

Why it is important to do this review

There is evidence to suggest that certain QI strategies support the implementation of best practices in the setting of diabetes in adults and result in improvements in intermediate outcomes, such as glycated haemoglobin (HbA1c)^11,12. However, these strategies are often not directly applicable to the paediatric population, where developmental, educational, behavioural, and family factors provide additional challenges and complexities to optimal diabetes care. Similarly, the setting in which care is delivered for paediatric populations is quite different to that of adult care, often involving greater resources and significant multidisciplinary collaboration, including input from an array of allied health professionals (i.e., physiotherapists, occupational therapists, dieticians, etc.). Therefore, understanding which of these strategies are most effective in improving the delivery of guideline-concordant practices for T1D in the paediatric setting would be of significant clinical use. This review builds upon a large Cochrane Effective Practice and Organisation of Care (EPOC) Living Systematic Review assessing effectiveness of QI strategies in adults with diabetes (previous iterations:^11,12). We now aim to conduct a complementary review for paediatric populations in an effort to establish the best available evidence for clinicians, investigators, policymakers, decision-makers and patients from birth to old age.

Criteria for considering studies for this review

Types of studies. This systematic review will include evidence from a wide range of study types in order to best capture data of interest. We will consider randomised controlled trials (RCT), cluster randomised controlled trials (cRCT), and quasi-randomised controlled trials (qRCT; i.e., allocation methods such as alternating, patient date of birth or medical record number). Studies must, however, report post-intervention HbA1c in order to qualify for inclusion, as this is the primary outcome of interest and the ultimate surrogate marker of T1D glycaemic control chosen for comparison. We will contact corresponding authors to request the data if a study meets all other criteria but fails to report post-intervention mean HbA1c. If we are unable to obtain HbA1c data for studies of relevance, then this will be identified in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) study flow diagram¹⁴ and individual studies will be summarised in a table entitled ‘Characteristics of studies without primary outcome of interest - HbA1c’. We will exclude all conference abstracts and proceedings from data extraction but will include them in the ‘Characteristics of studies awaiting classification’ table.

Types of participants. In the present review, studies of children and adolescents (<18 years) with a diagnosis of T1D being managed within a Pediatric Endocrinology outpatient setting will be considered. We aim to include children and adolescents in which a diagnosis of T1D has been established either by local (e.g., Diabetes Canada)¹ or international guidelines (e.g., International Society for Pediatric and Adolescent Diabetes [ISPAD])¹⁵ at the time of randomisation. However, we will not exclude studies in which the specific diagnostic criteria or guidelines followed have not been explicitly reported. In instances where mixed population data are reported (e.g., a 14 to 20-year-old population which is still managed in a Pediatric Endocrinology outpatient setting, or a population containing both T1D and T2D), studies will be included if they report subgroup data for our participants of interest or if such participants represent ≥80% of the reported population. If data are not reported as such, then the relevant study will be listed in the ‘Characteristics of studies awaiting classification awaiting classification’ table and the corresponding author will be contacted for subgroup data relating to our population of interest.

We will not consider rare genetic causes of insulin-dependent diabetes, such as Wolfram Syndrome (i.e., DIDMOAD [Diabetes Insipidus, Diabetes Mellitus, Optic Atrophy and Deafness] Syndrome) and Wolcott–Rallison Syndrome, given the specific comorbid nature of such conditions. Studies of children with a diagnosis of T2D or monogenic diabetes, inpatient populations and individuals >18 years will also not be considered.

Types of interventions. This review will explore diabetes QI strategies intended to improve management of children living with T1D (Table 1). As this is a novel review topic, the types of interventions that will be reported in the literature is not yet clear. Therefore, we have included a wide range of potential interventional components, which have been selected directly from the 2015 EPOC taxonomy¹⁶, with the notable addition of patient/carer education and exclusion of interventions which are solely comprised of continuous/flash glucose monitoring or insulin pump-related technologies, given the array of high quality systematic reviews available in the field^17–20.

Briefly, interventions are subclassified into those concerned with modes of delivering care, interventions focusing on financial arrangements, or those aimed at implementation strategies. In contrast to the patient-directed interventions examined in the review on QI strategies in adult diabetes, we expect carer or parent-directed interventions to be more prominent, particularly in the preteen years before responsibilities shift to the patient themselves²¹.

It is anticipated that the interventions will be complex, involving multiple QI components combined in varying combinations. For example, an intervention may include elements of case management, shared care, and telemedicine whereas another intervention may include elements of case management, shared care, and patient education. Our taxonomy has 43 components, which in theory, could yield 2⁴³ unique combinations. Given the novel nature of this review, we do not know what unique versions of QI interventions (i.e., combinations of components) will be most common. Where possible, we will aim to categorise interventions into their relevant subclassification (i.e., delivering care, financial arrangements, and implementation strategies) in order to contrast and discuss the effectiveness of such interventions in a meaningful manner; however, it is likely that there will be overlap of these also.

Types of outcomes. Outcomes of interest, which include HbA1c, other measurements of glycaemic control, diabetes complications, screening, and healthcare engagement, are outlined in Table 2. The basis of these outcomes and their assessment is drawn from the 2018 ISPAD guidelines²². However, as trials from different regions will likely follow their respective local guidelines, the targets for outcomes have been drawn from several societal guidelines where applicable (e.g., dichotomous cut-offs), including the National Institute for Health Excellence (NICE)²³, Diabetes Canada²⁴ and the American Diabetes Association (ADA) guidelines²⁵. This approach facilitates inclusion and narrative synthesis of all relevant studies and will not introduce heterogeneity into a quantitative synthesis if performed, as only HbA1c levels (%) will be considered for meta-analysis.

Search methods for identification of studies

Electronic searches. The search strategy outlined in the Extended data²⁶ will be employed in conducting searches. This strategy, which was designed by a qualified and experienced Information Scientist (TR), was developed from the established strategy for the adult companion systematic reviews^11,12, with added paediatric-specific filters and additional search terms relevant to the 2015 EPOC taxonomy. The search, which does not have any language limitations and spans 1946 to January 15, 2021, has undergone two independent peer reviews in compliance with the Peer Review of Electronic Search Strategies (PRESS) for systematic reviews²⁷. A filter for RCT, cRCT and qRCT type studies has been included. Material for the present review will be uncovered via searches in the following databases:

Searching other resources. The Health Systems Evidence Database will be searched and reviewed for grey literature or ancillary publications of relevance. This database facilitates the searching of implementation strategy-based interventions, such as those described in the 2015 EPOC taxonomy. We intend to perform a check of reference lists in included studies, as well as any relevant systematic reviews identified, in order to identify any additional references of relevance.

Data collection and analysis

Selection of studies. All citations resulting from the search strategy described above will be imported into the Covidence online platform, duplicates will be removed, and the remaining titles and abstracts will be screened for relevance prior to full text review. Assessment of study eligibility, as well as subsequent extraction of data, will be performed by at least two independent assessors (PMR/MZ). An initial training and quality assurance exercise will be conducted with each assessor who is naive to such QI interventions. New assessors will be asked to screen 100 studies and extract data from a small subset of these in order to assess quality and fidelity. Any conflicts arising from this screen and extraction will be discussed with the lead author and reviewers will then proceed to an additional round of screening and extraction. If conflicts are minimal in this round, then the reviewer will be allowed to progress to autonomy. Exclusion and inclusion of studies will be summarised in a modified PRISMA study flow diagram¹⁴.

Data abstraction and management. An electronic data abstraction form will be developed through modification of that which has been trialled and optimised through the previous iterations of the adult diabetes QI companion review^11,12. Two authors with extensive experience of data abstraction from the adult review (PMR/MZ) will gather information on aspects including design, baseline population characteristics, intervention QI strategy composition, and primary and secondary outcomes. Such data will be displayed in the ‘Characteristics of included studies’ table. Any discrepancies which arise during the screening or abstraction phases will be resolved by consensus or the involvement of a third reviewer (JMG) where this is not possible.

Dealing with duplicate and companion publications. Duplicated reports or those which report on data from the same study (e.g., companion publications) will be linked together and treated as a single study for the purposes of this review. Duplication of data within the final analysis will be prevented and only the report for the longest follow-up period will be considered for meta-analysis or qualitative best-evidence summary.

Assessment of bias in included studies

The Cochrane Risk of Bias (RoB) 1.0 tool will be applied to the included studies by two independent assessors (PMR/MZ)¹⁵, as it has been validated through use in the adult diabetes QI review^11,12. Supporting quotations and individual reasoning will be recorded for each judgement and summaries thereof will be presented.

Measures of treatment effect

We will include a summary of each included study, including their population demographics, content of QI and any active control interventions (i.e., component strategies), and outcomes. We will synthesize evidence across studies in a narrative summary primarily; however, if a subset of studies is judged to display a high degree of similarity (i.e., PICO), then a meta-analysis will be considered, and summary data of similar intervention-control comparisons reporting the same outcome data will be presented. Continuous data will be presented as mean difference, while dichotomous data will be presented as odds ratios. All parameters will be presented with 95% confidence intervals. For multiple arm studies, we will report all relevant comparisons (e.g., intervention A vs. control; intervention B vs. control). Should meta-analysis be feasible, we will include only a single effect size for each study; therefore, in the case of multi-arm studies, we will select the comparison representing the most active intervention vs. control.

Unit of analysis issues

Although a transparent and structured narrative summary remains the primary intended mode of evidence synthesis for this review, measures will still be taken to adjust for the discordance of designs in all relevant cases, in order to improve the accuracy and interpretability for the reader. For instance, cRCTs may produce spuriously precise confidence interval values, owing to failure to account for correlation. Therefore, in cases where only unadjusted cluster data is reported, the intraclass correlation coefficient (ICC) will be recorded and utilised to calculate the design effect and correct the estimate using established methods^13,30. Specifically, sample size will be divided by the design effect size in order to correct for the clustered design, as per instructions in the Cochrane Handbook for Systematic Review of Interventions¹³. If the ICC or number of clusters is not reported for a given study and attempts to retrieve these from the relevant authors are unsuccessful, then a single constant will be taken from a similar type study and imputed. Similarly, crossover RCTs have the potential for unit of analysis issues and carry-over/period effects, which may ultimately lead to spuriously imprecise confidence intervals. Therefore, we will only consider data from the first phase of intervention, prior to crossover in such trials.

Dealing with missing data

In the event that data of interest is not publicly available, we will contact the corresponding author to request missing data via email and telephone where possible, as this has been shown to improve response rate³¹. In the event that missing data is unavoidable, the data will be imputed as appropriate as per the Cochrane Handbook for Systematic Reviews of Interventions¹³. We will subsequently address the predicted impact of such methods on the outcomes and conclusions of the review in the discussion section.

Assessment of heterogeneity

We expect a high degree of heterogeneity in included studies given the wide range of study designs, interventions, and comparators deemed eligible in our inclusion criteria. We will explore heterogeneity by subjectively comparing studies’ populations, interventions, comparators, and effect sizes using data summaries and visual aids (study characteristic table, grids of intervention components, Forest plots of effect sizes). Should a subset of studies be deemed sufficiently similar to be combined quantitively, we will explore heterogeneity through the Cochran’s Q test and I² statistic. We will assume a random-effects model for any meta-analysis conducted. We do not anticipate having sufficient data to statistically explore heterogeneity of meta-analyses with subgroup or meta-regression analyses; however, we will seek to identify factors hypothesized to be associated with variation in effect sizes as reported by individual studies and from our narrative synthesis.

Assessment of publication bias

In the event that a meta-analysis is performed, Egger’s test and Begg’s test will be utilised in order to assess for the presence of publication bias, in addition to visual inspection of funnel plots if the number of studies eligible for meta-analysis is sufficient for such assessments (i.e., n>10). If a meta-analysis is performed and a significant degree of publication bias is detected, then we will explore the potential impact of studies which were excluded for not reporting HbA1c (i.e., those to be summarised in table ‘Characteristics of studies without primary outcome of interest - HbA1c’).

Assessment of reporting biases

Where a study protocol is available, two assessors will work independently to review the planned outcomes of interest. If a clear discrepancy exists between the protocol plan and the published manuscript reporting, then we intend to highlight this as an instance of reporting bias. If the protocol is in line with the published manuscript, but was published retrospectively, then we will determine it to be unclear as to whether reporting bias was present. Similarly, if no protocol is available and the manuscript methods match the reported outcomes, then we will be forced to declare the presence or absence of reporting bias as unclear. Ultimately, we will present a ‘Risk of bias graph’ and a ‘Risk of bias summary figure’.

Data synthesis

Narrative. The primary means of synthesis for the present review is intended to be a narrative summary of the study outcomes, which will be presented and discussed as per the Synthesis Without Meta-analysis (SWiM) reporting guidelines³². This method aims to improve transparency and structure in the reporting of non-statistical evidence synthesis. Studies will be compared and discussed through the lenses outlined in Table 3.

Meta-analyses. We will conduct random-effects meta-analysis of similar studies in a subset of studies are deemed to be sufficiently similar (see ‘Assessment of heterogeneity’) to warrant meaningful interpretation of their meta-analysed effect estimate. We will assume a random-effects model for all meta-analyses³³ using the metafor package in R³⁵.

Summary of findings table. A ‘Summary of findings’ table will be produced to present the evidence in a digestible manner. This table will present a range of study characteristics, such as the study PICO and design (including the intervention target - i.e., patient/parent/carer). In addition, details of the intervention, relating to both its individual components, framework (including the availability and source of any relevant materials) and overall purpose, will be displayed to facilitate replication. Finally, the direction and absolute magnitude of effect size for each outcome, number of studies and included participants available for each outcome, and overall confidence in each effect estimates will be summarised. This table will be generated using RevMan v5.4 and RevMan Web v2.0.1.

Certainty of evidence. The overall certainty for each outcome will be assessed via the GRADE method, which explores the internal and external validity of the study, in conjunction with a validated checklist for reproducible reporting³⁶. As with other assessments, two independent reviewers will assess each study (PMR/MZ), while discrepancies will be addressed through consensus where possible or through involvement of a senior author where not achieved (JMG).

Underlying data

No underlying data are associated with this article.

Extended data

Mendeley Data: Effectiveness of quality improvement strategies for type 1 diabetes in children and adolescents: a systematic review search strategy. https://doi.org/10.17632/ny39z7w5wc.2²⁶.

This project contains the following extended data:

Reporting guidelines

Mendeley Data: PRISMA-P checklist for “Effectiveness of quality improvement strategies for type 1 diabetes in children and adolescents: a systematic review protocol”. https://doi.org/10.17632/ny39z7w5wc.2²⁶.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).