Involving patients, family, partners and/or carers in measuring and monitoring patient safety in hospitals: Protocol for an umbrella review

Review design and protocol registration

Umbrella reviews (also commonly referred to as overview of reviews, review of reviews, meta-reviews^30,33) serve to gather, assess, and synthesise evidence from multiple reviews on a specific topic,³⁰ in order to aggregate evidence across multiple syntheses, identify patterns in the results, and evaluate whether the existing reviews offer consistent or conflicting evidence.²⁹ This is important as multiple existing reviews on a given topic complicate evidence-based decision making and are challenging for end users to contend with.^30,33,34 Consequently, an umbrella review methodology was adopted, given their suitability for topics that have been researched or reviewed previously and for which the determination of evidence-based, or best, practice is important. Existing evidence syntheses in the area^1,19,23–26 have been limited by a focus on individual methods and single dimensions of safety in isolation,⁵ and an umbrella review will comprise an importance support for progressing efforts to involve patients in MMS.

This umbrella review will be conducted in line with the Joanna Briggs Institute (JBI) methodological guide for umbrella reviews³⁵ and best practice guidance for the conduct of umbrella reviews.^33–35 Further, the review will be reported in accordance with the Preferred Reporting Items for Overviews of Reviews (PRIOR) checklist.³⁰ The current protocol was prepared as per the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement,³⁶ coupled with careful attention to the JBI Reviewer’s Manual³⁵ and PRIOR checklist.³⁰ The PRISMA-P checklist has been provided in Extended data, Appendix 1. The protocol was registered with the International Prospective Register of Systematic Reviews, PROSPERO.

Search strategy

Searches will be conducted across five electronic databases, specifically: MEDLINE (OVID); CINAHL (EBSCO); PsycINFO (EBSCO); Scopus (Elsevier), and; Academic Search Complete (EBSCO). The search strategy was developed by the research team, and includes Subject Headings as well as free-text keywords related to: 1) patient safety (e.g., exp Patient Safety/, ‘patient safety’, ‘patient harm’), 2) patient, family, partner and carer involvement (e.g., exp Patient Participation/, ‘patient participation’, ‘family involvement’), and 3) the use of a review methodology (e.g., ‘systematic review’, ‘meta-analysis’). A search strategy was initially developed for MEDLINE (see Table 1 for MEDLINE search strategy) and adapted as necessary for use in the other databases (see Extended data, Appendix 2 for search strategies used across all electronic databases). The development of the search strategy was informed by consideration of previous reviews of patient-report patient safety measurement tools that were known to the review team,^{1,2,19,23–26} as well as umbrella reviews of measurement tools for use in healthcare.^37–39 In addition, the search strategy has been reviewed and agreed by a University research librarian. Consideration of PRESS (Peer Review of Electronic Search Strategies⁴⁰) and reporting standards for literature searches by Atkinson and colleagues’⁴¹ has supported us in ensuring search quality, transparency and replicability.

To complement electronic database searches, both backward and forward citation searching, as per best practice recommendations,⁴² will be employed. First, the reference lists of all studies deemed eligible for inclusion following the initial electronic searches will be screened. Similarly, the reference lists of a previous umbrella review⁴³ and systematic reviews^44–47 that explored patient engagement in healthcare safety more broadly will also be reviewed to identify any additional relevant review papers. Second, using the included studies and the aforementioned reviews,^43–47 we will conduct forward citation searching using Scopus and Google Scholar. These two citation indexes were selected to ensure a comprehensive capture of more recent work. All records identified through these methods will be tracked and reported separately in the PRISMA flow diagram.⁴²

Eligibility criteria

Eligibility criteria are outlined according to the PICo-T format for umbrella reviews,³⁵ comprising the Population (P), Phenomena of Interest (I), Context (Co), and Types of Studies (T). Accordingly, our eligibility criteria are as follows:

Population. The population of interest, within relevant reviews, will include adults (defined as individuals aged over 18 years) receiving care within an inpatient hospital setting. Recognising the valuable insight of the patient’s family and carers, we will also include reviews that engage family members, partners, or carers in the patient’s stead. Conversely, we will exclude reviews that focus solely on the MMS safety through engagement with other stakeholders, such as healthcare providers, without direct patient, family or carer involvement. Further, while some reviews may span multiple healthcare settings (e.g., both paediatric and adult populations), we will exclude those where data specific to adult inpatients cannot be clearly extracted.

Phenomena of Interest . The phenomenon of interest is the involvement of patients (or family members, partners, or carers) in MMS in inpatient hospital care settings. For reviews which describe a broader focus on patient involvement in patient safety (e.g., interventions designed to enhance safety through patient participation, of which involving patients in MMS is just one potential form/element), two independent reviewers will systematically consider the full-text manuscript to determine whether a clear focus within the study (e.g., within the study aim or data extraction) relates to patient involvement in MMS, and whether relevant data on such approaches are included and can be extracted. Conversely, we will exclude reviews that do not specifically synthesise approaches to MMS from the patient, family, partner or carer perspective. We will also exclude reviews where the focus of the approaches, tools, or instruments are broader than patient safety (e.g., general quality of care or patient experience), unless they contain at least one patient safety-specific subscale or section for which data can be extracted. Finally, we will exclude reviews of studies that rely upon soliciting patient input in response to hypothetical vignettes and scenarios only, or those designed to capture general opinions on patient safety rather than focusing on specific patient experiences, and the elicitation of these, related to safety.

Context . This umbrella review is specifically focused on patient safety within the context of adult inpatient hospital care. We define patient safety in accordance with the World Health Organisation⁴⁸ as “the prevention of errors and adverse effects to patients associated with health care.” Inpatient hospital care is characterised by the formal admission of a patient to a healthcare facility for treatment, diagnostic tests, or surgical procedures requiring a stay of at least one night (typically 24 hours) that concludes with a formal discharge, occurring when the patient returns home, is transferred to another facility, or in the event of death.^49,50 While we recognise the importance of safety across the continuum of care, other healthcare domains, including primary and community care, outpatient services, and paediatric or maternity care, differ significantly in their patient population, and clinical and operational structures. Consequently, research relating to these settings, falls outside the scope of this review.

Study Type . Systematic reviews, and comprehensive reviews of any design type (e.g., scoping reviews, rapid reviews) with evidence of a systematic search strategy in at least two databases will be included, similar to previous mixed-method umbrella reviews of healthcare research.^51–53 Narrative review articles that do not follow a systematic approach, and primary studies will be excluded. There will be no restriction by country or publication year, though publications are required to be peer-reviewed and written in the English language.

Study selection

Studies retrieved in electronic database searches will be first exported into Endnote© (https://end-note.com/; Zotero (https://www.zotero.org/) may be considered as a free alternative that can perform equivalent functions) to support the removal of duplicate studies as per Bramer et al.’s de-duplication process.⁵⁴ Subsequently, all remaining records will be exported to Microsoft Excel© (https://www.microsoft.com/en-ie/microsoft-365/excel; Google Sheets (https://www.google.com/sheets/about/) may be considered as a free alternative that can perform equivalent functions) and screened according to their titles and abstracts. ROM and SL will first pilot the eligibility criteria on a random sample of the titles and abstracts of 10 reviews, similar to Moens et al.³⁸ Subsequently, ROM will screen the remaining returns, whereby the full texts of articles judged to meet the inclusion criteria, or where there was insufficient information to determine eligibility, will be retained for full-text review. These will be screened by ROM and a final decision on inclusion will be made. During the full-text screening stage, the specific reasons for the exclusion of any studies will be systematically documented within Microsoft Excel© and summarised in the final PRISMA flow diagram⁵⁵ to ensure methodological transparency and rigor. If there is ambiguity surrounding the suitability of an article, this will be discussed by the two authors (ROM and SL), in consultation with the wider research team where necessary, until consensus is achieved.

Data extraction

A data extraction form, informed by previous umbrella reviews^38,39,43 and systematic reviews of patient involvement in patient safety^1,19,23 has been developed. Table 2 presents the full range of variables for which data will be extracted. The form will be piloted in advance of beginning formal data extraction by two authors (ROM and SL). Where data are missing, this will be reported in the final review. Duplicate data extraction will be independently completed by two authors for each study, with disagreements resolved by discussion until consensus is achieved.

Methodological quality assessment

The methodological quality of the included reviews will be critically appraised using the Critical Appraisal Skills Programme (CASP) systematic review checklist.⁵⁶ The CASP systematic review checklist focuses on the methodological validity and practical relevance of a review.⁵⁶ The CASP checklist was selected for its explicit guidance and methodological flexibility^57–59 and ability to accommodate various review types, such as rapid and scoping reviews, without compromising on rigor.⁵⁶ The CASP checklist has been successfully applied in previous umbrella reviews of measurement tools.^38,39 The CASP checklist has three parts: part A, are the results of the study valid? (five items); part B, what are the results? (two items); and part C, will the results help locally? (three items). Similar to previous umbrella reviews of measurement tools,^38,39 two items will not be applied in our appraisal as they are specifically relevant to interventions: (7) “How precise are the results?”; and (8) “Can the results be applied to the local population?”. CASP items require a response of ‘yes’ (1 point), ‘no’ (0 points), or ‘can’t tell’ (0 points), returning a total score out of 7. To ensure accuracy, two reviewers will complete the quality appraisals for each review study in tandem, and any disagreements will be resolved through discussion or consultation with a third reviewer.

Certainty of evidence

To assess the certainty of evidence, we will employ a dual-framework approach, applying GRADE^60,61 to quantitative outcomes and GRADE-CERQual^62,63 to qualitative findings. While some have advised against appraising the certainty of the evidence in mixed methods reviews,^64,65 organisations such as the National Institute for Health and Care Excellence support the complementary use of GRADE and GRADE-CERQual⁶⁶ frameworks, and these frameworks have been applied in previous healthcare systematic and umbrella reviews.^67–73 Applying both frameworks will provide a more robust and holistic evidence base for decision-making than a single-framework assessment could achieve.⁷⁴

This assessment will be conducted at the level of each distinct approach to measurement (e.g., surveys, interviews, incident reporting) to determine the robustness of each method identified. Following established methodological guidance,^61,63 each approach will begin with a ranking of High certainty and is downgraded to Moderate, Low, or Very Low based on serious concerns across standardised domains. For quantitative data, downgrading will be based on methodological quality (informed by CASP scores), inconsistency of results across reviews, indirectness of the evidence, imprecision in findings, and potential publication bias.^60,61 For qualitative data, confidence will be evaluated based on methodological limitations, the coherence of findings, the adequacy of the data, and the contextual relevance.⁶³ Two reviewers (ROM and SL) will perform these assessments in tandem, with discrepancies resolved through discussion (as is recommended when using the GRADE-CERQual⁶³) and consultation with a third reviewer where necessary. The final certainty ratings for each approach will be summarised in Summary of Findings tables^60–63 to facilitate evidence-based decision-making. If both quantitative and qualitative data are provided for a specific data variable (e.g., qualitative and quantitative data for effectiveness/cost-effectiveness), a convergent-segregated approach will be conducted by performing separate quantitative and qualitative syntheses, then integrating the results derived from each of the syntheses.^66,75

Data synthesis

Given the anticipated diversity within the review papers included, content analysis⁷⁶ will be undertaken to synthesise the extracted data. This involves the development of categories/codes either de novo or through engagement with existing theories that are applied through the consideration of data extracted from each distinct study.⁷⁷ It is anticipated that content analysis will be used as part of data extraction and synthesis. This will comprise an initial stage of data synthesis, and will allow us to bring consistency of language across studies and extracted data and facilitate later comparison of studies. As presented in Table 2, it is anticipated that data pertaining to the following variables will be coded deductively by two authors independently during this process:

Methods of measurement. Approaches to MMS from the perspective of patients, family members, and carers will be categorised using a hybrid deductive-inductive approach.^78,79 While initial categorisation is informed by established frameworks (e.g., surveys, interviews, incident reporting, and record reviews)^39,80,81; this review recognises that these frameworks are largely derived from healthcare provider perspectives. Consequently, the classification will remain flexible and iterative, allowing for the identification and inclusion of novel, patient-led measurement approaches (e.g., narrative storytelling, real-time digital feedback) that may not be captured by traditional, healthcare professional-focused taxonomies.

Dimension of safety. The dimension of patient safety measured by the study’s approach to safety measurement will be coded according to the five dimensions of the MMS framework,^5,28 which includes past harm, reliability of safety critical processes, sensitivity to operations, anticipation and preparedness, and integration and learning. This framework is well-established in safety research as it allows for a multidimensional understanding of safety, which moves beyond retrospective incident reporting toward a more holistic, proactive view of safety system performance.^5,28 Consequently, it is commonly used in reviews examining the MMS in healthcare.^39,80,81

Feasibility and contextual appropriateness. Data provided on the feasibility and contextual appropriateness of the approaches to measurement will be coded according to the APEASE (Affordability, Practicability, Effectiveness/cost-effectiveness, Acceptability, Side Effects/Safety, Equity) framework.³¹ This provides a structured lens to evaluate not just if a tool works in a vacuum, but if it is viable for use by patients, and their families, within the complex environment of an acute hospital.³¹ When evaluating methods of MMS, the APEASE criteria provide a methodical way of assessing the acceptability, feasibility and contextual appropriateness of such approaches,³¹ in that, if all six criteria are achieved then an approach or intervention is more likely to be successful and lead to behaviour change (i.e., patients will use the approach to provide safety data).⁸²

Psychometric evidence. Evidence reported regarding the approach’s/tool’s psychometric properties will be categorised as validity, reliability, or usability, similar to the approach undertaken in Moens et al.^38,83 Recognising that included reviews may vary significantly in their depth of reporting, we will not apply a rigid, granular psychometric threshold (e.g., as used in specialised tool-validation reviews like COSMIN⁸⁴). Instead, we will adopt a flexible approach that accounts for the anticipated diverse nature of identified methods. Many approaches to involving patients in MMS, such as patient-report surveys (often targeting past harm or anticipation and preparedness dimensions of the MMS framework^5,28) may provide substantive psychometric data (e.g., the reviews by Madden et al.²³ and Li et al.¹⁹). Conversely, emerging measurement approaches, particularly those aimed at assessing the ‘sensitivity to operations’ or ‘integration and learning’ dimensions of the MMS framework (e.g., safety walk-rounds or the integration of safety lessons into practice^5,28), may prioritise practical utility and contextual responsiveness over formal psychometric properties.¹ Anticipating this variation, our synthesis will capture the breadth of evidence available, ensuring we do not inadvertently emphasise formal validation at the expense of capturing innovative, patient-informed approaches that are essential for holistic safety monitoring.³

Following this, a narrative synthesis approach will be undertaken to analyse the resulting data, guided by Popay et al.’s⁸⁵ approach to narrative synthesis. A meta-analysis will not be performed as the focus of umbrella reviews remains on summarizing existing evidence syntheses rather than generating new pooled estimates.³⁵ Accordingly, different methods and tools will be used as part of this narrative synthesis, such as text-based summaries of studies, the categorisation of similar studies, the use of tables and figures to present summary data, and the transformation of data using content analysis.⁸⁵ We will assess and report the overlap of primary research studies across the included reviews to ensure transparency regarding the evidence base.³⁰

Patient and Public Involvement (PPI)

This umbrella review is part of a wider programme of research (the Patient Involvement in Patient Safety (PIPS) project), the development of which was supported by the Irish Centre for Applied Patient Safety and Simulation’s (ICAPSS; https://www.universityofgalway.ie/icapss/) established PPI group. The research team, and protocol authorship, includes a patient representative (WR) who has contributed to protocol development and will support completion of the review. Further, PPI members will contribute to the development of accessible, lay-friendly summaries of the review’s findings, ensuring the findings can engage members of the public.