Keywords
rapid realist review, quality care metrics, care processes, quality indicators
rapid realist review, quality care metrics, care processes, quality indicators
Version 2 of this article reflects amendments based on valuable comments and input of reviewers. A renewed focus on process outcomes specifically, clarifies and strengthens the article by considering whether care interventions adheres to best practice guidance. More importantly, nursing and midwifery quality care process metrics are highlighted as providing the framework to identify care delivery gaps which enable quality improvement action and provide a mechanism for care provider accountability for the quality of care delivered. Explanations of terms such as 'quality care process-metrics and 'high quality care', furthermore provide clarity and underpin the article background. Expanded methodological considerations of a rapid realist review, likewise explains its utilisation in situations where policymakers and practitioners require guidance to inform emergent decisions in practice. Also, the 'rapid' review is deliberately intended to streamline evidence synthesis and provide practical outcome-based results over a short period of time, while the usage of an expert panel will ensure direct engagement with policy makers and knowledge users to guarantee the review's relevance to the practice context. The inclusion of additional sub research questions within the article to support the main research question, contributes to a more focused exploration of the contexts and mechanisms which leads to positive or negative implementation outcomes, and will determine the dominant outcome patterns in identified contexts. The process of title and abstract screening of selected documents is outlined more explicitly and the application of the CCAT as a guide to become more objective in assessing papers emphasised. Similarly, an explanation of the quality appraisal process to be undertaken by two authors comparing before agreement is reached to ensure rigour and the inclusion of only the highest quality papers and grey literature, is more notably laid out.
See the authors' detailed response to the review by Mary Ellen Glasgow
See the authors' detailed response to the review by Maura Dowling and Martina Giltenane
The delivery of care to any patient/resident/woman/child and family should be of high quality, consistent, safe and patient-centred. Nurses and midwives are at the centre of care delivery, delivering high quality and safe care to patients and their families1. The World Health Organisation’s (WHO) global strategic directions (2016–2020) provides a framework for the WHO and key stakeholders to create, implement and evaluate nursing and midwifery accomplishments to ensure acceptable, good quality, and safe nursing and midwifery interventions2. High quality care delivery is important both in Ireland and internationally and quality measures such as metrics play a part in helping to standardise care and ensure consistency in quality3. In many organisations, there is a wealth of data but often, there is no way to “collect, analyse and interpret data that will track the quality of care delivery”1. The WHO (2006), defined high quality of care as the extent to which health care services provided to individuals and patient populations improve desired health outcomes. In order to achieve this, health care must be safe, effective, timely, efficient, equitable and people-centred4.
In 2011, Nursing and Midwifery Planning & Development Units (NMPDU) developed and implemented quality care-metrics (QCP-Ms) in over 100 practice areas across the Republic of Ireland5. These QCP-Ms were endorsed by the Office of the Nursing & Midwifery Services Director (ONMSD) Health Service Executive1. QCP-Ms are “a measure of the nursing and midwifery clinical care processes, in healthcare settings in Ireland, aligned to evidenced-based standards and agreed through consensus”6. It is important to measure the degree to which nurses and midwives adhere to fundamental care processes to access and assure the quality of care delivered to patients and clients6. Quality measures are frequently classified into three types: structure, process and outcomes7. Structure reflect factors such as the availability of staff and facilities, process considers whether care interventions adhered to best practice guidance, and outcomes consider the changes because of care delivered. Further, nursing metrics are agreed standards and benchmarks8. According to the HSE, QCP-Ms are sensitive to the influence of nurses and midwives appropriate for all care settings, aligned to evidence-based standards and agreed through national consensus. Nursing Quality Care Process Metrics provide the framework to identify gaps in care delivery, enabling action planning for quality improvement and provide a mechanism by which care providers can be accountable for the quality of their care delivery5.
In June 2018, ONMSD completed phase one of work, which included a systematic review and a modified four-round Delphi study. Phase one identified important aspects of nursing and midwifery interventions/care processes that should be measured (Nursing & Midwifery-Sensitive Process Metrics)1. Implementing changes in practice are complex9 and some changes are more likely to be implemented than others10. QCP-Ms phase one culminated in the development and launch of seven research reports with defined suites of metrics for the following clinical practice areas – acute care, midwifery, children’s, public health nursing, older persons, mental health and intellectual disability nursing11. A total of 91 metrics were launched (acute care – 11 metrics; midwifery – 18 metrics; children’s – eight metrics; older person’s care – 19 metrics; public health nursing – 14 metrics; mental health nursing – nine metrics; and intellectual disability nursing – 12 metrics)11. A national procedural guideline document for each suite of QCP-M’s was also developed to guide implementation, measurement and support fidelity of interpretation11. The implementation of the QCP-Ms project into an individual service is structured within a framework consisting of four stages: “initiation, planning, implementation and mainstreaming”5. These stages are further subdivided into 15 individual steps; from step one where an invitation is sent to each service to step 15 where the QCP-Ms have been implemented within their service and the project is monitored, reviewed and evaluated5.
The rapid realist methodology aims to highlight the impact interactions have among contexts and what impact mechanisms have on intervention outcomes12. The basic question of a rapid realist review (RRR) is ‘what is it about this intervention that works in this context and why’?13. An RRR works on understanding what are the contexts (C), mechanisms (M), and outcomes (O) that enable or constrain the implementation of an intervention11. This RRR, as part of a larger evaluation, namely phase two, will look at how in re-+lation to the intervention of quality care process metrics in practice, using rapid realist methodology to focus on how interventions work across contexts; what worked, what did not work, for whom, why and in what circumstances.
Realist review seeks to develop a common understanding of underlying factors and causative mechanisms and, according to Pawson et al. (2005), seeks to understand the components of the social world and stratifications of social reality. In realist inquiry there is a focus not only on ‘what works’ but on ‘what works for whom, why it works, and in what circumstances’12,14. Realist review often involves analysis of existing data. Pawson and Tilly address the question ‘what works, for whom, in what circumstances, and how?’ within a range of interventions12. The assumption is that programmes are ‘theories incarnate’9, which means when a programme is designed, it is underpinned by one or more theories about what ‘might cause change’12,15. According to Pawson et al., (2005), the iterative, flexible nature of realist reviews do not align well with protocol-driven, standardised processes common to established systematic review methods. The steps undertaken in this study will be informed by an RRR methodology, will be iterative in nature and will be revisited iteratively throughout the RRR process.
RRR methodology was developed as a tool for applying a realist approach to a knowledge synthesis process, thereby producing a product that is useful to policy makers in responding to time-sensitive and/or emerging issues where there is limited time and resources13. When undertaking an RRR, theories within the programme are explicit about how, and for whom, to what extent, and in what contexts a programme might ‘work’15. Data collected should include: “programme impacts and the processes of programme implementation, the specific aspects of programme context that might impact on programme outcomes, and how these contexts shape the specific mechanisms that might be creating change”16. By testing context-mechanisms-outcomes (CMO) configurations, will enable us to understand how, why, for whom, and in what contexts the intervention, i.e., QCP-M’s are more or less likely to work and how it produces its desired and undesired outcomes14,15. Underpinned by realist evaluation methodology (12) rapid realist review (RRR) has emerged as a popular approach in health service research and is well suited to a national evaluation of implementing QCP-Ms in Nursing and Midwifery practice. Unlike a systematic review which will control context, a realist review embraces contextual complexity making it a very appropriate approach for studying healthcare implementation and quality improvement17. It focuses on understanding how an intervention (i.e., QCP-Ms) works (or not) within a particular context, taking account of individual behaviours and system influences12,18. A RRR will identify the enabling and constraining dynamics influencing implementation. RRR review involves analysis and interpretation of existing data, in essence, it is the application of the realist approach to retrospective literature reviews (Pawson, 2002). RRR acknowledges that theories cannot and do not always offer explanations or predict outcomes in every context; for example, in patient safety programmes19. However, RRR is an approach that suits situations where policymakers and practitioners require guidance to inform emergent decisions in practice. The ‘rapid’ review is deliberately intended to streamline evidence synthesis and provide practical outcome-based results in a short time frame. Using an expert panel, it directly engages policy makers and knowledge users in the process which ensures the review remains relevant to the practice context13,17. While this review directly informs QCP-Ms implementation in Nursing and Midwifery Practice in Ireland, it will also have transferrable learning for any other discipline or health system implementing similar improvement efforts.
What factors enable the successful implementation of a suite of quality care process nursing/midwifery metrics across all areas in nursing and midwifery practice?
Additional sub-research questions
In nursing and midwifery quality care process metrics, what contexts and mechanisms lead to positive implementation outcome?
In nursing and midwifery quality care process metrics, what contexts and mechanisms lead to negative implementation outcomes?
What were the dominant outcome patterns in identified contexts?
This study aims to conduct an RRR that involves a synthesis of the international literature (published and grey) that generates programme theories to determine a better understanding of enablers and constraining influences related to the implementation of nursing/midwifery quality care process metrics. This study will aim to unpack the mechanism of how complex interventions (QCP-Ms) work (or why they fail) in particular contexts and settings.
Formal ethical approval is not required for this RRR. This RRR will be undertaken over a three-month period commencing middle of January 2021. The RRR will be informed and reported according to RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) standards16. The review design is based on Weetman et al. (2017) study design using eight steps20. The design is summarised in Figure 1. A PRISMA-P checklist has been completed and is available as an additional file (see Extended data21).
Locating existing theories on what factors enable the successful implementation of a suite of QCP-Ms across nursing and midwifery practice is pertinent to step 1. We will conduct a preliminary background search in PubMed and EMBASE searching article titles, abstracts, keywords, and subject headings to guide the development of the second search strategy. Creating a good and clear search is vital to ensure the best possible and most relevant return of articles. We will use Boolean operators: AND, OR, NOT, to help define our search. Documents sourced within the scoping search will be reviewed and interrogated for theories related to what aids in the successful implementation of interventions such as QCP-Ms. The initial programme theory will be developed throughout the review process, but it will serve as a starting point for the RRR. From the findings of this scoping search and content expertise of the research team, an initial programme theory will be defined and refined throughout the RRR process. Once the initial programme theory has been developed, the searching phase will commence.
We will search electronic databases including Excerpta Medica Database (EMBASE), PubMed Central, The Cumulative Index to Nursing and Allied Health Literature Complete (CINAHL Complete), APA PsycINFO, Applied Social Sciences Index and Abstracts (ASSIA) and Cochrane Database of Systematic Reviews (CDSR) (see Extended data21) and grey literature will also be selected to include only publications within the last ten years (see Extended data21). A PICO framework will be used to structure the key words used in the search strategy22. ‘P’ in the PICO framework refers to the population, namely nurses and midwives. ‘I’ refers to an intervention, and this was the QCP-Ms. ‘C’ refers to the comparison or control group. ‘C’ can also refer to study characteristics, i.e., study design. ‘O’ refers to outcome and relates to the core research question: What factors enable the successful implementation of a suite of quality care process nursing/midwifery metrics across all areas in nursing and midwifery practice (Table 1)?
The search strategy will be developed and agreed by the research team. Following this, the eligibility criteria will be developed. Table 2 provides an overview of the eligibility criteria for this RRR. Independent reviewers will use the online software ‘Covidence’ to import the search results and begin the title and abstract screening process23. Following this, the titles and abstracts will be screened by two teams of two reviewers independently. Similarly, full text articles will be screened by two teams of two reviewers independently. (Table 2). An arbitrator from within the research team will be appointed to discuss any discrepancies that may have emerged. If there are any discrepancies regarding which articles to include or exclude and/or why, an independent arbitrator will be consulted to make the final decision (see Extended data21, for the PRISMA flow diagram template18 that will be completed, including all numbers finalised, at the end of the RRR).
There are many different critical appraisal tools outlined within the literature that enable the quality assessment of papers. These include The Crowe Critical Appraisal Tool (CCAT)24 or Critical Appraisal Skills Programme (CASP)25 checklists that include: Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case-Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule24. The CCAT critical appraisal tool will be utilised to ensure the highest quality papers and grey literature will be included in the review. More importantly, the CCAT is a guide to become more objective in assessing papers (personal correspondence, M. Crowe, September 24, 2020). Grey literature will not be excluded based on quality scores but will be reviewed by the local panel for inclusion or exclusion. Moreover, in terms of rigour, quality appraisal will be undertaken independently by two authors and then compared before agreement reached.
To ensure the most suitable information is extracted, a draft data extraction tool has been developed, based on RRR methodology15,26 (see Extended data21). This tool will be pre-tested before use to ensure it captures all relevant information accurately. Data will be extracted from the article and checked by a second member of the research team. When extracting data, if an article does not include information relevant to a question in the form, the researcher will record this as ‘not reported’. The data extraction process will populate the data extraction table with evidence. Once the research team and sub-teams extract the data, the content from each team’s data extraction tables will be amalgamated into one single data extraction table to include all articles.
This RRR will be undertaken in consultation with a local reference panel. The local reference panel will consist of eight key intervention stakeholders’, and knowledge users such as healthcare professionals who will further develop and implement the interventions. RRR is grounded in the local context, with explicit, extensive, iterative engagement with a local reference group comprising representatives of potential knowledge users12. The benefits of including key stakeholders and knowledge users in the process of a review include increased relevance, clarity and awareness of review findings27. In addition to the local reference panel, an expert panel will consist of researchers from two universities in the east and one in the west of Ireland. An expert panel with the guidance of a librarian will help to identify the relevant articles that should be included in the review and contribute to tailoring the search strategy, synthesis of findings and verifying appropriate interpretation of results. This expert panel will consist of researchers and practitioners who have experience in RRR methodology pertinent to nursing and midwifery healthcare settings and also have a connection or previous collaboration with the three universities involved with this review. As this review will be conducted in the midst of a healthcare pandemic, all contact with stakeholders and knowledge users will be conducted virtually through secure online communication platforms. Initially, the local reference panel and an expert panel will define the research questions, review the inclusion/exclusion and search strings of the RRR to ensure clarity and consistency. Following title and full-text screen by the team of researchers, the expert panel will meet the research team to review selected articles to ensure all relevant articles are included. A final consensus meeting will be convened with the expert panel and local reference panel once synthesis of the literature is complete, to highlight the different enabling and constraining contextual factors and mechanisms which influence the implementation of QCP-Ms across seven practice areas.
We will develop an approach to synthesis, including the following steps, as outlined by Rycroft-Malone et al. 201228.
1. Organisation of extracted data into evidence tables
2. Theming by individual reviewers
3. Comparison of reviewers’ themes for a specific article and formulation of chains of inference from the identified themes
4. Linking of the chains of inference, and tracking and linking of articles
5. Hypotheses formulation (mechanism, context, outcome chains)
The thematic analysis framework29 will be operationalised to analyse the findings from each selected paper within six non-linear planes; (1) familiarity with the information, (2) generate codes, search for themes, (3) review the themes, (4) define and (5) names the themes and (6) produce the findings. Selected papers will be imported into the software NVivo 2330. Results and discussion sections will be coded in order to identify context, mechanism, outcome configurations in the findings.
The final stage involves the refining and testing of the programme theory, following synthesis of the data. It is envisaged that the programme theory will explain how and why QCP-M’s produce outcomes from changes in contexts and their resultant interactions with mechanisms. The local reference and expert panel will assist in refining the final theory by providing us with their expertise in the field. After completing all steps in this review, any stage may be revisited in order to ensure data saturation and ‘theory saturation’21. Once the review has been deemed to have reached data and theory saturation, no further documents will be added, and steps will not be repeated.
Findings from this review will help to give insights into realism as a framework and how nursing and midwifery QCP-Ms have been implemented previously. Findings related to the barriers and facilitators to healthcare interventions/initiatives being evaluated could enable us to identify ways in which we can evaluate the bigger QCP-M’s project. This RRR will provide a nationally and internationally unique approach to measuring nurses and midwives’ contribution to quality and safe care, representing a departure from traditional methodologies and will provide a robust lens into phase two of the evaluation research project.
Figshare: Quality Care Metrics (QCP-Ms) in Nursing and Midwifery Care Processes: A Rapid Realist Review (RRR) Protocol, http://doi.org/10.6084/m9.figshare.1304033321.
This project contains the following extended data:
- PRISMA-P_Supp_A.docx (Supplementary Material A – PRISMA-P checklist)
- Supp_Mat_B.docx (Supplementary Material B - potential academic databases and grey literature databases)
- Supp_Mat_C.docx (Supplementary Material C – PRISMA flow diagram)
- Supplementary Material C (1) Data Extraction Tool.docx (Supplementary Material D – data extraction tool)
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Maura Dowling: qualitative methodologies; qualitative evidence synthesis; concept analysis. Martina Giltenane: qualitative methodologies, modified Delphi, quality improvement and measurement.
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Nursing education and leadership, quality improvement.
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Maura Dowling: qualitative methodologies; qualitative evidence synthesis; concept analysis. Martina Giltenane: qualitative methodologies, modified Delphi, quality improvement and measurement.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 2 (revision) 28 Jan 21 |
read | |
Version 1 17 Nov 20 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with HRB Open Research
Already registered? Sign in
Submission to HRB Open Research is open to all HRB grantholders or people working on a HRB-funded/co-funded grant on or since 1 January 2017. Sign up for information about developments, publishing and publications from HRB Open Research.
We'll keep you updated on any major new updates to HRB Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)