Understanding the Mechanisms that Operate within CHIME: A Realist Review Protocol

Realist review phases

Systematic and scoping reviews have a step-by-step systemic approach to the review of the literature^33–35. This is important for both review types so that they can enhance the reproducibility and transparency of how papers were searched for within the wider literature base^36,37. Realist reviews differ from systematic and scoping reviews as they do not conform to a prescribed method to review the literature²⁶. Instead, because of the realist’s focus on theory creation and explanation, the review process will be supported by a set of guiding principles rather than a prescribed structure^29,38. As such, for the purposes of this realist review protocol, an essential step in creating the protocol was identifying what methodology to follow. This protocol followed the format proposed by Masterson et al.³⁹ and McCormack et al.⁴⁰. The rationale for this is twofold: 1) both studies adhered more strictly to the RAMESES-I guidance and 2) both studies were situated within either the co-design space³⁹ or within an aspect of mental health service provision⁴⁰. McCormack and colleagues⁴⁰ in particular note that a realist review comprises of five generalised steps that were originally devised by Pawson and colleagues⁴¹ in their seminal work. These steps are as follows:

These steps are similar to those utilised in generic systematic reviews, but they are notably different in terms of theoretical depth required to complete each step⁴². Figure 2 illustrates the five major steps of the realist review process adapted from McCormack et al.⁴⁰. Additionally, a sixth step will be added to examine the ethics and dissemination of results, given that the present paper represents a protocol for a proposed realist review. Each step is described in detail below.

Figure 2. Figure depicts the five phases of Pawson and colleagues process of conducting realist reviews as adapted from the work of McCormack and colleagues.

Step one: setting up the review advisory panel and constructing the initial programme theories. Realist reviews are, by their very nature, iterative in their approach⁴³. This means that the refinement of the initial programme theories constructed here occurs throughout the review itself⁴⁴. In realist reviews, the first step is to identify a research focus and construct and reach an agreement regarding initial programme theories. As noted previously, for this realist review, our focus will be to explore how and why the mechanisms within the CHIME [Connectedness, Hope, Identity, Meaning and purpose and Empowerment] framework work in individuals who are in recovery from mental health challenges. Given that realist reviews are explicitly theory orientated along with an emphasis on “how” and “why” within the focus statement above, this review approach is suitable for exploring the mechanisms that bring a person from a place of unwellness to the characteristics of CHIME as noted by Leamy et al.,^17,32,45.

The next subphase in this first step is to break down this focus into a series of “If...then...” statements⁴⁶. If... then statements are simply explanatory accounts or statements that allow one to begin the process of programme theorising⁴⁷. As noted in Table One, each element of CHIME received its own if... then... statement. This is because each element of CHIME works independently and together to support an individual’s personal recovery journey.

Once the if... then... statements are created, they are presented to the team members [ÉNS, JPB, and TB] for review and refinement. Once these if... then... statements are finalised, the next sub-step is to create the initial programme theories⁴⁸. Initial programme theories are useful as they help reviewers understand how, for whom, why, and under what circumstances complex interventions such as CHIME work to achieve personal recovery in individuals with mental health challenges⁴⁹. Essentially, an initial programme theory comes from the if... then... statements and consists of three essential parts: context, mechanism, and outcome (CMO)⁵⁰. In a realist review CMOs, the context refers to the outside parameters of the formal programme architecture. In other words, it relates to the environment in which the mechanism occurs⁵¹. This can be in the backdrop of a historical event, or perhaps social and cultural norms⁵². However, it is a contextual interaction with the mechanism that produces the outcome. The mechanism is the underpinning generative force that, when interacting with the context, produces the outcome. It is subdivided into resources and how people respond to them. It is often hidden and extremely sensitive to variations in context⁵³. Finally, the outcome is the result of the mechanism’s interaction with the context. These outcomes are often behavioural and can be either intentional or unintentional⁵². In short, these components come together to be known as a CMO configuration or an initial programme theory. The premise behind a CMO configuration is that the context and mechanism lead to an outcome. This process involved multiple readings of the if...then... statements in order to clearly assign elements to the context, mechanism, and outcome. Once MJN was satisfied that these elements were aligned correctly, the CMO configurations were circulated to the supervisory team for initial analysis and refinement, resulting in an initial set of plausible programme theories, as presented in Table Two.

Once the initial CMO configurations are constructed, the next subphase is to present these CMO configurations to a group of experts for review. This is necessary to ensure that the theories we test the literature with are relevant to the situational context of modern mental health services, which can be determined by consulting experts in this area. From MJNs’ previous work experience, the expert panel was brought together and consisted of six individuals: three academics: Dr. Éidín Ní Shé, Dr. John Paul Byrne, Dr. Tina Bedenik, the Head of MHER, Michael Ryan, the Chair of the National Implementation and Monitoring Committee [NIMC], and former Advancing Recovery in Ireland [ARI] co-lead: Catherine Brogan and Involvement Centre Co-Ordinator and former Peer Educator: David Dwyer. These individuals were sent an invitation letter [Appendix A] containing the above CMO configurations, along with the meeting date and associated details. Due to the proximity of all parties to one another in the country, the expert panel convened online via MS Teams and lasted approximately an hour and a half. At this meeting, the entire PhD project was presented to the expert panel, with specific attention given to work package one: the realist review. As part of this meeting, each CMO configuration was presented individually, with adequate time given for discussion of all aspects of each CMO configuration under examination. As a result, CMO configurations were added and others were refined with the support of the expert panel. After the meeting, the original CMOs were amended, and a sixth CMO was added based on the discussion. Additionally, these were added to the minutes, which were prepared and circulated among the expert panel members [Appendix B], who were then given the appropriate amount of time to respond and make further amendments to the edited CMO configurations. The finalised and amended CMO configurations are presented in Table Three below.

Step two: searching for evidence. According to Pawson et al.⁴¹, the next step of the realist review process will be to test these CMO configurations to see if the literature agrees and aligns with these hypothesised initial programme theories. In order to do this, the first step is to create a search strategy⁵⁴. Search strategy development should be an iterative process that results in a strategy to search the literature that is both sensitive and specific to the topic under investigation⁵⁵. Achieving this can be challenging and often requires several hours to properly construct⁵⁶. As such, obtaining the advice and expertise provided by a librarian is seen as crucial to the process^57,58. In keeping with best practice, KW, an information specialist librarian from the Royal College of Surgeons in Ireland, was consulted in the search strategy development. As a result of the expertise of KWs, Table Four, Table Five and Table Six were constructed to contain the search terms that were devised to support the research team in gathering citations for this realist review from three databases: CINAHL, PubMed, and psychINFO. Grey literature will be gathered through expert panel members, but also by searching the grey literature source: Lenus – a repository used by staff of the Health Service Executive to store publications constructed by staff within the organisation as well as storing other organisation-specific reports and literary resources.

The next step was to construct a pre-determined inclusion and exclusion criteria [Table Seven]. A pre-determined inclusion/exclusion criteria is important, as it will be used to support the shortlisting of articles for this review. For this reason, each inclusion/exclusion criterion was selected for a reason. The rationale for this is now discussed below:

In the identification of appropriate resources for inclusion in this realist review, qualitative and mixed methods papers will be included, whereas quantitative papers will not. The rationale for this is that papers with qualitative data allow for theoretical depth to occur, which is not possible with other quantitative methodologies and positivist positionings. Literature reviews of any kind, including meta-synthesis, will be excluded because of the reviewers’ positionality towards the original raw data used in the original study¹. In other words, these review papers will be excluded as the reviewer is removed at least twice from the original raw data that was collected by the original study authors. As such, if reviews were included, it is possible that such reviews could misinterpret the original findings and make conclusions that are misleading due to the misinterpretation of the data presented by the original studies. Equally, the studies that report on the raw data included in such reviews may also be subject to bias or may not have been of sufficient quality, but yet included in such reviews due to, for example, a lack of papers identified for inclusion. As such, the findings of the review may not be accurate because of the inclusion of poorly conducted studies, which may impede the intended impact of the realist review. The articles to be collected will be from 2011 to the present [2025]. Articles published before 2011 will not be included as they would have been published before the literature review that created CHIME was published. Articles not published in English were excluded due to the inability of the review team in translating reviews from other languages to English. Articles will be included only if they are relevant to mental health and illness. Articles from a dual diagnosis of any kind will be excluded because of the difficulties in distinguishing changes relating and not relating to mental health and illness alone. Finally, papers were also excluded if they spoke of an aspect of CHIME on their own and included if they spoke of all concepts of CHIME amalgamated together. Articles that examined other areas of mental health recovery were excluded from the review.

Once the inclusion/exclusion criteria are defined, the next step is to search the literature to determine whether these initial programme theories are credible or not. The planned search will use an exhaustive screening process that describes a systematic process whereby all citations that arise from applying the search strategy to databases will be included as part of round one screening. As part of the creation of this realist review protocol, the search strings were applied to the relevant databases, and all citations that arose were then placed into a citation software package Covidence to help manage the remainder of the screening process, which will be discussed in further detail as part of step three of the realist review process devised by Pawson et al.⁴¹. Round one of the exhaustive screening process will be undertaken by both MJN and Information Specialist Librarian KW.

Step three: select and appraise evidence. After round one of exhaustive searching concludes, a number of citations will have been found and added to the citation software package Covidence for the review team to review and narrow down further. Round two screening then occurs. Round two screening involves firstly eliminating any duplicates that may have been gathered as part of round one screening and then applying pre-determined inclusion/exclusion criteria [Table Four] to the abstracts of the remainder of the citations gathered over the course of round one screening. Round two screening will be undertaken by MJN and ÉNS, who will screen 10% of the papers at this phase to ensure rigor and adherence to the pre-determined inclusion/exclusion criteria (Jones et al., 2024). Finally, round three screening will involve the application of a pre-determined inclusion/exclusion criteria [Table Four] to the full text of citations included in this study. Again, this round was conducted by MJN with ÉNS, reviewing 10% of papers at this stage of searching also. Any citations left after the three rounds of screening were included in this realist review. As such, the process described above is similar to that of an original systematic review with a particular focus on thoroughness and transparency⁵⁹.

This realist review will appraise included studies for richness, relevance, and rigor in line with the realist methodology of Pawson et al.⁴¹. According to Pawson et al.,⁴¹ relevance relates to whether the paper addresses the theory under investigation. In addition, rigor relates to how the research supports the conclusions drawn by the author of the paper being examined⁴¹. Finally, richness refers to whether a paper has sufficient depth to meaningfully contribute to theory-building²⁹. Unlike systematic reviews that appraise for quality purposes^60–62, in realist reviews, one appraises relevance as the development of theory²⁹. However, there is no agreed upon method for assessing relevance in realist reviews⁶³. As such, in this review, when appraising included studies, the following questions will be posed to the literature:

Step four: extract the data. Data will be gathered only from primary sources because of the issue of positionality when it comes to secondary sources, as noted by Norton¹. When it comes to primary sources, each citation will be examined thoroughly to identify the key aspects of each study, including the CMO configuration. In general, the introduction of citations will identify the context, the results of these same citations will identify the outcome, and the discussion of the text will help identify the mechanism. Data collected from the citations that match these areas will be extracted by MJN into a data extraction form that will be developed by MJN prior to the commencement of this stage of Pawson and colleagues’ realist methodology. This data extraction document will be validated through consultation with the supervisory members of the review team (ÉNS, JPB, and TB). Once approved, this data extraction tool will document the following key aspects from each citation:

The process of data extraction is iterative, meaning that the data extraction tool will be constantly refined over the course of the realist review process²⁶. In addition, relevance will be captured using the Crowe Critical Appraisal Tool⁶⁷. It was decided to utilise this tool as it has been utilised in another realist review within the health sciences space⁶⁸.

Step five: analyse and synthesise data. The primary purpose of data synthesis is to determine what works, for whom, how, and under what circumstances, all in an attempt to refine the initial programme theories into middle-range theories⁴². In this way, the word synthesis in this context differs from the same word for systematic reviews, as in this process, the purpose of synthesis is to make progress in the explanation of initial programme theories into middle-range theories⁶⁹. In this realist review, two mechanisms are used to refine the initial programme theory. Firstly, summative content analysis will be conducted. Summative content analysis was devised by Hsieh and Shannon⁷⁰, and it involves an iterative process of identifying keywords based on the stated headings in stage four of Pawson and colleagues’⁶⁹ realist methodology to create a set of middle-range programme theories. Once these middle-range programme theories are defined, the second process is a validation exercise where our subject experts appointed during stage one of the realist methodology will reconvene to approve the newly developed middle-range programme theories.

Step six: ethics and dissemination. In line with similar realist reviews, ethical approval will not be required, as although we have an expert panel consisting of academics [ÉNS, JPB, TB] and expert panel members [MR, CB, DD], they have all been included in the authorship of this paper. In addition, no ethical approval is required, as no primary data collection will take place for the purpose of this review. They will also be included in the authorship of the realist review itself. The complete review will be presented as a second article and submitted for publication in a peer-reviewed journal. In addition, the results of this proposed realist review may also be presented through other means, including conferences and recovery workshops delivered by recovery college personnel. This will be made possible through the Recovery and Engagement Programme Manager in charge of the recovery education workstream of MHER.