What and how do different stakeholders contribute to intervention development? A mixed methods study.

Design

This SWAT is a mixed method study using an explanatory sequential design¹⁶. Quantitative data (self-completion participant survey) were collected and analysed first, followed by the qualitative data (consensus group meetings) which were collected and analysed second in sequence¹⁷. The quantitative results provided an overview of participant ratings of acceptable and feasible intervention content, while the qualitative analysis allowed for further exploration of why participants rated intervention content the way they did by using a topic guide informed by survey findings.

Recruitment

People with diabetes

People with diabetes were recruited using an information flyer developed by the research team including a graphic designer (http://doi.org/10.5281/zenodo.4321202). The flyer was distributed using a range of recruitment strategies including social marketing recruitment, community outreach recruitment, health system recruitment, and partnering with other organisations. All individuals who contacted the study team and returned a short demographic survey (Supplementary File 1 in the Extended data¹⁸) were randomly assigned (using an online random number generator) to either the meeting for the people with diabetes only, or the combined meeting.

Health care professionals

HCPs were recruited through local professional networks known to the study team. An email invitation was sent to 50 HCPs (practice nurses, diabetes nurse specialists, general practitioners, and specialist physicians). All HCPs were allocated based on their availability to the HCP-only meeting or combined meeting. Further details on the recruitment process have been described in detail elsewhere¹³.

Data collection

Quantitative phase

Before each consensus meeting, participants were sent an evidence summary of barriers to and enablers of attendance at diabetic retinopathy screening, and interventions to address non-attendance (Supplementary File 2 in the Extended data¹⁸), and a self-completion survey (Supplementary File 3 in the Extended data¹⁸). The evidence summary and survey were designed with input from the Irish National Adult Literacy Agency and a Patient and Public Involvement (PPI) group and revised based on their feedback.

The survey outlined 39 proposals for intervention content that were grouped at the practice-level (‘ways to encourage the practice staff to make sure person attends’) and patient-level (‘ways to encourage the person to attend diabetes eye screening’). The proposals contained operationalised behaviour change techniques (BCTs), defined as an “observable, replicable, and irreducible components of an intervention” that have the potential to change behaviour¹⁹. The proposals (operationalised techniques) were short statements/descriptions of how the selected BCT would be put into practice²⁰, in line with the study focus on increasing diabetic retinopathy screening uptake. The BCTs in the survey were selected to address known barriers to and enablers of screening attendance based on previous formative research conducted by the IDEAs research team¹⁵ and existing evidence of their effectiveness either in interventions to increase retinopathy screening attendance or interventions in other settings^21,22. A total of 24 unique BCTs were operationalised across the 39 intervention proposals in the survey. Further details on these 24 BCTs has been provided in Supplementary File 4 in the Extended data¹⁸.

In the survey, participants were asked to rate the acceptability and feasibility of each proposal. All items were rated on a Likert response scale ranging from 1 to 5 (from ‘strongly disagree’ to ‘strongly agree’) with higher scores indicating greater acceptability or feasibility. These survey questions were adapted from existing measures developed by Weiner et al. to rate implementation acceptability and feasibility²³. Acceptability was defined as the perception among end-users that the intervention proposal was agreeable or satisfactory. Feasibility was defined as the extent to which the intervention proposal could be successfully implemented in general practice. People with diabetes received a paper format of the survey while HCPs received an electronic format.

Qualitative phase

Following completion of the surveys, participants took part in one of three consensus group meetings. Each meeting was held for two hours in University College Cork and was facilitated by the same facilitator experienced in consensus group techniques/processes. This facilitator was a male professor of health services research who held no relationship with participants.

During the meetings, a summary of the ratings of acceptability/feasibility was presented to participants. This was followed by a series of small group discussions (facilitated by members of the research team) where participants were asked to discuss how each intervention proposal would work in practice (See Supplementary File 5 in the Extended data¹⁸ for Facilitator Guide). Facilitators asked participants to discuss and give feedback on both practice-level and patient-level proposals. Prompts about patient-level proposals included 1) who should deliver the message to remind patients to attend diabetes eye screening? 2) how should the message be delivered? 3) when should the message be delivered? and 4) what should the message contain? Participants were asked to focus their discussion on proposals where the consensus on acceptability and feasibility based on the survey was unclear. However, given the semi-structured nature of the meetings, participants also made new suggestions. The small group discussions and the feedback to the larger group were digitally audio recorded with participant consent.

Data analysis

Participant survey responses were entered into SPSS software (version 26, RRID:SCR_016479) and analysed using descriptive statistics. Consensus meeting transcripts were analysed using NVivo 12 software (RRID:SCR_014802). If this software were unavailable, it would be possible to conduct the analysis using Excel and Word.

Comparing end-users’ feedback on the acceptability and feasibility of intervention content

To examine participants’ ratings of the acceptability and feasibility of intervention proposals, the five-point Likert scale used in the survey was collapsed into three categories: ‘disagree’ [1 strongly disagree, 2 disagree], ‘neither disagree or agree’ [3] and ‘agree’ [4 agree, 5 strongly agree]. Contingency tables were generated for each intervention proposal by participant type (HCP or people with diabetes) and Fisher’s exact test was used as appropriate²⁴. Results were examined to identify proposals which had a difference (≥10%) between the proportion of HCPs and people with diabetes who agreed that intervention proposal was feasible and/or acceptable.

Guided by the survey results, interview transcripts were analysed using deductive content analysis. A codebook (developed a priori by LOM) designed to mirror the self-completion survey to identify and code feedback on specific proposals was used. Participants in the combined meeting were asked to reach group consensus on intervention proposals, therefore it was difficult to attribute feedback exclusively to people with diabetes or HCPs or both. Therefore, the people with diabetes only meeting and the HCP only meeting were analysed before the combined meeting was analysed, to allow the researchers to see whether feedback from the combined meeting echoed that of the people with diabetes only and HCP only meetings.

To compare participants’ feedback on the acceptability and feasibility of intervention proposals, thematic analysis was performed by LOM, guided by joint displays of the survey results and qualitative coding. The joint displays were examined for recurring patterns between survey ratings and discussion during the consensus meetings, to identify reasons for agreement/disagreement e.g., what was or was not acceptable/feasible to whom, and why. An overview of this sequence of mixed methods is provided in Figure 1.

Figure 1. Overarching sequence of mixed methods.

Comparing end users’ suggestions for new intervention content

To identify and compare end-users’ suggestions for new intervention content, two researchers (ER and FR) conducted a deductive content analysis²⁵ to identify suggested changes to proposed intervention content and suggestions of additional intervention content. Both researchers read the consensus meeting transcripts multiple times (data familiarisation) and then independently extracted all suggestions made by participants in relation to intervention content and mode of delivery. A suggestion was defined prior to data analysis as any suggestion about intervention content or mode of delivery proposed by a member of the group, at any stage during the meeting, that was agreed with by one or more other members of the group. Agreement or disagreement between participants was ascertained based on explicit verbal expression or sounds or noises which conveyed their agreement or disagreement (e.g., mmm). The two researchers met to discuss the suggestions they had extracted. Any differences were discussed, and agreement was reached by consensus on the list of suggestions put forward by participants. Each new suggestion was then coded (yes/no) according to whether it would be feasible to incorporate into the intervention to be delivered. The scope of the intervention was defined as:

To identify how each new suggestion aligned with existing behavioural change techniques, they were mapped to Behaviour Change Technique Taxonomy (BCTTv1)²⁶. Further information on how this mapping was conducted is provided in Supplementary File 6 in the Extended data¹⁸.

Comparing end users’ contributions to the final intervention

Using deductive content analysis, one researcher (ER) categorised (yes/no) all recommendations (including feedback on proposals and suggestions for new intervention content) according to whether they were incorporated into the final intervention. Full details about the decision process regarding the final intervention content has been published elsewhere¹⁵. The final decision on the intervention content was made by a subgroup of the IDEAs study research team and a GP collaborator, basing decisions on the APEASE (affordability, practicability, effectiveness, acceptability, side effects, equity, sustainability) criteria. Practicality and acceptability criteria were populated based on findings from the rating survey and the discussions during the consensus meetings. The effectiveness criterion was based on a rapid evidence review of different approaches to improve screening uptake. Remaining criteria (affordability, equity, side-effects (unintended consequences), sustainability) were based on group discussions about what was feasible, bearing in mind previous formative research with patients and healthcare professionals and organisational factors relating to the primary care environment.

Patient and Public Involvement (PPI)

A PPI contributor (GF) was involved in the SWAT from the outset. GF is a person with diabetes, previously known to the lead author (ER). She contributed to the initial discussions about the study which ultimately informed the SWAT grant application, reviewed the grant application prior to submission and made changes to its content including the addition of disseminating the research amongst people with diabetes. GF was also involved in the development of materials used to recruit people with diabetes and assisted the research team with recruitment by posting recruitment flyers online via social media networks. She contributed to and reviewed each draft of this manuscript and is a co-author on this publication. The IDEAs study worked with a dedicated PPI group throughout the duration of the trial. Before the IDEAs PPI group was established, the lead author worked with a separate primary care research PPI group to develop and refine the materials that were sent to participants prior to the consensus meeting.

Ethical approval

The study received ethical approval from the Social Research Ethics Committee (SREC) at University College Cork (Log number 2018-122, approval received 13/08/2018). Written informed consent was obtained from all participants prior to completing the rating survey and taking part in the consensus meetings.

Comparing end users’ feedback on the acceptability and feasibility of intervention content

In total, 30 participants (13 people with diabetes and 17 HCPs) completed and returned the surveys. Missingness within the data ranged from 3.3% to 6.7%, depending on the survey proposal. There was incomplete data for 6 participants (4 people with diabetes, 2 HCPs). Table 1 presents the 31 proposals which had differences (≥10%) between the proportion of HCPs and people with diabetes who agreed the proposal was acceptable and/or feasible¹⁸.

Concerns about intervention content

Following integration of the survey results and qualitative feedback from the consensus meetings, themes related to the preference for and several main concerns about acceptable and feasible intervention content (Figure 2). Table 1 presents where these relate to intervention proposals and whether it was a joint concern, or preference, of both people with diabetes and HCPs, HCPs only or people with diabetes only.

Figure 2. Concerns about intervention content organised by health care provider (HCP) concerns, joint concerns, or people with diabetes’ concerns.

The results are organised according to the joint preference, joint concerns, HCP concerns and people with diabetes’ concerns. Examples of intervention proposals that relate to each area of concern are presented, along with the survey results and a short summary of participants’ feedback from the consensus groups.

Joint preference

Participants in all three meetings considered several intervention proposals to be acceptable and feasible because they believed some people with diabetes have a limited understanding of the screening process. In the survey, both people with diabetes and HCPs agreed the proposal to use someone in the practice who would explain the difference between routine eye checks and the screening test was acceptable (92.3% vs. 94.1%, respectively), though they differed in agreement with feasibility (75% vs. 94.1%, respectively). Data from the meetings provided no indication as to why people with diabetes rated feasibility lower than HCPs, however both groups flagged that there is confusion among some people with diabetes about the difference between routine eye tests and retinal screening. Participants in the combined meeting agreed that messages delivered to patients should outline the difference between routine eye tests and retinal screening and emphasise that damage can be asymptomatic to dispel the “false sense of security”. Similarly, participants in the people with diabetes only meeting thought messages should aim to increase patient understanding of the screening process. For example, highlighting the possible consequences of non-attendance and “alert you (people with diabetes) to the dangers involved”. Participants in the HCP only meeting agreed messages should emphasize that screening is free.

In the survey, less people with diabetes than HCPs agreed the proposal to arrange practical support was acceptable (66.7% vs 82.4% respectively), though less HCPs agreed it was feasible (58.3% vs HCPs 41.2%). Participants in the people with diabetes only meeting felt many people with diabetes are not aware of the need to organise transportation for after the screening procedure, and so messages should tell people they would need support rather than arranging it for them. HCPs had concerns about the feasibility of this proposal, which are discussed below under the concern straying outside their area of responsibility.

Joint concerns

Some HCPs and people with diabetes had concerns about proposals which might rely on active participation from people with diabetes, for example, the proposal for the person with diabetes to tick off a checklist when they have consented to/attended to screening. In the survey, a larger proportion of people with diabetes than HCPs agreed providing a checklist would be acceptable (69.5% and 35.3%, respectively) and feasible (75% and 35.3%, respectively). In the people with diabetes only and combined meeting, some people with diabetes felt having a checklist would help people be “proactive” in the management of their diabetes, while others thought that this would put too much responsibility on the person who “might lose or forget it”. Some of those in the HCP only meeting thought that only motivated and engaged patients would use the checklist.

Participants from all three meetings were concerned about achieving the balance between communicating the risks of diabetic retinopathy while not scaring people when informing them about screening. This concern related to several proposals to use other people with diabetes or HCPs to deliver messages. In the survey, both HCPs and people with diabetes agreed it would be acceptable to use a message from someone who has retinopathy and tells them it is important to go to screening before it is too late, there may be no symptoms and everyone with diabetes is at risk. However, 83.3% of people with diabetes agreed it would be feasible compared to 64.7% of HCPs. Participants across all meetings believed that “scaremongering” or “shock tactics” would not encourage people to attend. Rather than “shock” people, messages should inform them of the “truth” about the possible consequences of non-attendance and be provided “by the right person, in the right way”. Both people with diabetes and HCPs agreed that the same message (tells them it is important to go to screening before it is too late, there may be no symptoms and everyone with diabetes is at risk) when delivered by HCP rather than another person with diabetes would be acceptable (100% and 88.2%, respectively) and feasible (91.7% and 82.3%, respectively). Participants in the people with diabetes only meeting thought the GP would be the best person to deliver a message to attend screening as people “trust” their GP and are “much more inclined to listen to them”. HCPs in the HCP only and combined meeting had concerns that that delivering these messages during consultations would take a considerable amount of time.

Health care professionals’ concerns

Concerns about the resource implications of delivering intervention proposals including time, staff, and money, were raised throughout all three meetings. Resource concerns were often a reason for HCPs’ lesser agreement with proposals, especially those which aimed to encourage practice staff to ensure the person attends. Few people with diabetes and HCPs thought the proposal to provide a new resource to the practice (e.g., researcher checks if person registered, consented and/or attended) was feasible (53.8% vs 58.8%, respectively). While both agreed the proposal to prompt practice to check the (DRS) register during consultation and register person if necessary was acceptable, a slightly lower proportion of HCPs thought it was feasible (82.4% and 70.6%, respectively). They emphasized not having time for multiple prompts and reminders like letters or emails; “we absolutely don't have the time. We can't take anything on, it’s just beyond unbelievable.”

HCPs in both meetings had concerns about proposals to tell practices about the benefits/consequences of their patients attending/not attending. This was reflected in the different proportions who agreed such was feasible (people with diabetes 90.9% vs HCP 70.6%, respectively). Some HCPs believed financial incentives might be best to motivate GPs to ensure their patients are registered and attend DRS. HCPs in the combined meeting suggested that once practices have a registration uptake at a particular level, they could receive financial remuneration and therefore be “incentivised to do it (register patients)”. There were also concerns about using feedback to motivate HCPs to encourage patients to attend, namely by providing practices with comparison numbers (% people attending in other practices/ nationally). This discussion arose around the proposal to give feedback on national or international uptake or targets. Some participants in the HCP only meeting felt “you would totally tap into [competitive] personalities” but there was a lack of consensus on this proposal in the combined meeting. Some participants in this meeting thought a comparator could be a useful motivator, whereas one GP noted that the differing demographic of patients across practices would make comparisons difficult. HCPs in both meetings argued that feedback needs to be specific and tailored to their practice and their patients, as national averages and practice comparisons are “totally useless” as they “cannot address that on a one-to-one level with a patient”.

As previously mentioned, the proposal to arrange practical support like transportation was not considered feasible by people with diabetes nor HCPs (58.3% vs 41.2%, respectively). HCPs in the HCP only and combined meeting felt this proposal strayed outside of their area of responsibility, as they mostly interpreted it as having to arrange the transportation for the patient themselves, something they felt was “not their (HCP) problem” as patients “need to take ownership and responsibility”.

People with diabetes’ concerns

Participants in the people with diabetes only meeting were concerned that some proposals threatened their privacy. For example, arranging practical or social support would make it difficult for those who wish to keep their diabetes private to do so. Both people with diabetes and HCPs thought the proposal to provide a new resource to the practice like a researcher was not feasible (53.8% and 58.8%, respectively). A few participants in the people with diabetes only meeting were concerned about privacy should someone within the practice other than their GP/PN have access to their information. Contrastingly, more people with diabetes than HCPs thought this proposal would be acceptable (83.3% vs 64.7% respectively). However, this may be explained by HCP concerns about resourcing this proposal.

Comparing stakeholders’ suggestions for new intervention content

Participants in the people with diabetes only meeting made 26 suggestions for new intervention content, of which 7 were deemed feasible to incorporate into the final intervention (30%). Participants in the combined meeting also made 26 new suggestions, of which 3 were feasible (15%). Participants in the HCP only meeting made 32 new suggestions, of which 7 were feasible (22%). Table 2 shows the suggestions for new intervention content that were deemed feasible to incorporate. New suggestions were deemed unfeasible to incorporate into the intervention if they could not be implemented in the Irish general practice setting. For example, participants in all three meetings suggested that the reminder message should be delivered by professionals outside general practice, that the national screening programme could modify their processes to make it easier for people with diabetes to register and attend the service, and that national-level changes (e.g., media campaign to improve attendance, establishing a national diabetes register) should be introduced to increase screening attendance.

New suggestions deemed feasible to incorporate into the intervention mapped to 12 BCTs in the taxonomy (Table 2). There were four additional BCTs identified in the new suggestions that were not present in the intervention proposals outlined in the survey: goal setting (outcome), restructuring the physical environment, material incentive (behaviour) and punishment. Additional information on the BCTs identified is provided in Supplementary File 7 in the Extended data¹⁸.

Comparing end users’ contributions to the final intervention

The final intervention included a practice briefing, audit and feedback with technical support, practice-endorsed reminders (delivered in person, by phone and letter) and an information leaflet targeting key attitudinal and knowledge barriers. The people with diabetes only meeting had 23/51 (45%) recommendations incorporated into the final intervention, of these 20 were feedback on the intervention proposals and three were new suggestions. The combined meeting had 19/49 (39%) recommendations incorporated into the final intervention, of these 17 were proposed and two were new suggestions. The HCP only meeting had 24/55 (44%) recommendations incorporated into the final intervention, of these 21 were proposed and three were new suggestions. Table 2 shows the new suggestions that were incorporated into the final intervention. All three meetings made new suggestions that were deemed feasible but not incorporated into the final intervention. These suggestions, along with the reasons for exclusion (based on the APEASE criteria), are outlined in Supplementary File 8 in the Extended data¹⁸.

Summary of main findings and links to existing literature

Although there is growing awareness in the literature that involving different intervention end-users in the development process may have a different impact on the final intervention developed^8,11,27, to our knowledge, this is the first study to examine and compare in detail the contributions of different intervention end-users as part of a consensus approach to inform intervention development.

There were three main findings. Firstly, people with diabetes and HCPs had both shared and unique opinions about the acceptability and feasibility of some aspects of the proposed intervention content. Some opinions were shared by both end-users and were echoed throughout all three consensus meetings, for example that there is a limited understanding of the screening process, or that we should balance informing people without scaring them when communicating about screening. However, HCPs also had unique concerns related to their role as healthcare providers, while people with diabetes had their own concerns about intervention proposals which might risk their privacy. Such differences suggest that while there is a common ground when it comes to preferences for and concerns about intervention content, there are some aspects of the intervention which may be a greater priority for different end-users. Secondly, participants in all three meetings made suggestions for new intervention content which mapped to BCTs that were not present in the proposed intervention content however, participants in the combined meeting made less feasible suggestions as they could not be implemented in the Irish general practice setting. Finally, participants in all three meetings made recommendations that were incorporated into the final intervention. However, participants in the combined meeting had fewer recommendations incorporated than the other two meetings.

In the meetings involving people with diabetes only and HCPs only, respective groups had different opinions about the delivery of messages to attend screening e.g., who should deliver the message, when the message should be delivered, and what the message should contain. Those in the meeting of people with diabetes only tended to base their recommendations on what would be most acceptable to the person with diabetes. In contrast, participants in the HCP only meeting focused more on what was feasible from a resource perspective. These concerns are consistent with reports of increased workload and staff burnout in Irish general practice^28,29. In addition, some HCPs perceived that certain intervention proposals would involve straying outside their area of responsibility. They tended to disagree with proposals which they equated to an extra job or responsibility, understandable given the increasing responsibilities in general practice for chronic disease management³⁰. Future intervention developers should consider these different perspectives of respective end-users so that they may involve them in the development process in the most effective way.

On the other hand, participants in this study also had joint preferences for intervention content. Both HCPs and people with diabetes were conscious that while it was important to outline the seriousness of retinopathy, there is a need to strike a balance between informing but not scaring people about the screening process and potential disease consequences from non-attendance. This aligns with the body of literature on the use, or avoidance, of fear appeals to encourage preventative health behaviours, evidence which has demonstrated that providing information about possible negative consequences may prompt defensive responses³¹. For instance, one US study found that avoidance of cancer risk information was associated with lower participation in colorectal cancer screening³². During the consensus meetings, people with diabetes and HCPs had concerns about intervention content which might scare or frighten people, such as having a message delivered by someone who is visually impaired or prompting the person to feel regret. Intervention developers should select behaviour change techniques that promote adaptive, rather than maladaptive behaviour, as suggested by a qualitative study of fear appeals as a method in behaviour change interventions³³. These joint contributions by participants in our study offer a useful perspective to intervention developers about how end-users will receive communication, but also demonstrates there are instances where end-users can share priorities for intervention content.

Our findings indicate that end-user groups’ contributions to the intervention development process can differ based on whether they are involved separately or simultaneously. Participants in the combined meeting of people with diabetes and HCPs made fewer feasible suggestions for new intervention content and fewer recommendations from this meeting were incorporated into the final intervention. This suggests their contributions may have been influenced by group composition. Our previous analysis of participants’ experiences of taking part in the consensus meetings found that, although members of the combined meeting appeared to work together, during follow-up data collection both end-user groups held different views about what intervention proposals would and would not work¹³. Both HCPs and people with diabetes that participated in the combined meeting were uncomfortable with asserting what the other end-user group should or should not do. To fill this void, participants went off task and made suggestions that outside the scope of an intervention intended for primary care¹³. This current study alongside our previous analysis suggests that it may be useful to involve each end-user group, those who will deliver the intervention and the intended target population, separately rather than simultaneously in a consensus process to inform intervention development. When involving different end-users together in a consensus process, researchers should also consider facilitating these groups differently, paying special attention to acknowledge potentially unique views while also reaching consensus. Previous research has recognised the potential complexity of multi-stakeholder involvement, highlighting the need to manage group interactions, potential power imbalances and synthesising the views of different groups³⁴. One approach which might have been useful in the context of our research and could be relevant to future work in this field, would be hold the separate stakeholder groups first to allow for independent discussion and feedback, followed by a combined group in which consolidated feedback may be compared and discussed.

Strengths and Limitations

This study has several strengths including the use of a mixed methods, explanatory sequential design. Consensus meeting data supported the quantitative analysis by providing explanations, where available, for different participant ratings provided in the survey. By integrating the two, we aimed to draw out new findings beyond the information gained from the separate results³⁵. Fetters et al. have reported that such qualitative methods are often applied in order to explore reasons why a phenomenon occurs or to describe the nature of an individual's experience¹⁷. The involvement of PPI contributors is a further strength of this research. A PPI partner (GF) was involved in the SWAT throughout the duration of the study and is a named co-author on this publication. A separate PPI group were involved in the development of the materials sent to participants prior to the consensus meetings. Supplementary files 3.2. and 3.3 in the Extended data¹⁸ show how the study invitation letter and survey were improved as result of PPI feedback. These improvements helped to ensure that materials were more accessible and acceptable to participants.

This study includes a number of limitations. Firstly, as this was a SWAT, the primary aim of the consensus meetings was to review and discuss proposals for intervention content for the host trial and not to explicitly compare end-user contributions¹⁵. While the semi-structured approach of the meetings allowed participants to discuss proposed intervention content and generate new ideas for such content, it made it difficult to compare end-user contributions as the content and nature of the discussions varied across meetings. For example, some groups did not discuss certain survey ratings and intervention proposals, and some groups discussed particular proposals in more detail than others. This meant that explanations for survey ratings are not present in qualitative form consistently for all intervention proposals. Adopting a more structured approach, for example the nominal group technique or Delphi method³⁶, during the consensus meetings may have made it easier to compare views on all proposals across groups.

An additional limitation is the absence of some key end-users from the consensus meetings. There were no people with type 2 diabetes available to participate in the combined meeting. Despite using a range of strategies to recruit a representative sample of people with diabetes, we encountered issues with participant availability when arranging the combined meeting. Existing research has established that people with type 1 and type 2 diabetes have different experiences when managing their condition and engaging with HCPs and health services^37–39. Therefore, the involvement of people with type 2 diabetes in the combined meeting could have potentially changed the nature of the discussion and led to different recommendations. There was also a lack of involvement of practice administrators in the consensus meeting. Participants in the HCP only meeting suggested that practice administrators would be best placed to deliver the intervention. Involving them in the consensus meetings may have led to different recommendations as they play a key role in undertaking clerical duties to support delivery of care, and as gatekeepers, help to preserve boundaries of organisation and controlling access to the practice⁴⁰. However, the literature finds they are often overlooked by policymakers, undervalued by GPs and patients and excluded from research⁴⁰. Future research in general practice should consider involving practice administrators to ensure that all user voices are heard.

A final limitation was the lack of capture of non-verbal cues such as when participants nod in agreement or disagreement. As this SWAT looked to examine and compare agreement with proposed intervention content, such non-verbal data may have been useful. While non-verbal cues can offer rich data⁴¹ and we may have been able to capture this through video recording of the meeting, it has also been found that the use of video-recording equipment during focus groups can inhibit participants’ interaction⁴².

Implications

The results of this SWAT informed the development of the IDEAs intervention which has been tested as part of a pilot cluster randomised trial with a view to progressing to a definitive trial¹⁴. Involving end-users in decisions about planning and conducting health research, policy and services is gaining increasing momentum and as such, PPI is now required by many health research funders, journals, and research ethics committees^43,44. However, evidence on the impact of PPI is largely based on anecdotal reflections from researchers and members of the public which are descriptive and selective⁴⁵. Numerous studies have called for planned and methodologically rigorous research to evaluate the impact of PPI on the research process^46–48. In this study, the lines between research participation and involvement were blurred, as is often the case with PPI⁴⁹. Although people with diabetes were research participants in the consensus meetings, their role in the consensus meeting was to discuss and make decisions about the intervention content and mode of delivery which could be viewed as PPI^50,51. This SWAT provides evidence on the contribution of different end-users to the intervention development process and how different end-users can have different priorities for intervention content.

The results of this study, coupled with the results of our analysis of participants’ experiences of taking part in the three separate meetings to inform intervention development¹³, suggest that it may potentially be more acceptable and useful to involve patients/members of the public and HCPs separately when conducting PPI activities. When involving stakeholders together in PPI activities, alternative approaches to facilitation may need to be considered. Furthermore, as the process and impact of PPI is heavily dependent on the context in which it is being conducted, further research exploring the experiences and contributions of different end-users is needed, including an exploration of different facilitation models. This would enable all individuals interested in involving patients and members of the public in health research, policy, planning and development of health care to design and conduct more appropriate and effective user involvement^8,52.

Underlying data

The consensus meeting data are not publicly available due to limitations based on the ethical approval received and participant consent. Participants of the consensus process were not asked for their consent to store their data in a public repository. Participants consented to their anonymised data being made available for further collaborative research purposes outside of the current study upon reasonable request from the corresponding author and provision of a written proposal to the Principal Investigator (Dr Sheena McHugh, S.McHugh@ucc.ie).

Open Science Framework: What and how do different stakeholders contribute to intervention development? A mixed methods study. https://doi.org/10.17605/OSF.IO/NJS9Y¹⁸.

The project contains the following underlying data:

Data are available under the terms of the Creative Commons CC0 1.0 Universal (CC0 1.0) Public Domain Dedication License.

Extended data

Open Science Framework: What and how do different stakeholders contribute to intervention development? A mixed methods study. https://doi.org/10.17605/OSF.IO/NJS9Y¹⁸.

This project contains the following extended data:

Data are available under the terms of the Creative Commons CC0 1.0 Universal (CC0 1.0) Public Domain Dedication License.