Knowledge of public patient involvement among health economists in Ireland: a baseline audit

Introduction

Patient and public involvement (PPI) is increasingly recognised as an essential component of health research. PPI is defined as research carried out ‘with’ or ‘by’ members of the public, rather than ‘to’, ‘about’ or ‘for’ them; the term ‘public’ includes patients, potential patients, carers and people who use health and social care services, as well as organisations that represent people who use these services (INVOLVE, 2009). PPI facilitates patients having a greater role in research prioritisation and research design, which is important given the potential of health research in finding solutions to their specific health needs (Popay et al., 2014). PPI also ensures that members of the public who ultimately pay for research through taxation have some say in how research funding is spent. There is certainly an argument that research funding needs to be more aligned with public and patient priorities to ensure that scarce resources are used efficiently and effectively (Brett et al., 2010; World Health Organisation, 2018).

Ireland is in the process of widening and deepening PPI in health research through the PPI Ignite programme, which is funded by the Health Research Board (Health Research Board, 2018). The Ignite programme aims to promote the use of PPI in health research in Ireland. Specifically, the HRB have awarded grants to five universities to support health and social care researchers therein to involve patients and members of the public in meaningful ways across the full research cycle (Hendrick et al., 2017). Fortunately, Ireland can learn from other countries in relation to optimal approaches to PPI in health research (Kreis et al., 2013; Oliver et al., 2008; Wilson et al., 2015). INVOLVE (2016) have published resources on good practice and approaches to PPI in the United Kingdom, including a Values and Principles Framework. There are also appraisal guidelines and frameworks for assessing the quality of PPI (Boote et al., 2006; Wright et al., 2010), while the GRIPP2 (Guidance for Reporting Involvement of Patients and Public) checklist has been developed to enhance the quality of PPI reporting (Staniszewska et al., 2011).

While more guidance is now available on PPI pathways in health research, there is much less clarity on the role of PPI within health economics research (Goodwin et al., 2018). Health economics research is concerned with issues of efficiency, effectiveness, value and behaviour in the production and consumption of health and healthcare. Health economists use economic reasoning and techniques to examine costs, benefits and consequences arising from the allocation of scarce health care resources. PPI can be used to help inform the economic evaluation of health care programmes and interventions, particularly in relation to what costs to include and what outcomes to capture (Pizzo et al., 2015). Some countries currently require PPI to be used in health technology assessment, but the practice is not widespread (Weeks et al., 2017). An economic evaluation of PPI itself covering costs, outcomes and impact may also be useful to demonstrate the gains from involving patients and the public in the research process. There is little information on the use of PPI in health economics research, but its main purpose currently seems to be to improve data collection (Kandiyali et al., 2018). The proportion of HTA organizations that both conduct and evaluate PPI activities remains unclear (Weeks et al., 2017). The most reported forms of evaluation relate to process and patient satisfaction. There has been very little evaluation on the overall impact of PPI in health economics research.

A better understanding of how economists use, interpret and apply PPI is important in health research, but there are differences between PPI and health economics which mitigate against its use by practitioners. Health economics is very much concerned with priority-setting across competing publicly funded health care programmes, while PPI draws on the practical and lived experience of the individual patient. Reconciling the private and the public is not always an easy task. Moreover, health economics is often perceived as a technically difficult discipline, abstract in its orientation and removed from the more immediate health concerns of the patient. There is an absence of a common language to bring the two sides together in any meaningful way. But if health economists want to really understand the health production process, either for modelling or measurement purposes, there is an onus on them to hear the concerns and insights of patients.

However, prior to any meeting of minds, we need to establish the views of health economists on PPI, including documenting their current PPI experiences and perceptions regarding the application of PPI within health economics research (Kandiyali et al., 2018). The current knowledge base regarding the interface between PPI and health economics in Ireland is non-existent, so any new data will be helpful in facilitating a greater rapprochement between economists, patients and the public. Health economics is now used routinely in all forms of health research, especially in clinical trials. If PPI is absent from economic evaluation of health care programmes, the voice of the patient will be weaker when it comes to resource allocation decision-making and priority-setting in health and social care. However, PPI also needs to be rigorously evaluated to determine if it’s having the desired effect on the research process through detailed cost benefit appraisal (Pizzo et al., 2015).

Methods

The main focus of the questionnaire was on audit, with an emphasis on the use and understanding of PPI among health economists in Ireland. The questionnaire was designed to elicit information on: demography; research group/organisation; current knowledge of PPI; current use of PPI; benefits and challenges of using PPI; evaluation frameworks; the role and value of PPI in health research; and the training needs of health economists in regard to the use and evaluation of PPI in research studies. These topics were identified from a detailed literature review on the relationship between PPI and health economics drawn from the following databases: Google Scholar, Medline, EMBASE and CINAHL. The following key words were used in the literature search: public and patient involvement and: health economics; health economists, economics; health technology assessment; economic evaluation; costs; cost effectiveness analysis; outcomes; and impact.

Following the development of a draft questionnaire, we conducted a pilot study, using the online software SurveyMonkey. The pilot was completed by four respondents, three of whom were health economists, while the fourth was an experienced researcher working in PPI at NUI Galway. Following the pilot, changes were made to the wording and sequencing of some questions.

The main survey was then administered to all members of the Health Economists Association of Ireland (HEAI). The vast majority of practising health economists in Ireland are members of HEAI. The Association was set up in 2001 to foster networking and the sharing of information related to health economics on the island of Ireland. The membership at the time of the survey was 135 people, the majority of whom were working in universities, research organisations, industry and the private consultancy sector. The final survey questionnaire was uploaded on SurveyMonkey and email contact was made with all of the members of the HEAI in the first week of July 2018 through the auspices of the Co-ordinator of the Association, Professor Ciaran O’Neill at Queen’s University, Belfast.

The Co-ordinator emailed all members of the HEAI incorporating the following message from the authors:

Please find attached a brief note of a study on patient public involvement (PPI) in health economics research that we are undertaking at NUI Galway. The link below takes you to the survey we are using to collect the opinions of health economists in Ireland in relation to the work.

The attachment that was sent to members of HEAI described the study as follows:

The Health Economics and Policy Analysis Centre (HEPAC) at NUI Galway, in conjunction with Public Patient Involvement (PPI) Ignite at NUI Galway is undertaking a small scoping study on the use and role of PPI in health economics research in Ireland. PPI is defined as doing research ‘with’ or ‘by’ patients and members of the public rather than ‘to’ or ‘about’ or ‘for’ them. It involves consultation, involvement and communication with patients and the public to incorporate their perspective into the research process.

The survey will take approximately 10–15 minutes to complete. We would appreciate that this survey be completed by 30th July 2018.

The research has five objectives:

The study will add to an evolving literature on the use of PPI in health economics research.

The online survey on SurveyMonkey opened with an explanation of the purpose of the study, including a reference again to the five objectives. A follow-up reminder to complete the survey was sent to potential respondents on the 24^th July 2018 and the survey closed on the 30^th July 2018. The data was subsequently analysed using Microsoft Excel 2013.

Results

The response rate for the survey was 33% of all HEAI members, divided almost equally between males and females (n=45). Almost half of respondents (47%) are in the 31–40 age bracket (Figure 1). Only 5% of respondents are aged over 60 years.

Figure 1. Respondents by age group.

Almost 75% of respondents have been working in the health economics field for more than 5 years, with 14% having 20 years or more service (Figure 2). One third of respondents hold positions as lecturers/professors in an academic institution; researchers constituted 22% of all respondents, while 15% hold dual roles of researcher and manager/director of their organisation.

Figure 2. Length of time working in health economics research.

While half of all respondents are either current or past users of PPI in their health economics research, knowledge of PPI is weak and fragmented across all respondents (Table 1). Almost half of respondents rated their understanding of PPI as only fair. A further 20% regarded their knowledge of PPI as weak. A quarter of respondents reported their knowledge of PPI as good, while only 8% said that their knowledge was excellent. Knowledge of the general PPI academic literature was poor overall; 45% stated that they have a weak knowledge of PPI literature, while 40% reported a fair knowledge of the literature. Only 5% of respondents reported an excellent knowledge of the general PPI literature.

The most common route through which health economists recruited PPI contributors is patient groups at 94%, followed by interest/advocacy groups at 78% (Table 2). Recruitment through carer’s groups was 50%, while word of mouth and social media reached 33% respectively amongst respondents. The least reported approach for PPI recruitment amongst health economists was radio interview and online videos at 17% and 6%, respectively.

Table 3 shows the different tasks carried out by patients and the public involved in health economics research. Two thirds of respondents stated that public/patients help to identify research priorities and potential outcomes, as well as helping with sharing/dissemination of results. PPI plays a significant role in the review of reports (56%) prior to publication. Nearly two fifths of people involved in PPI advise on the instruments used to measure outcomes; a similar proportion support the production of lay summaries of scientific reports. One third of respondents say that PPI supports participant recruitment. There is much less involvement in data collection and writing reports. There is no evidence of any PPI in data analyses.

When asked to rank the reasons why they involve patients and/or members of the public in research activities, more than half of health economists choose to help ensure economic reasoning reflects patient and public values as the primary reason (Table 4). A further 20% reported to conform with funding requirements as the primary reason for using PPI. Helping to promote fairness and inclusion of a range of stakeholders in economics research ranked second highest at 59%, while the need to ensure that research survey materials are understood by the public and patients ranked third highest at 35%.

Approximately 37% of health economists currently evaluate PPI activities in relation to their own research activities. The most evaluated activity reported is the monitoring of process, specifically in relation to tasks and recruitment (63%) (Table 5). This is followed by patient/public satisfaction at 34%. One quarter of respondents evaluate protocols for meetings and advance information for participants. A further 13% of respondents evaluate the cost of including PPI in their research projects, including costs associated with the recruitment process and attendance at meetings. Only 6% of respondents evaluated the overall impact of PPI. The majority of respondents do not publish or share the results of internal evaluation of their PPI activities. When it does occur, the sharing of evaluation results with participants at meetings is the preferred dissemination method favoured by respondents.

Discussion

PPI is relatively new in overall health economics research in Ireland. Consequently, it is not surprising that knowledge of PPI amongst health economists in Ireland is weak. A poor knowledge and understanding of PPI implies that its adoption in health economics research is also likely to be weak. If the basics of PPI are not fully understood by health economists then it is difficult to see PPI being used effectively in research programmes. Training is one of the avenues through which the knowledge base of PPI in health economics can be expanded and improved, thereby ensuring greater adoption of PPI in health economics research programmes. The health economists in this study recognised the need for more education and training in relation to PPI, with 94% saying that they would undertake training if it were made available. The main areas of training need identified by respondents were: the recruitment of patients; PPI procedures and protocols; managing conflict of interest; best practice on the remuneration of PPI contributors; and methods of evaluating PPI. Respondents also wanted more information on incorporating PPI feedback into research design and the preparation of lay reports on technical pieces of research. Guidance on how to successfully incorporate PPI into grant funding applications was also referenced.

Only a small number of health economists in Ireland have conducted any evaluation of PPI within their own research activities, mainly in relation to process and protocols. Without knowledge of impact, it is impossible to be definitive about the value of PPI in health research. The challenge is to ensure that more health economists include PPI in their work and in their evaluation methodologies. It should be easier for economists to undertake evaluation of PPI, given the methodologies at their disposal, especially in relation to instruments for the measurement of costs and benefits. NICE have recommended that organizations should develop an evaluation plan at the same time as the PPI activities are established, incorporating patients or members of the public on the evaluation team.

The most difficult part, however, is to assess the relationship between PPI and health impact, as mediated through practice and policy change arising from the research process. Any change can have a long time horizon and it can be difficult to attribute research activity directly to changes in health outcomes.

There are still many barriers to PPI, some of which are referenced in this study. Health economists report PPI to be a challenging task due to limited time, inadequate resources, knowledge deficiencies as to the meaning of involvement and the absence of skills to conduct effective PPI. The evidence is that funding bodies do not always view PPI as a high priority and consequently do not provide enough funding to cover PPI activities (Blackburn et al., 2018; Gove et al., 2018). The absence of funding can undermine the potential of PPI, particularly in economics research where the philosophy of PPI is not yet rooted (Pizzo et al., 2015).

Health economists also reported that a lack of commitment and burnout on the part of PPI contributors can be a challenge for researchers. The population and size of Ireland means that the same cohort of PPI contributors are often recruited for different PPI projects, leading to a major sustainability challenge. The overuse of the same people in PPI can lead to participation fatigue, as well as predictability in relation to the involvement. PPI must reflect the variation in patient and public experience, if it is to be relevant to in decision-making and priority-setting.

The tokenistic attitude of some researchers is also a major challenge for PPI adoption, often resulting in PPI contributors feeling unimportant, undervalued and unable to contribute fully to the overall PPI process. The conflicting worlds of researchers and PPI contributors, where the values and assumption of researchers do not match the expectations of PPI contributors can also impact negatively on the PPI experience. For example, health economists reported that they sometimes felt under pressure to compromise their methodologies to accommodate PPI, which can negatively impact the integrity of their economic research. It would be interesting to examine what PPI contributors’ opinion of health economics research is and whether they view economic models and methods as sometimes inhibiting fruitful dialogue between the two sides. Unfortunately, this study only solicited the views of health economists.

In conclusion, PPI is becoming an important element of the health research landscape in Ireland. Most PPI support in health economics research is focused on determining research priorities and deciding on appropriate outcome measures. Economists in Ireland value PPI as a means of ensuring that economic reasoning reflects patient and public values, linked to a commitment to inclusion and fairness in the research process. However, it is evident from this study that health economists in this country do not have a strong knowledge of PPI, either in practice or in theory. Training is the process through which knowledge of PPI can be strengthened and health economists expressed a strong desire to upskill in this area. Those economists who are currently using PPI in research face similar challenges and barriers reported elsewhere, including recruitment, ethical issues, tokenism, time and funding. Some research funders in Ireland, most notably the Health Research Board, are now advising researchers to budget for PPI activities in their grant applications. This is a welcome initiative that perhaps other funders could be encouraged to follow. But, equally importantly, research leaders must be prepared to assign adequate budget in their grant applications to fund PPI activities. Dedicated funding within research grants for the economic evaluation of public and patient involvement in research would also help to accelerate the diffusion of PPI knowledge and methodologies among health economists.

Data availability