Stakeholder engagement in health technology assessment: a scoping review protocol

3.1 Research question

The overall objective of the review is to answer the question “What are the existing international practices for stakeholder engagement in the HTA process?” This question will be addressed through the following specific questions:

3.2 Study design

A scoping review is a “type of evidence synthesis that aims to systematically identify and map the breadth of evidence available on a particular topic, field, concept, or issue, often irrespective of source (such as primary research, reviews, non-empirical evidence) within or across particular contexts”¹⁸. As the research question aims to identify a wide range of practices of stakeholder engagement in HTA, a scoping review design would be best suited for it¹⁸. The review will be conducted in accordance with the JBI methodology for scoping reviews¹⁹. The protocol for the review has been registered in Open Science Framework (OSF) and can be accessed through this link: https://doi.org/10.17605/OSF.IO/C3ESF.

The completed scoping review will be reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist²⁰.

3.3 Inclusion/exclusion criteria

3.3.1 Study design

To be included, peer-reviewed publications and documents (e.g. handbooks/guidance documents) must describe processes or approaches for stakeholder engagement in the HTA process using the definition of the HTA process as described in Section 3.3.4. The web pages of organisations or agencies undertaking HTA will be considered for inclusion if they describe the process for stakeholder engagement and fulfil all of the inclusion criteria. There will be no restriction on study designs and all qualitative, quantitative and mixed-method studies addressing the research question will be considered and assessed for eligibility. Inclusion will be limited to studies conducted among stakeholders who have conducted or been part of the HTA process.

Commentaries, perspective or opinion pieces, and editorials reporting findings of a primary study will be included if it meets other inclusion criteria. Otherwise, commentaries, perspective or opinion pieces, editorials, protocols, and conference abstracts will be excluded. Studies that describe stakeholder engagement in a context other than the HTA process will also be excluded.

Studies in a language other than English will be translated into English using DeepL, and assessed for eligibility.

3.3.2 Population

Stakeholders have been defined as “individuals, organisations or communities that have a direct interest in the process and outcomes of a project, research or policy endeavour”^8,9. Stakeholders are responsible, or affected by health and healthcare related decisions that can be informed by research evidence²¹. The population for inclusion is therefore broad. We will include studies and documents whose population includes any or all of the stakeholder groups as listed in the 11 ‘P’s framework developed by Tugwell et al. as these categories were designed to be comprehensive and inclusive of all interest groups¹⁰. However, some stakeholders might fit under different categories and during the process of the review, other groups that meet the definition of HTA stakeholder may also be identified and will be included. Therefore, broad population groups that will be considered for inclusion will be (but not limited to):

3.3.3 Concept

Stakeholder engagement is defined as an iterative process of actively soliciting the knowledge, experience, judgment and values of individuals selected to represent a broad range of direct interest in a particular issue, for the dual purpose of: creating a shared understanding; and making relevant, transparent and effective decisions⁹. The process is multi-directional, resulting in “informed decision-making about the selection, conduct and use of the research”¹⁰. Terms such as but not limited to “partnerships”, “involvement”, “consultation”, “co-production”, “co-creation”, “co-design” and “patient and public involvement”, are often also used to refer to engagement¹⁰. For the purpose of this review, any studies/documents using stakeholder engagement or any of these terms in the context of HTA will be considered eligible for inclusion.

3.3.4 Context

HTA is defined as a “multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its lifecycle”¹. The purpose is to inform decision-making in order to promote an equitable, efficient, and high-quality health system²². For the purpose of this review, the HTA process will refer not only to the multidisciplinary assessment process, as defined above, but also to those aspects which occur prior to or after the decision to undertake an HTA has been made. These include: requesting for HTA to be undertaken, topic prioritisation, defining the scope, managing of HTA, and deciding the findings of the HTA²². For the purpose of this review, de-novo evidence synthesis for HTA undertaken by bodies conducting HTA, HTA that involves critical appraisal of manufacturer submission only, and HTA that involves critical appraisal of manufacturer submission and de-novo evidence synthesis will be included. Any HTA submitted as part of a dossier or documents for reimbursement will not be included.

Inclusion will be limited to stakeholder engagement in the HTA process, as distinct from stakeholder engagement in any of the individual domains conducted outside of a HTA process (e.g. stakeholder engagement in systematic reviews and other evidence synthesis processes). Some agencies that have multiple functions might develop guidance documents applicable to encompassing the wide range of activities they engage in (e.g. HTA development and clinical guideline development). Such documents will only be included if, after assessing the purpose and scope of the document, it is found that they encapsulate the HTA process as defined above.

3.4 Database search

A literature search will be conducted in MEDLINE Complete via EBSCO, Embase via Elsevier, CINAHL Complete via EBSCO, PsycInfo via EBSCO and The Cochrane Library via Wiley. The search strategy will be developed by a medical librarian and peer-reviewed by a second librarian. A sample search strategy for Medline has been shown in Table 1 below. Grey literature sources will be searched extensively using a three pronged approach: grey literature database search, hand searching of websites of HTA agencies, and targeted Google search. The grey literature database search will be run in a database such as GreyMatters. The websites of international HTA agencies will be hand searched to identify guidance documents for stakeholder engagement in the HTA process. The countries and agencies conducting HTA will be identified through reviewing members of AFROHTA, Americas RedETSA, HTAsiaLink, HTAR and INAHTA. Additionally, the websites of AFROHTA, Americas RedETSA, HTAsiaLink, HTAi, HTAR, INAHTA and ISPOR will also be hand searched for relevant literature. A targeted domain specific Google search will be run using the keywords “stakeholder”, “public”, “patients”, and “health technology assessment”. This targeted search will limit the Google search to the materials of different HTA agencies websites. The first 10 pages of the search hits will be screened and literature will be identified for full text review. Additionally, forward citation searching of eligible studies and searching reference lists of identified systematic reviews and included studies will also be conducted. Searches will be limited to the period from 2014 to December 2024 to capture the most recent approaches since the development of the HIQA guidelines (2014)².

3.5 Data management

For data management purposes, the results of the database search will be exported to Covidence (www.covidence.org). Two reviewers will independently review the titles and abstracts and subsequently review the full texts of the identified records. A list of grey literature documents identified through searching of websites will be maintained in a Micosoft Excel spreadsheet and screened by the two reviewers. Those peer-reviewed papers and grey literature documents that meet the inclusion criteria for this scoping review will be included. Any disagreement regarding the eligibility of documents will be resolved through discussion, and using a third reviewer where necessary.

3.6 Data charting

Data charting will be performed by one reviewer and cross-checked by another. A data charting tool developed based on the research questions will be piloted, the two reviewers will trial out the data charting form and enter data from 2–3 articles/documents into the charting form. A sample data charting template is shown in Table 2 below. After trialling the data charting form, the reviewers will discuss any uncertainties and if necessary changes will be made to the charting form. Data will be charted for the following headings:

As this is a scoping review aiming to provide an overview of a diverse range of outcomes, no quality appraisal will be undertaken.

3.7 Data analysis

The quantitative findings will be synthesised using descriptive statistics (frequency, percentage) and described narratively. The qualitative findings will be synthesised using a basic qualitative content analysis approach as recommended by JBI scoping review guidance and will follow the approach described by Pollock et al.^23,24. This is a descriptive approach to analysis and involves an open-coding process whereby concepts or characteristics are allocated into overall categories²³. The basic qualitative content analysis approach consists of three phases as described by Elo and Kyngäs which has been stated to be applicable to scoping reviews as well²⁵. The three phases are: i) preparation, ii) organising, and iii) reporting²⁵. The preparation phase involves assessing if a basic qualitative content analysis is needed or not and then using either an inductive or deductive approach to conduct the analysis. This scoping review will follow an inductive approach for qualitative content analysis. The second phase, the organising phase, involves familiarisation with the data followed by open coding of the data, development of a coding framework and organisation of extracted data within the coding framework. The third phase, the reporting phase, involves presentation of the data through tables, heat maps, word clouds, along with a narrative description of the results^23,25. The choice of data visualisation methods will be dependent on the underlying body of evidence identified.