Developing and assessing the usability of a prescribing quality and safety dashboard in Irish general practice: protocol for a qualitative study

Setting

This study is set in general practice in the Republic of Ireland, where there is a mix of eligibility in terms of access to primary care and medicines. Individuals covered under the means tested General Medical Services (GMS) scheme, are eligible for free prescription medicines, with a small co-payment. In 2023 this represented 31% of the population. For those not covered by the GMS scheme there is a monthly threshold of €80 per household for the cost of prescribed medications, beyond which the state covers any further costs¹⁰. Individuals covered by the GMS scheme, and approximately 10% of the population covered by the Doctor Visit scheme, have access to free GP visits and are registered with a single GP. However, the majority of patients (private patients) pay for GP visits and may not be registered with a single practice, making it challenging to calculate an accurate practice population denominator.

Within Europe, the UK and Nordic countries have led the way in advanced e-prescribing infrastructure with a central repository which can then be used for feedback^11–13. In the UK there is routine national level prescribing surveillance where prescribing is actively monitored through structured frameworks, using prescribing data to provide feedback, financial incentives, and targeted interventions for GPs¹⁴. Ireland relies more on voluntary adherence to Health Service Executive (HSE) guidance. Examples of such guidance include the Preferred Drugs initiative which provides recommendations on the use of preferred drugs within ten common drug classes that are selected based on effectiveness, cost effectiveness and practical considerations such as dosing regimens¹⁵. Similarly, the HSE Antimicrobial Resistance and Infection Control team has developed guidance for community settings on “green” antibiotics, which are preferred to minimise the risk of anti-microbial resistance and adverse drug reactions¹⁶. GPs currently receive feedback on benzodiazepine and antibiotic prescribing from the HSE based on dispensing data from the GMS scheme. However, this may not give the full picture compared to GP prescription data which includes both private prescriptions and medicines covered under drug re-imbursement schemes.

Dashboard description

In collaboration with the data analytics platform, MedVault (a private company that supports general practices in using their clinical data to improve care and manage reimbursement claims), the lead author (CMC) has developed prescribing safety dashboards using Snowflake for data processing and Amazon QuickSight for visualisation. Practices will have ongoing access to their dashboards from February 2025 for six months. Each practice will be able to view their own prescribing data and compare it to aggregate data from other participating practices, see Figure 2. However, practices will remain anonymous to one another, MedVault and the research team.

Figure 2. Sample page from prescribing quality and safety dashboard.

The prescribing dashboard was designed to provide insights into prescribing patterns at the practice level. As practices have one unique login, it is not feasible to provide prescriber level feedback. The dashboards will initially display trend data over five years with comparative benchmarking, with a subsequent update in May 2025. Table 1 lists the included metrics with reasons for their inclusion.

Dashboard development

Data from participating practices (n=27) were scrubbed from practice servers by MedVault, see Table 2, and transformed within Snowflake, where key prescribing safety metrics were generated and structured for visualisation.

The WHO Anatomical Therapeutic Chemical (ATC) classification system, which categorises medicines according to the organ or system they act on and their therapeutic, pharmacological and chemical characteristics¹⁷, was used to identify drug groups and classes. There were errors in the WHO ATC code mapping process within practice systems so this was updated using a codebook developed for prior research that mapped branded and generic drug names to WHO ATC codes¹⁸. Using the high-dose PPI prescribing metric as an example, all PPIs were identified using the ATC code beginning with A02BC. For metrics displayed as prescribing rates, rates were calculated annually using the total practice population, whereby numbers of private patients were calculated by counting individuals who had at least one encounter in the preceding three years. An encounter was defined as any recorded contact with the practice (including a prescription, telephone consultation or nurse consultation). High-dose PPI prescribing was the only metric that required dose information. The relevant dose was identified by searching the drug name variable for dose indicators corresponding to each PPI in the dataset (e.g., '40' for omeprazole, '30' for lansoprazole). For high-risk non-steroidal anti-inflammatory drug (NSAID) prescribing, the anonymous patient ID was used to identify instances where patients who had received more than one prescription for either an angiotensin-converting enzyme (ACE) inhibitor or angiotensin II receptor blocker and a diuretic had also received at least one prescription for a NSAID in the same year. Instances of patients who had received more than one prescription for an anticoagulant and at least one NSAID prescription in the same year were identified in a similar manner. For all comparative metrics, an identical dataset was cloned in Snowflake and imported into QuickSight with row-level permissions applied, ensuring that each practice could only view its own data. This graph was displayed alongside a graph showing data from all practices, allowing each practice to compare its performance with others. The DU90% indicator (Table 1) was calculated on an annual basis for each practice. The total number of unique ATC coded items were counted by practice and by year and then sorted in descending order, so the cumulative proportion of prescribing for each drug could be calculated. Items whereby the cumulative proportion was <90% were identified as belonging in the DU90% segment for that practice in that year. A bar chart displaying all the drugs in descending order of quantity prescribed in each practice DU90% was developed in QuickSight, see Figure 3. This was the only metric that was not comparative. See extended data for the SQL code used to derive prescribing safety metrics.

Figure 3. Practice DU90% segment.

Study population

General practices in the Republic of Ireland that currently use the MedVault data analytics platform are eligible to participate. While the participating practices represent only a small proportion of practices nationally, they are broadly representative in terms of size and location and use two of the three predominant electronic health record systems in Ireland. MedVault practices who had previously consented to share anonymous prescription data (n=27) and receive interactive prescribing dashboards were asked to consent to be contacted about this qualitative study. Contact details for GPs interested in participating in this qualitative study were stored separately and not linked to prescribing data. For this study a purposive sample of approximately 12 prescribers from different practices will be recruited to ensure a range of perspectives and experiences. This sample size was chosen based on the concept of information power, which suggests that a smaller sample size is sufficient when the sample is specific, the study aim is clearly focused, and the expected quality of dialogue is high²⁸. All participants will have relevant experience and the sample will be relatively homogenous, thus ensuring high sample specificity. The narrow aim to evaluate the dashboard usability and the think-aloud method employed are expected to yield high-quality, in-depth data. Participants will be individually recruited and consented. If more than 12 volunteer a diverse sample from within this will be selected.

Data collection

The lead author (CMC) will conduct think aloud exercises and semi-structured interviews. She is a practicing GP who developed the dashboards. She has an existing professional relationship with GPs from one of the recruited sites but otherwise has no existing relationship with potential participants. Participating GPs will be aware of her professional background and of the fact that she developed the dashboards. Microsoft Teams, will be used to screen and audio record the think aloud exercise and subsequent semi-structured interview. Cameras will be turned off during the session, and no video recordings or facial data will be collected. Only screen activity and audio will be recorded for analysis. Recordings will be transcribed using the live transcription feature, and verified manually. During the think aloud exercise GPs will navigate the dashboards and verbalise their thoughts and reactions in real-time to assess how GPs engage with and interpret their data using these visualisation tools. GPs will be reassured that their individual prescribing data will not be stored or reported and that the purpose of the activity is to assess the usability, layout, and clarity of the dashboards. Screen recordings will be deleted after analysis. The GPs will be given four discrete tasks for the think aloud process. This will be followed by the semi-structured interview, where the topic guide will include questions exploring perspectives on prescribing safety, receiving prescribing feedback and views and opinions of the dashboards, see extended data. This approach will provide insights into the usability of the dashboards and how prescribers interpret and act on the feedback provided. The interview topic guide was developed with multidisciplinary input and iteratively refined based on pilot testing. The estimated duration of the entire interview including the think aloud process is 45 minutes.

Analysis

Transcripts will be read and re-read to support familiarisation with the data and to capture the context and nuances of participants’ responses (CMC). For the think aloud process, screen recordings will be reviewed alongside the transcripts, with verbal data checked and amended for accuracy. Notations will be added to indicate relevant on-screen activity (e.g. navigation paths, pauses, points of confusion), enabling interpretation of how participants interacted with the dashboard in real time. Coding for the think aloud exercises will focus on understanding GPs' engagement with the dashboards, identifying usability issues, and assessing the interpretation and perceived usefulness of the feedback. This will be done using a deductive coding framework based on Nielsen’s five quality components of usability: learnability, efficiency, memorability, error recovery and satisfaction²⁹. These components will form the primary themes, with sub-themes developed from the data. This combined analysis will allow for a richer understanding of usability, layout, and interpretability, by linking verbal feedback with observed behaviour.

An inductive thematic approach will be used to explore GPs’ perspectives on prescribing safety and feedback. The analysis will be organised around three broad topic areas: general views on prescribing safety, experiences and perceptions of feedback, and views on the dashboards. Within each area, themes will be developed inductively from the data, allowing patterns to be identified based on participants’ own words and reflections. This approach will allow themes to be developed directly from the data, helping to capture GPs’ ideas and experiences based on the work of Braun and Clarke³⁰. The transcripts will be systematically coded, with initial coding identifying key phrases, concepts, and categories that are developed from the data. These codes will then be sorted into broader themes that capture the main topics and patterns across the interviews.

Strengths and limitations

The mixed methods approach will strengthen the depth and richness of the findings; with the think-aloud method allowing for the observation of how GPs interact with the dashboards in real-time and the follow-up interview questions probing how GPs reflect on receiving this feedback. The major limitation of this work is the potential for social desirability bias as the lead author and interviewer (CMC) is also a GP and developed the dashboards. This dual role may discourage some participants from expressing criticism or scepticism openly. However, the interviewer’s clinical background and familiarity with the dashboards also enable more informed probing and interpretation of participants’ responses. To mitigate against the risk of biased interpretation of data interviews and codes will be reviewed by a second member of the study group. The interviews will take place within 2–4 weeks of the dashboards being made available. It is anticipated that some GPs will have explored the dashboards, while others may have limited or no exposure. Apart from notifying practices that the dashboards were available, no specific instructions or encouragement to access them were given. Limited exposure prior to the think-aloud exercise may enhance the assessment of usability and clarity. However, this also limits the opportunity for GPs to reflect on or act upon the data, which may reduce the depth of insight gained during the follow-up semi-structured interviews.

Implications for practice, policy and research

With recent planned enhancements in e-prescribing and the data infrastructure of Irish primary care, this research will provide important insights into how routine data may be harnessed for audit and feedback purposes. Importantly, it will explore the usability of a prototype dashboard and how GPs interact with this form of data-driven support.