Developing composite indices of geographical access and need for nursing home care in Ireland using multiple criteria decision analysis

Defining the decision problem

This stage requires understanding the background to the problem, triangulating clearly articulated aims, generating a list of stakeholders that could be invited to participate, and deciding on which steps they will be required to take part. The balancing of both scientific/technical and social/human factors - referred to as a “socio-technical” approach in the literature²¹ - can be something of an art, but must be considered as part of this process. The paper describes a process whose primary purpose was as an evidence or knowledge generating exercise, and stakeholders invited to take part were acting as independent experts who could share their experience. MCDA approaches are particularly well suited to converting such expert experience into quantifiable scores¹¹.

Selecting criteria and measuring performance

There are a number of factors to consider in choosing criteria for an MCDA model²⁰:

These are guiding principles rather than hard rules, and as elsewhere any final model remains a simplification. Estimates of performance on each of the criteria can be gathered in a variety of ways, ranging from evidence synthesis to expert elicitation where relevant quantifiable data are unavailable¹¹.

Accessibility-related criteria were calculated by location, while need was estimated using secondary data from the national census. Expert opinion was therefore only required in weighting the criteria and in helping to interpret the results.

Access. Data used for calculating accessibility came from combining several standalone datasets. A list of all registered NHs nationally is available from the HIQA website²³, which was downloaded in June 2019. The geolocation of many homes was provided therein; others were calculated manually by inputting the associated address into Google Maps. Data from HIQA inspection reports (covering the period Feb 2018-May 2019) were used to measure NH care quality. Prior to February 2018, a separate inspection regimen was used, which would have otherwise made prior comparisons challenging. Fair Deal fees per resident were calculated using a dataset provided by the NTPF, covering the period January 2019 – September 2019. Such data on fees was not available from the homes that are operated by the HSE.

We defined four types of accessibility relevant to nursing home care that broadly correspond to headings described by McIntyre and the WHO described in the Introduction, reflecting: the availability of nearby services (availability), the range of choice of NHs (acceptability), the NTPF fee paid for those services (affordability), and a simple measure of NH quality based upon HIQA inspection reports (information). Their formal definitions are presented below. Each was based upon relationships between NH locations and each electoral district’s (ED) centroid. EDs are the smallest legally defined administrative areas in the State, allowing the greatest possible granularity for our results. A total of 3,409 EDs were included, with their boundary files being downloaded from the Central Statistics Office (CSO) website. The four criteria used to build a composite measure of “overall accessibility” were:

The indicators for these four criteria are used as a basis for aggregating into a composite index of overall accessibility at ED level nationally.

Need. As part of a scoping exercise, a targeted literature search was carried out to identify factors that influence need for nursing home care. The search string ("nursing home*" OR "elderly care home*") AND TOPIC: (admission* OR admit*) AND TOPIC: (predict*) was used, with the last search conducted on 12 December 2019. Papers describing the risk of admission related to patients with specific, single issues for which we did not have access to data are not included (these included falls, delirium and Parkinson’s disease). Six papers^28–33 were identified from a search of “Web of Science” that used meta-analyses, literature reviews or multinomial models to identify factors that were associated with nursing home admission. All six were US-based. Several of the papers described factors in terms of the influential “Behavioural Model of Health Services Use” model headings³⁴, which describes the drivers of accessibility and healthcare utilisation in terms of predisposing factors, enabling factors and need. They are reported using this model in Table 1.

This created a longlist of variables for potential inclusion. To be included in our final MCDA need model, variables further had to be judged to be relevant in an Irish context and thereafter to be mappable to relevant data from the most recent (2016) census. Similar approaches using census data that have been reported elsewhere^35,36. have noted the utility of such open data approaches in developing a composite index for public use. After liaising with the CSO, we gained access to summaries of data confined to populations over 65. However, these could only be made available at the level of 31 geographical units (rather than the 3,409 units at ED level) for data privacy reasons. These corresponded with either county or city borders and, for convenience, we will refer to these throughout the paper as ‘counties’.

Where possible, direct or plausible proxy data were chosen which reflected these variables of need. These were not always perfect: our ADL measure for example excluded certain aspects such as toileting, which is not recorded in the census. Other promising candidate variables, such as social activity levels and prior nursing home admission, were excluded entirely as relevant data was not available.

Gender’s impact upon need for care appeared to be inconclusive varying between studies. We assumed that having a male majority meant lower need before talking to stakeholders, with scope to change this depending on their views and experience. The proportion that are “white American” was not felt to be relevant in an Irish nursing home setting and was not included in the model shown to participants. The formal definitions used for each variable, its median and the location of its maximum and minimum scores are shown in Table 2.

Scoring alternatives

Measurements were transformed using “value functions” which are used to ensure that scores on different criteria are normalised to a common 0-100 scale. Choosing an appropriate value function for a variable depends on the context of the data and relies somewhat upon the experience of decision makers and those designing the study. While value functions can be assumed to be linear for practical purposes in many cases²¹, it is important that in principle that each point on the scale represents equal (within criterion) increments¹⁶. Therefore, an ED scoring 50 on a criterion should be about halfway between the best performing and worst performing EDs on that criterion; if this lacks face validity then a linear value function would be unsuitable.

All functions for the value functions used are described in this section. Linear value functions were felt to be acceptable for most variables, including all of those related to need as they used a single, “per person” scale¹³. All were assumed to follow a negative linear function (i.e. $\frac{(\max (x_i)-x_{ij})}{(\max (x_i)-\min (x_i))}$), for criterion i and county j. Figure 2 shows the distribution of the need criteria after the application of these linear value functions, with 100 implying highest need and 0 the lowest. Certain geographical patterns are evident. For example, cognitive impairments rates are noticeably higher in Dublin than elsewhere, and need relating to ADLs is particularly high in the northern counties (alongside Dublin city centre).

Linear value functions were also considered acceptable for two accessibility criteria (distance to quality homes and affordability), but for those relating to “choice” and “availability”, alternative value functions were required.

EDs with a HHI of 1 have a ‘choice’ of only one NH within the relevant radius. However, it was felt that the value function must further acknowledge that areas without any home in the given radius had, in some sense, even less choice. Such areas, with a HHI of “NA” were given the worst possible score on HHI of 0; the transformed HHI of other variables was calculated using the formula $\frac{e^{-HH{I}_j}}{e^{-\min (HHI)}},$ to ensure that the ED with the lowest HHI (and hence greatest competition) would itself be given a score of 1. This function meant that in practice EDs with access to a single home (and thus a 100% market share for a single home implying a HHI of 1) were given a score at approximately 0.373 on this criterion $(\text{i}.\text{e}.\frac{e^{-1}}{e^{-0.01442}}).$

Availability was highly skewed. It was felt that a diminishing marginal of utility of each additional bed was more plausible than each additional unit being of equal value. The accessibility score was therefore log transformed - thus reducing the impact of the most extreme upper values - and hence a positive linear transformation applied (i.e. an equation of $\frac{\log (\mathit{max}(x))-\log (x_j)}{\log (\mathit{max}(x))-\text{log}(\mathit{min}(x))}$)

Figure 3 shows the distribution of the accessibility criteria after the application of the relevant value functions. It is clear, for example, that accessibility relating to affordability is particularly low in Dublin (i.e. fees are higher there), and that this gradually improves as EDs get further away. Fees are also generally relatively high near Cork City (in the south of the country) and in Sligo in the northwest. Areas of deep purple on the HHI map indicate EDs that have no homes within the 12.5-km radius. Availability is noticeably higher in Dublin than elsewhere. The minimum distance to a quality home is more dispersed around the country, which corresponds to the locations of homes marked green in Figure 1.

Figure 1. Nursing home locations, sized by number of beds and coloured by most recent inspection.

Green, fully compliant; red, some criteria non-compliant; grey, no inspection over relevant period.

Figure 2. The nine crieria measuring nursing home need, after application of value functions.

These are displayed by county, yellow in this case means highest need.

Figure 3. The four measures of accessibility to nursing homes, after the application of their value functions.

They are displayed by ED, with purple indicating low access and yellow high access.

Weighting criteria

Weighting is normally carried out by “stakeholders”, who have relevant expertise and are capable of expressing preferences and quantifying value judgements³⁷. Care must be used in selecting whose preferences are used³⁷ and the group must be large enough to be representative but small enough to be manageable and for decisions to be made within a reasonable timeframe³⁸. Our model uses the views of three individuals (two from non-governmental organisation ALONE (respectively the Head of Services and Projects Manager) and one from the NTPF (Contract Manager). ALONE is well placed to represent the views of older people, given its formal role as advocates for the group and its experience of matching individuals with NHs. The NTPF were approached to ensure that a payer perspective was also included.

Swing weighting requires that stakeholders explicitly consider only from the range the possibilities under consideration (from best performing action to worst, on each criterion)³⁹, rather than the feasible range. If all neighbourhoods had similar levels on a specific criterion, then its importance to the decision (and hence weighting) will be relatively small.

In the prior stage, all criteria were normalised on a 0–100 scale, employing suitable value functions to ensure internal consistency for each criterion in terms of utility. This weighting stage allows participants to ensure that the criteria can be scaled appropriately to reflect “how big is the difference… and how much do you care about that difference?”¹⁶, with swing weights effectively operating as exchange rates between criteria. This allows a total overall score to thereafter be created using a weighted sum approach.

Participants are asked to imagine living in a neighbourhood with the worst score on all criteria and where there existed an opportunity to increase one criterion to match its ‘best’ score nationally. In this hypothetical scenario, whichever criterion they choose is thereafter considered the most important by the participant⁴⁰. Stakeholders are then asked which criterion they would change next, next after that, and so on. Each criterion is assigned a relative weight; for example, the swing for X is 80% as important as that of the most important criterion. These are thereafter collated and normalised. Individual and collated weights for both access and need are shown in Table 3.

Access weights. An overall figure for ALONE was calculated using the geometric average of its two participants, and then the ALONE average was combined with the NPTF preferences again using the geometric average (with the two weighted equally) to create an overall average swing weight, which was subsequently normalised.

The ordering of preferences was identical for both ALONE participants, and quite similar for the NTPF representative. The weights are particularly dominated by the two criteria relating to distance – the nearby availability of beds and the distance to a fully compliant home. That distance to travel was found to be so important mirrors prior findings from Shugarman and Brown⁴¹.

Need weights. Criteria were initially ranked from most to least important by each stakeholder. For one participant from ALONE, initial rankings were thereafter transformed into weights using the procedure outlined by Alfares and Duffuaa⁴², which she felt were an acceptable reflection of her preferences. The other ALONE participant developed her own weights based upon her initial rankings, but for certain criteria (mean age, IADL and living alone) preferred to represent these with a range rather than a point estimate. For the purposes of our baseline model, the geometric average of her upper and lower limits of the range were used; this is described further in Sensitivity analyses results. She also thought that the proportion by gender in the community and the proportion of homeowners were unlikely to be at all influential of need in her experience, and should be given a weighting of zero. These were similarly ranked lowly by both other participants, who were satisfied to use the geometric average again to collate a collective judgement. Again, overall the views of all participants are similar across criteria, described in sensitivity analyses. As previously, a collective ALONE judgement was combined with that of the NTPF representative (on a 50-50 basis), and swing weights were normalised.

Calculating aggregate scores and incorporating uncertainty

Scores calculated for each geographical unit on each criterion were combined with the associated weight on each criterion to calculate overall accessibility scores for each electoral district, and overall need scores for each county. Both were performed using a standard weighted sum approach as shown below:

where

Each neighbourhood could thereafter be ranked by its overall score, whether for need or accessibility. Scores were subsequently mapped to identify geographical patterns in both accessibility and need, as well as the ‘ratios’ between these, which can highlight potentially underserved areas systematically. Such mapping was further carried out using cartograms that are warped by the local populations over 65, in order to better highlight if areas of high population are poorly served. Clustering within patterns is investigated and described, though some questions remain as to why certain areas remain underserved. Results of sensitivity analyses are presented in Sensitivity analyses results.

Sensitivity analyses results

Baseline results were compared against recalculated results using:

One often-used approach in MCDA to carry out sensitivity analyses is to calculate the changes in weightings required to change the final ranking of alternatives (i.e. neighbourhoods in our case). Such an approach is unsuitable in this case given the large number of neighbourhoods being compared – any perturbation whatsoever in the weightings would likely lead to some change in the ranking of EDs given there are 3,409 of them. Instead, Table 4 shows the correlations between the results of the baseline MCDA models reported previously, and the three alternative weighted models. They are extremely similar in all cases, and as a result the alternative models barely impact the results.

Correlations between rankings rather than scores are similar (and slightly higher), with the lowest such correlation being 0.9766 for the baseline ranking of need vs equal needs.

These results are not particularly surprising. It is a recurring theme in MCDA studies that despite the fact that weightings (and other stages) require fallible human judgements, findings of MCDA approaches are typically very robust⁴³.

Underlying data at nursing home-, district-, and county-level are available at Open Science Framework⁴⁴.

Limitations

Datasets had to be constructed and matched manually (due to the lack of a single unique identifier used across them) meaning that comprehensive and perfect datasets were not available. NH fees used related only to private NHs and, even then, only 424/460 private homes could be matched to a fee.

A further 195 homes not inspected over the relevant period could not be accredited as ‘fully compliant’ by our measure (and hence were considered as equivalent to non-compliant homes for this criterion). Of the remainder, 112 were fully compliant, and 274 not fully compliant.

Distance to travel used Haversine-, rather than road-distance, which may offer a better reflection of barriers to care and accessibility.

It was not possible to elicit views from representatives of the NH market to offer a provider perspective, which might have offered alternative viewpoints or helped to clarify why certain areas are underserved or seemingly unappealing for development.

Analyses were, as previously stated, constrained by the questions asked in the census. If other data sources had also been included it is possible that the need measure could have been more comprehensive. Notwithstanding this, we believe that the measure described in this paper is meaningful and requisite for describing the needs of local populations.

Underlying data

Open Science Framework: Developing composite indices of geographical access and need for nursing home care in Ireland, using multiple criteria decision analysis. https://doi.org/10.17605/OSF.IO/3DX2C⁴⁴.

This project contains the following underlying data:

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).