Effectiveness and cost-effectiveness of out-of-hours palliative care: a systematic review

Background

Demographic ageing brings a growing burden of complex and life-limiting disease, posing a major challenge for health systems worldwide^1,2. The value of and need for an integrated palliative care approach to improve quality of life for people with life-limiting illness and their families is widely recognised^3–6.

While preferences vary by conditions and personal circumstances, studies routinely find that a majority of people with terminal illness prefer to stay at home, provided that they can access appropriate services and supports^7–12. Typical professional hours run from 9.00am to 5.00pm, Monday to Friday, or similar, meaning that three quarters of a patient’s week occurs ‘out of hours’. Meeting the policy goal of universal palliative care available according to need therefore requires comprehensive out-of-hours care provision. Poor patient outcomes have been linked to inadequate community supports and a lack of patient confidence in out-of-hours access^13,14.

In the United Kingdom, guidelines from the National Institute for Clinical Excellence have recommended 24-hour specialist palliative care access for people with cancer since 2004¹⁵. Nevertheless, priority-setting exercises with patients, carers, volunteers, and health and social care professionals have identified “the best ways of providing palliative care outside of working hours” as the top research priority in both the United Kingdom and Ireland^16,17.

Policy context

Ireland was among the first countries in the world to adopt a national palliative care policy in 2001^18,19. Policymakers have initiated an update of the policy, to start in 2020²⁰, which will be conducted within the Sláintecare agenda²¹. Among multiple goals, Sláintecare aims to reorient the Irish health and social care system away from the acute setting to community-based services²². In this context of system-level reform and palliative care policy review, the Irish Department of Health funded this study to establish best available international evidence on out-of-hours palliative care.

Palliative care is an interdisciplinary field that occurs across settings and cross-cuts many other medical specialisms. Out-of-hours services are therefore likely to be diverse, potentially including the further extension of existing services, e.g. a specialist inpatient team working evenings and/or weekends; development of new services, e.g. telehealth services offering symptom advice or counselling; or the upskilling of other branches of health care, e.g. palliative care education for emergency department triage nurses, general practitioners, or paramedics.

Development of out-of-hours services should be guided by high-quality evidence on best practice. This could include studies evaluating a range of important considerations such as patient and public preferences, population need, workforce planning, commissioning and approaches to implementation and integration of services. However, making policy recommendations requires evidence about their effectiveness or cost-effectiveness.

Rationale

Multiple systematic reviews have been reported in the field of palliative care. Some have examined effectiveness and/or cost-effectiveness across multiple settings and configurations^23–33, while others have focused on specific models and settings, including hospital inpatient^34,35 and outpatient³⁶, home care³⁷, and day care settings³⁸, as well as care provided by unpaid family carers³⁹. However, we were not aware of any review identifying and organising systematically the evidence on out-of-hours services.

We therefore conducted a systematic review of peer-reviewed studies and grey literature that specifically addressed this research question:

Studies that comparatively evaluated the effect of out-of-hours specialist or generalist palliative care for adults on patient and caregiver outcomes, and on effectiveness and cost-effectiveness were eligible, irrespective of design.

Protocol and registration

We registered the review protocol on PROSPERO CRD42018111041⁴⁰.

Eligibility criteria

Studies reporting on the following PICOS (Participants, Interventions, Comparators, Outcomes, and Study design) were eligible for this review.

Types of participants. Studies were eligible if they examined adults (18 years and over) in their last year of life and/or had a terminal illness and/or had other serious/complex medical needs, and/or were a carer for someone who met these criteria.

We placed no restrictions on diagnosis/need, or on patient/carer perspective. Studies of children (under 18 years of age) and studies pooling children and adults without reporting the results separately were ineligible.

Types of interventions/exposure/comparators. Out of hours was defined as outside of typical working hours (i.e. 9.00am–5.00pm, Monday to Friday) and therefore including overnight, weekends, and public holidays. In the event of any ambiguities with respect to intervention timing in otherwise eligible studies, we decided prior to beginning our review to resolve these through discussion and consensus among the core researchers of the review team, and, where appropriate, through contact with the author of the study in question.

The best-established models of care internationally are inpatient hospices, inpatient and outpatient hospitals, home care, and day care. We did not define the term ‘palliative care’ before starting our review, seeking studies of out-of-hours care for people with life-limiting illnesses, resolving ambiguities through discussion and consensus among the review team, and, where appropriate, contacting the corresponding author. We did not restrict eligibility by either setting or generalist/specialist configuration of staff.

Eligible interventions therefore included both new models of care provided outside of typical working hours, and already established models of care where the effect of out-of-hours provision specifically was evaluated and reported on.

We required that studies were comparative in nature. Eligible comparators therefore included usual care and/or alternative models of out-of-hours palliative care.

Types of outcomes. Our primary outcomes of interest were the effectiveness (patient/carer outcomes) and cost-effectiveness (economic outcomes) of interventions.

We took a broad approach to all outcomes. Patient and/or carer outcomes could be quantitative or qualitative, and included quality of life and experience, as well as survival. Effectiveness outcomes had to be reported by the patients, carers, or a credible proxy. Perceptions of patient and/or carer outcomes from healthcare staff or administrators were not deemed eligible. Eligible economic outcomes included any resource utilisation typically considered to fall within the societal viewpoint (e.g. cost to payers, service users, and families, as well as unpaid care). We did not require resources to represent the literal cost of provision, but also classed non-cost measures of resource utilisation as eligible – e.g. insurance programme charges and frequency use data combined with validated unit costs. Any study reporting within cost-consequence frameworks, such as cost-effectiveness analysis and cost-utility analysis, was also eligible.

Ineligible outcomes were those that related to other parties, e.g. the experience or capacity of clinical staff or the perceptions of healthcare workers of the effectiveness and cost-effectiveness of services.

Types of studies/reports. Study designs vary considerably in research on serious illness in the context of profound ethical and practical considerations⁴¹. As such, we did not restrict our search to any one design. We planned to include the following types of studies: prospective/retrospective cohort studies, before-and-after studies, randomised controlled trials, economic evaluations, qualitative/descriptive studies, and pilot studies.

We excluded studies that did not comparatively measure the effect of interventions on our outcomes of interest, since comparative evaluation was considered intrinsic to our research question.

Time period. Studies were only eligible if they finished data collection no earlier than 1 January 1996 and were published no earlier than 1 January 2000.

We based this decision both on the rapidly changing palliative care landscape in the period 1998–2018, and on our own national context: Ireland’s current official policy was written in 2001 (and therefore had the chance to incorporate relevant research prior to that point).

Cultural and linguistic range. Given the skills of the research team, only English-language materials were eligible for inclusion.

All returned studies in a language other than English were recorded and are reported separately in our results.

Database search: information sources and search terms

Two information specialists (CH and DM) searched the following electronic databases:

Searches were conducted on November 12^th, 2019.

Information specialists (CH and DM) and subject experts (BMJ, PM, RMcQ, MR) devised searches for keywords in the titles, abstracts, subject headings, and controlled vocabulary of the databases (Table 1). We searched only for articles published from 1 January 2000 onwards, in line with our eligibility criteria.

Grey literature search: information sources and search terms

Two information specialists (CH and DM) searched the following grey literature sources:

Searches were conducted on November 12^th, 2019. Keywords from the database search were applied (Table 2).

Other sources

We decided at the outset that all studies found to be eligible and passing quality assessment would be reviewed for references to other potentially relevant studies.

We also checked other systematic reviews for citations of relevant studies: We checked all studies included in all systematic reviews returned by our database search, and all studies included in 17 other reviews that we knew at the outset and covered all major palliative care settings^23–39.

Study selection

Screening of titles and abstracts. Two information specialists (CH and DM) executed the searches and made the retrieved citations available in EndNote. Two team members (BMJ and PM) uploaded these citations to the online reviewer tool Covidence and reviewed titles and abstracts independently using the eligibility criteria described. Conflicts between the two reviewers were resolved using discussion and consensus.

Screening of full-text reports. Two team members (BMJ and PM) independently reviewed all studies that were advanced to full-text screening on Covidence using the eligibility criteria described above. Conflicts between the two reviewers were resolved using discussion and consensus.

Assessment of methodological quality/bias. Following agreement on eligibility, each study was assessed for methodological quality using one of a number of standardised instruments developed by the Critical Appraisal Skills Programme (CASP), which also provides recommendations for exclusion of studies. Since multiple study designs were eligible, we decided prior to data collection to use the specific CASP tool most appropriate to each study (e.g. the CASP Case Control Study Checklist, the CASP Economic Evaluation Checklist, the CASP Qualitative Checklist, and so forth).

Two team members (BMJ and PM) quality-assessed all eligible studies independently. Conflicts between the two reviewers were resolved using discussion and consensus.

Other sources. Two information specialists (CH and DM) compiled all returned grey literature in EndNote. For Google Scholar, the first 10 pages (100 items) were collated; for all other sources, all returned items were collated. One team member (RMcC or PM) reviewed each grey literature item for potential relevance to this review. Studies published in the peer-reviewed literature were discarded if they had already been returned by the database search. Where the reviewer was uncertain of relevance, s/he conferred with another team member (BMJ).

One team member (PM) performed the review of the reference lists of other systematic reviews.

Data collection process

Two team members (BMJ and PM) were to extract data independently. Conflicts between the two reviewers would be resolved using discussion and consensus.

Data items

We decided prior to data collection to extract the following data items from studies that were eligible and of sufficient quality: design (e.g. randomised controlled trial, prospective cohort study, case-control study, etc.); country of origin; care setting; model of care; level(s) of provider expertise/training; sample size; patient characteristics; carer characteristics; recruitment and sampling; ethical issues, including consent; research question; outcomes; approach to confounding; statistical methods; and findings.

Synthesis of results

We decided to perform meta-analyses of included studies where possible due to homogeneity of methods, participants, interventions, and reporting. Given the wide range of outcomes of interest, it was not possible to specify all outcome measures or synthesis methods prior to data collection.

Prior versions of this work

An earlier version of this systematic review, along with other components of the HRB/DOH-funded project, were previously published on the HRB website⁴². The only substantive change to the review between versions is that we updated the search strategy from August 2018 to November 2019. No new studies of relevance were identified.

An oral presentation of this project was made at the SPHeRE Network 5th Annual Conference, Dublin, February 2019. A poster presentation was made at the 16^th World Congress, European Association for Palliative Care, Berlin, May 2019.

Database search

Our search of nine databases is summarised in Figure 1.

Figure 1. PRISMA diagram: database search, grey literature review and study selection.

The search yielded 1,694 citations, of which 958 were duplicates. We reviewed the remaining 736 unique titles/abstracts, of which 696 did not meet the eligibility criteria. One paper was excluded at this stage due to not being in the English language; it was in Dutch⁴³.

We then reviewed the remaining 40 full texts, of which one met the eligibility criteria⁴⁴.

We reviewed this study using the cohort study checklist. Under this checklist, the first section is ‘Section A: Are the results of the study valid?’. Section A consists of two questions:

Two independent reviewers concluded that the answer to each question was ‘No’. Specifically, reviewers recorded concerns about how the comparison group were identified. The authors do not provide eligibility criteria (Question 1) or methods of recruitment/identification (Question 2). The study was therefore not advanced to full CASP assessment.

The database search therefore identified zero peer-reviewed studies evaluating the impact of out-of-hours palliative care for adults on patient/carer outcomes and/or economic outcomes.

Other sources

Our search of grey literature sources yielded 180 items, of which 108 were peer-reviewed articles already returned by the database search. The remaining 72 items were reviewed for duplicates, which were discarded (n=4). None of the remaining 68 items were found to contain evaluations relevant to this systematic review.

Our review of the citations found in other systematic reviews examined one published systematic review returned by our database search⁴⁵, and from 17 other reviews that we were already aware of^23–39. No additional eligible papers were identified.

Main results

We identified no study of sufficient quality that evaluated the effectiveness or cost-effectiveness of out-of-hours specialist or generalist palliative care for adults.

The reasons for this lack of evidence are important to understand if future research is to address an established priority for patients, carers, volunteers and professionals. Some reasons may be generic within palliative and end-of-life care. Primary research studies face practical and ethical challenges, resulting in a small evidence base in all settings relative to policy relevance^31,33. The relative newness of palliative care as a specialism is also a potential factor. For example, medical research activity is highest in the United States, where palliative care is heavily hospital-based and hospital costs are a major public policy issue⁴⁶. Yet a systematic search of the literature to 2018 identified only one prospective economic study of this model of care³⁵. Since out-of-hours services are new even within this relatively young field, it is perhaps not surprising that research is underdeveloped.

Other reasons may be specific to out-of-hours services. From an overall service perspective, out-of-hours is one component of a model of care⁴⁷. Where out-of-hours care is one element of the model under evaluation, it may not be identified specifically in reporting under keywords, MeSH terms or abstracts. There is therefore a risk of under-identification in our methods. However, any study isolating the effect of out-of-hours within a wider evaluation of palliative care provision was eligible for our review, and any study evaluating that specific effect would likely report this in the abstract. Studies most likely to go unidentified by our review are those where a service is provided on a 24-hour basis, and the out-of-hours elements were not separated in analysis. In this case, the study would not have been eligible under our criteria in any case.

Finally, it is important to note that our findings do not mean that no literature on out-of-hours palliative care exists, simply that evaluations specifically were not identified. Our review identified, but did not include as eligible, topics including general practitioner perspectives on out-of-hours palliative care^48,49; pilot programmes on, inter alia, prescribing and telehealth^50,51; and an ongoing systematic review to identify quality improvement projects in out-of-hours palliative care⁵². These studies and others advance understanding outside of our evaluative focus, as well as potentially improving future evaluative studies.

Strengths and limitations

Any systematic review is vulnerable to missing relevant material, either through mis-specification of search terms or errors in review. We minimised these risks by establishing clear PICOS and eligibility criteria prior to data collection, employing a combination of subject and information specialists in executing searches, using two reviewers independently throughout the process, and examining studies that were included in multiple other prior reviews of palliative care in different settings.

We excluded studies from prior to 2000, which in principle may have excluded relevant evidence. Time cut-offs are inherently arbitrary but in this context were deemed important, palliative care practice from over two decades ago being very different to today in all countries. Specifically, this project was commissioned by Irish policymakers ahead of a review of national policy²⁰; prior policy was published in 2001 meaning that all relevant evidence to that point ought to have been collated already and so we established 2000 as an obvious cut point. As noted in Figure 1, one study was excluded due to the time criterion⁵³. We excluded one study due to our English language criterion⁴³.

What this study adds

This study illustrates that while out-of-hours palliative care is a recognised priority for patients and policymakers, no evidence base exists on which services are beneficial for patients and worthy of health care funding.

The lack of evidence underscores the need for future studies to incorporate measurement of the effectiveness and/or cost-effectiveness of out-of-hours services. In principle there are two ways that such evaluations might be initiated. First, data are already collected by statutory bodies and other providers on existing out-of-hours services. Appropriate analyses of these data could produce the sort of evidence that this review hoped to identify, albeit statutory data tend to focus more on process than outcomes, which limits analytic scope¹⁹.

Second, original research must be conducted to collect data and evaluate out-of-hours care across its multitude of settings and practitioners. Consistent with other areas of palliative and end-of-life care research, this agenda will have to be flexible and pragmatic in matching methodological approaches to specific problems⁴¹.

Conclusion

We conducted a systematic review of peer-reviewed and grey literature in order to identify evidence on the impact of out-of-hours palliative care for adults on patient and carer outcomes, and on economic outcomes.

We searched nine databases using both information and subject specialists, and we searched grey literature, including doctoral theses and policy repositories. Our database search yielded only one relevant study, which two independent reviewers judged to be of insufficient quality to include in the review. Our search of other sources found no relevant material.

The evidence base on out-of-hours palliative care is very small relative to importance to patients and policymakers. These evidence gaps must be urgently addressed.

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Reporting guidelines

Open Science Framework: Appendix to: [Effectiveness and cost-effectiveness of out-of-hours palliative care: a systematic review]. https://doi.org/10.17605/OSF.IO/6EP9A⁵⁴.

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).