Skip to content
ALL Metrics
-
Views
37
Downloads
Get PDF
Get XML
Cite
Export
Track
Study Protocol

Multivariable prediction models for long-term outcomes after hip fracture: A protocol for a systematic review

[version 1; peer review: awaiting peer review]
PUBLISHED 09 Jun 2022
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS AWAITING PEER REVIEW

Abstract

Background: Hip fracture results in high mortality and, for many survivors, long-term functional limitations. Multivariable prediction models for hip fracture outcomes have the potential to aid clinical-decision making as well as risk-adjustment in national audits of care. The aim of this study is to identify, critically appraise and synthesise published multivariable prediction models for long-term outcomes after hip fracture.

Protocol: The systematic review will include a literature search of electronic databases (MEDLINE, Embase, Scopus, Web of Science and CINAHL) for journal articles. Search terms related to hip fracture, prognosis and outcomes will be included. Study selection criteria includes studies of people with hip fracture where the study aimed to predict one or more long-term outcomes through derivation or validation of a multivariable prediction model. Studies will be excluded if they focus only on the predictive value of individual factors, or only include patients with periprosthetic fractures, fractures managed non-surgically or younger patients. Covidence software will be used for data management. Two review authors will independently conduct study selection, data extraction and appraisal. Data will be extracted based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist. Risk of bias assessment will be conducted using the Prediction model Risk of Bias Assessment Tool (PROBAST). Characteristics and results of all studies will be narratively synthesised and presented in tables. Where the same model has been validated in multiple studies, a meta-analysis of discrimination and calibration will be conducted.

Conclusions: This systematic review will aim to identify multivariable models for hip fracture outcome prognosis that have been derived using high quality methods. Results will highlight if current models have the potential for further assessment for use in both clinical decision making and improving methods of national hip fracture audits.

PROSPERO registration: CRD42022330019 (25th May 2022).

Keywords

hip fracture, femoral fracture, prediction, patient-reported outcomes, mortality, systematic review protocol

Introduction

Hip fracture is a serious event, unfortunately resulting in long-term functional limitations for many survivors15. It is estimated that one in five patients with a hip fracture will die in the first year6. Within this timeframe, less than half recover their pre-fracture level of mobility, while approximately 40% experience a hospital readmission and 30% recurrent falls1,5.

Much focus has been placed in recent literature on the ability to predict outcomes, particularly mortality, for individuals early in the course of hip fracture7. Accurate prediction could aid clinical decision making and prognosis discussions with family members in the context of gaining informed consent for procedures and practices. Assessing functional prognosis could also help to identify patients that require further supports in the long-term to enable them to continue to live as independently as possible. Several studies have also identified outcomes that are important in the context of recovery from hip fracture and a number of core outcome sets have been developed. Non-mortality outcomes frequently cited include hospital readmission, re-operation, pain, residence, osteoporosis medication use, mobility, activities of daily living and quality of life1,8,9.

In many countries worldwide, national hip fracture audits and registries have been established with the aim of encouraging and incentivising best practice care in the acute period after hip fracture10. A feature of many of these audits is the publication of performance of individual hospitals in comparison to each other. With some exceptions, these data are generally presented without any risk adjustment or accounting for varying case-mix or levels of baseline predictors of poor prognosis among patient cohorts. This has the potential to mask or falsely amplify care inequity within countries. Furthermore, several national audits are beginning to record long-term mortality and functional outcomes at time points of 30 days, 120 days and one year after fracture11. These data, in particular patient-reported outcomes, are prone to incompleteness that does not occur randomly3,12. Accurate prognostic models could be used in statistical imputation techniques to account for missing data13.

Multivariable prediction models are defined by the Cochrane Prognosis Methods group as a statistical “combination of multiple predictors from which risks of a specific endpoint can be calculated for individual patients”14. To be useful for application in clinical practice or to aid risk adjustment and account for missing data, they should undergo several stages of development, including derivation in one sample with external validation in a separate dataset. Several high-quality guidance documents have been recently published to inform transparent reporting, data extraction and critical appraisal in this methodological field1517.

A systematic review conducted in 2015 identified 25 tools that aimed to predict morbidity and mortality in persons with hip fracture7. Several of the tools explored, however, were generic and had not been derived in persons with hip fracture using best-practice statistical methods. Furthermore, no critical appraisal of tools was conducted and some of the included studies reported no validation metrics. Since publication of the review, there has also been a proliferation of research in this field. The aim of this systematic review is to identify, critically appraise and synthesise published multivariable prediction models for long-term outcomes occurring after hospital discharge or at least 30 days after hip fracture.

Protocol

Study design

A systematic review of studies that have derived or validated multivariable prediction models for long-term outcomes after hip fracture will be conducted. This will include mortality and specified non-mortality outcomes after hospital discharge or at a fixed time-point at least 30 days after fracture. This protocol follows the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) reporting guidelines18,19. The systematic review protocol has been registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 25th May 2022 (registration number CRD42022330019). If applicable, important protocol updates will be registered via PROSPERO. The review will be carried out and reported in accordance with the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist16 and the PRISMA checklist20.

Selection criteria

Studies will be included if:

  • Participants were selected based on a diagnosis of hip fracture. Hip fracture will be defined as a fracture of the femur in the femoral head, neck, pertrochanteric or subtrochanteric regions, including periprosthetic or pathological fractures21. Studies of multiple conditions will be included if participants are selected due to hip fracture and hip fracture data are presented separately.

  • The aim of the study is to derive or validate a multivariable prediction model or clinical prediction rule, where the derivation of the model/ rule was conducted in a hip fracture population and was either presented in the paper or was previously published in peer-reviewed literature. We define a multivariable prediction model based on the Cochrane Prognosis Methods group14, as a statistical “combination of multiple predictors from which risks of a specific endpoint can be calculated for individual patients”14.

  • The aim of the study is to predict one or more long-term outcomes. Outcome domains for inclusion have been informed by hip fracture core outcome sets and the practice of international hip fracture registries8,9,11. They include mortality, hospital readmission, re-operation, pain, residence, osteoporosis medication use, mobility, activities of daily living and quality of life. Other specific outcomes will be considered for inclusion if they are sufficiently relevant to the trajectory of hip fracture recovery. “Long-term” is defined as after discharge from an inpatient setting or at a fixed time-point at least 30 days after fracture, admission to hospital or surgery.

  • The model/ rule is designed to be applied during the acute phase after hip fracture (within 30 days of fracture, admission or surgery or within the acute hospital setting).

  • The study design is a prospective or retrospective cohort study. Control arms of randomised controlled trials will also be considered for inclusion where relevant.

  • They are published in peer-reviewed journal articles.

  • They are reported in any language.

Studies will be excluded if any of the following apply:

  • Only patients with periprosthetic fractures or fractures managed non-surgically are included.

  • The study focusses only on younger people (under 60 years old).

  • Only short-term outcomes were collected (less than 30 days after fracture or only within the initial inpatient setting).

  • Only the prognostic value of individual factors is presented.

  • Rules or models were derived in non-hip fracture populations.

  • Cross-sectional studies or case control design.

  • Study published only in abstract or thesis form.

Search strategy

A search of electronic databases will be performed up to May 2022, including MEDLINE (Ovid) (RRID:SCR_002185), EMBASE (RRID:SCR_001650), Scopus, Web of Science and CINAHL. Keywords and controlled vocabulary terms (e.g., MeSH (RRID:SCR_004750) and Emtree ) will be used to combine the topics of the population (hip fracture), exposure (prediction model) and outcomes (mortality and specified non-mortality outcomes). Hip fracture terms were adapted from other relevant systematic reviews2224. Validated search strings for finding prognostic and diagnostic prediction studies were also incorporated25,26. Outcome search terms were informed by studies on hip fracture core outcome sets and common complications post hip fracture surgery8,9,27. No date or language restrictions will be applied. Citation lists of previous systematic reviews and included studies will be hand-searched. The search strategy for MEDLINE (Ovid) is presented in Table 1. The full search strategy for all sources can be found as Extended data28.

Table 1. Search strategy for MEDLINE (accessed via Ovid).

NumberSearch terms
1.exp Femoral Fractures/
2.((hip or hips or cervical or femoral$ or femur$ or intracapsular or "intra capsular" or subcapital or "sub capital" or
transcervical or "trans cervical" or basicervical or "basi cervical" or extracapsular or "extra capsular" or trochant$ or
subtrochant$ or pertrochant$ or intertrochant$) adj5 (fracture$ or break$ or broke$)).ti,ab,kf.
3.(((head or neck or proximal) adj5 (fracture$ or break$ or broke$)) and (femoral$ or femur$)).ti,ab,kf.
4.or/1-3
5.("clinical prediction" OR "prediction model*" OR "prognostic model*" OR "risk model" OR "risk score" OR "risk
adjust*" "decision rule*" OR "diagnostic accuracy" OR "diagnostic rule*" OR "diagnostic score*" OR "diagnostic
value" OR "predictive outcome*" OR "predictive rule*" OR "predictive score*" OR "predictive value" OR "predictive
risk*" OR "prediction outcome*" OR "prediction rule*" OR "prediction score*" OR "prediction value*" OR "prediction
risk*" OR "risk score*" OR "validation decision*" OR "validation rule*" OR "validation score*" OR "external validation"
OR (derivation AND validation) OR (derived AND validated) OR (sensitivity AND specificity)).ti,ab,kf.
6.("Stratification" or "Discrimination" or "Discriminate" or "c-statistic" or "c statistic" or "Area under the curve" or "AUC"
or "Calibration" or "Indices" or "Algorithm").ti,ab,kf.
7.'receiver operating characteristic'/de
8or/5-7
9.4 AND 8
10."Hip Fracture Score".mp.
11.9 OR 10.
12.(Mortality or death or died or survival or readmi* or re-admi* or rehospitali* or re-hospitali* or re-operat* or
reoperat* or failure or revision or periprosthe* or peri-prosthe* or peri-implant or osteonecro* or "avascular
necro*" or non-union or "non union" or extrusion or cutout or cut-out or re-fracture or refracture or "second
fracture" or recurrent or subsequent or dislocation or infect* or complicat* or adverse or event* or reaction or
transfusion or thrombo* or embol* or residen* or living or home or location or lived or function* or adhere* or
medicat* or persist* or pain* or mobil* or walk* or ambulat* or quality or QOL or activit*).ti,ab,kf.
1311 AND 12

Study selection

All records from database searching will be uploaded to Covidence Systematic Review Software (RRID:SCR_016484)29. After removal of duplicates, titles and abstracts will be screened by two review authors. Full text articles will be added to Covidence for all records deemed to be potentially eligible and these will be reviewed by two authors independently. Reasons will be recorded for excluding studies at the full-text stage. If disagreements arise regarding eligibility, they will be resolved by discussion or a third author. The study selection process will be reported using a PRISMA flow diagram20.

Data extraction

A data-extraction form will be developed based on the CHARMS checklist16, with the addition of subject-specific fields. This will be piloted independently by two authors on two included papers and if necessary the form will be edited for clarity. Data will then be independently extracted by two reviewers using Covidence software. Disagreements will be resolved by discussion and, if necessary, by referral to a third reviewer. Items for extraction will depend on whether the paper describes a derivation/ development or validation study. Table 2 provides further details.

Table 2. Data items for extraction from derivation and validation studies.

ItemsExamplesDerivation
studies
Validation
studies
Study
characteristics
Country, yearYesYes
Study designProspective cohort, retrospective cohort, trialYesYes
Patient
characteristics
Age, sex, fracture typeYesYes
Predicted
outcome
Outcome (e.g., mortality, readmission, re-operation, residence, pain, mobility
or functional ability) and time-point (e.g., 30-days, one year)
YesYes
Model
development
Sample size, type of model, handling of continuous variables, selection of
variables, missing data, method of internal validation
YesNo
Model
presentation
Full regression model, simplified model, risk groupsYesNo
Model
performance
Measures of discrimination (e.g., area under the Receiver Operating
Characteristic curve statistic), calibration (e.g., ratio of Observed to Expected
events) and clinical utility (e.g., sensitivity/ specificity) including measures of
uncertainty (e.g., standard errors and 95% confidence intervals)
YesYes

Risk of bias in individual studies

Risk of bias assessment will be performed using the Prediction model Risk of Bias Assessment Tool (PROBAST)17, which consists of 20 signalling questions within four domains (participants, predictors, outcome and analysis)17. Included studies will be critically appraised by two authors independently using PROBAST and consensus reached through discussion. The models will be classed as low, high or unclear risk of bias.

Data synthesis

Studies will be narratively synthesised with results and characteristics being presented in tables.

Where the same model has been validated in multiple studies, a meta-analysis of area under the Receiver Operating Characteristic curve statistics (AUC) and ratios of Observed to Expected events (O:E) will be conducted. An AUC of 0.5 represents chance, values between 0.7 and 0.9 represent moderate discrimination and a value of 1 represents perfect discrimination15. O:E ratios are a measure of calibration, which is the agreement between model prediction and observed outcomes. An O:E value of between 0.8 and 1.2 represents good calibration30. Where specific measures or measures of uncertainty are not reported methods described by Debray et al., will be used to estimate measures30,31. A random effects meta-analyses of O:E and AUC values will be conducted with REML estimation using the metaan procedure in Stata 17 (RRID:SCR_012763) (Stata Corp, College Station TX)31,32. This will be conducted as recommended on the log scale for O:E ratios and logit scale for AUC values31,33. The proportion of heterogeneity due to between-study variation will be estimated using the I2 statistic.

Study status

Pilot searches were conducted in April 2022. Main searches were conducted in May 2022. Title and abstract screening and final study selection will be completed from May to July 2022. Data extraction and quality appraisal will be completed from August to October 2022.

Conclusions

The aim of the proposed systematic review is to identify, critically appraise and synthesise published multivariable prediction models for long-term outcomes occurring after hospital discharge or at least 30 days after hip fracture. The study will aim to identify models that have been derived using high quality methods and will establish their stage of development (derived or validated). It will highlight models that would be appropriate for impact analysis and assessment of use in clinical practice using randomised trials. Most significantly, these models could be useful for national hip fracture audits and registries in reporting risk-adjusted estimates of hospital performance and in accounting for missing data during the assessment of longer-term outcomes. This could facilitate further developments in quality improvement for this specific frail, older population who are vulnerable to poor outcomes. Identified models could also be useful for individual clinicians in decision-making and prognosis discussions with family members and to help to identify patients with hip fracture that require further supports in the long-term to enable them to continue to live as independently as possible.

Comments on this article Comments (1)

Version 1
VERSION 1 PUBLISHED 09 Jun 2022
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
VIEWS
665
 
downloads
37
Citations
CITE
how to cite this article
Walsh ME, Kristensen PK, Hjelholt TJ et al. Multivariable prediction models for long-term outcomes after hip fracture: A protocol for a systematic review [version 1; peer review: awaiting peer review]. HRB Open Res 2022, 5:44 (https://doi.org/10.12688/hrbopenres.13575.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Comments on this article Comments (1)

Version 1
VERSION 1 PUBLISHED 09 Jun 2022
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Are you a HRB-funded researcher?

Submission to HRB Open Research is open to all HRB grantholders or people working on a HRB-funded/co-funded grant on or since 1 January 2017. Sign up for information about developments, publishing and publications from HRB Open Research.

You must provide your first name
You must provide your last name
You must provide a valid email address
You must provide an institution.

Thank you!

We'll keep you updated on any major new updates to HRB Open Research

Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.