Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review

Niamh A. Merriman; Mary E. Walsh; Niamh O'Regan; Marie Carrigan; Pamela Hickey; Louise Brent; Catherine Blake

doi:10.12688/hrbopenres.13894.2

Home Browse Diagnostic test accuracy of screening tools for the detection of neurocognitive...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Revised

Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review

[version 2; peer review: 1 approved, 2 approved with reservations]

Niamh A. Merriman ¹, Mary E. Walsh¹, Niamh O'Regan², [...] Marie Carrigan³, Pamela Hickey⁴, Louise Brent⁴, Catherine Blake¹

Niamh A. Merriman ¹, Mary E. Walsh¹, [...] Niamh O'Regan², Marie Carrigan³, Pamela Hickey⁴, Louise Brent⁴, Catherine Blake¹

PUBLISHED 03 Sep 2024

Author details Author details

¹ School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
² University Hospital Waterford, Waterford, Ireland
³ Health Information and Quality Authority, Dublin, Ireland
⁴ National Office of Clinical Audit, Dublin, Ireland

Niamh A. Merriman
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Mary E. Walsh
Roles: Conceptualization, Methodology, Writing – Review & Editing

Niamh O'Regan
Roles: Conceptualization, Methodology, Writing – Review & Editing

Marie Carrigan
Roles: Conceptualization, Methodology, Writing – Review & Editing

Pamela Hickey
Roles: Conceptualization, Methodology, Writing – Review & Editing

Louise Brent
Roles: Conceptualization, Methodology, Writing – Review & Editing

Catherine Blake
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background

Neurocognitive disorders (NCDs), including delirium, cognitive impairment, or dementia are prevalent in up to 39% of older adults in acute care, particularly older trauma patients. Undiagnosed NCDs result in poor outcomes, such as increased incidence of depressive symptoms, longer length of stay, and mortality.

Objective

This study aims to identify the diagnostic test accuracy of screening tools for the detection of NCDs in older trauma patients in acute settings.

Design

Systematic review protocol.

Literature search

Electronic databases (MEDLINE, Embase, CINAHL, PsycInfo, Cochrane Library) will be searched for journal articles. Search terms related to NCDs, delirium and cognitive screening tools, and diagnostic accuracy will be included.

Study selection criteria

Cross-sectional, prospective, or retrospective cohort studies of adults aged ≥60 post-trauma, in an acute setting, will be included where the study aimed to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia against a reference standard of a clinical decision, based on standardised diagnostic criteria or a validated tool.

Data synthesis

Two review authors will conduct study selection, data extraction, and appraisal. Data will be extracted based on the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) checklist. Studies will be assessed for methodological quality by two independent review authors using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. Narrative summaries will be generated describing risk of bias and concerns regarding applicability. Quantitative synthesis of study findings will be conducted.

Conclusion

This systematic review will aim to identify screening tools with the best diagnostic accuracy for detection of 1) delirium and 2) cognitive impairment or dementia in adults aged ≥60 post-trauma in acute care settings. Results will inform clinical practice to enhance the probability of patients with NCDs receiving appropriate care and management.

Registration

PROSPERO https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42024518730 (11/03/2024).

Keywords

systematic review protocol, diagnostic accuracy, cognition, delirium, older adults

Corresponding author: Niamh A. Merriman

Competing interests: No competing interests were disclosed.

Grant information: This review is funded by the Health Research Board Ireland through the Applying Research into Policy and Practice fellowship (ARPP-2023-002) as part of the larger project entitled “The HipCog Study: evaluating the impact of cognitive impairment on outcomes for
older adults with hip fracture”. The funder will have no input on the interpretation or publication of the study results.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2024 Merriman NA et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Merriman NA, Walsh ME, O'Regan N et al. Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review [version 2; peer review: 1 approved, 2 approved with reservations]. HRB Open Res 2024, 7:33 (https://doi.org/10.12688/hrbopenres.13894.2) First published: 04 Jun 2024, 7:33 (https://doi.org/10.12688/hrbopenres.13894.1) Latest published: 03 Sep 2024, 7:33 (https://doi.org/10.12688/hrbopenres.13894.2)

Revised Amendments from Version 1

The protocol has been revised following peer review. We have clarified the screening tool inclusion criteria to include self-report tools or those administered by any healthcare professional. We have clarified the study selection criteria to exclude studies in which participants are identified in environments other than acute care (e.g., primary care, memory clinic, outpatient, rehabilitation settings, or community settings). We have included head trauma as a category for sub-group analysis of the study population.

See the authors' detailed response to the review by Carole Fogg
See the authors' detailed response to the review by Parichita Choudhury and Andrew Ho

Introduction

Neurocognitive disorders (NCDs), such as delirium, cognitive impairment, or dementia are prevalent in 21% to 39% of older adults in acute care^1,2. In general, older adults post-trauma (i.e., following physical trauma) in acute care have complex ongoing healthcare needs, with high prevalences of delirium and cognitive impairment³, particularly in patients with hip fracture^4,5. Trauma is defined as a physical injury of sudden onset and severity which requires immediate medical attention⁶. Delirium is an acute and fluctuating NCD, characterised by impaired attention, altered awareness, and other cognitive and neuropsychiatric disturbances⁷. Cognitive impairment is an NCD, characterised by a disruption to some cognitive function such as memory, though independence in functional abilities is preserved^8,9. Dementia is an NCD in which there is deterioration in cognitive function beyond that which might be expected from the usual consequences of biological ageing and which affects a person’s ability to function independently¹⁰. Despite the high prevalence of NCDs, 56% to 62% of cases are undiagnosed or undetected by healthcare professionals^11,12. Undiagnosed NCDs can lead to poor outcomes, such as increased incidence of depressive symptoms, longer length of stay, and mortality^13,14.

Over half of patients admitted to hospital following physical trauma are over 60 years old¹⁵, with the majority of injuries resulting from low-energy falls of less than 2 metres¹⁵. Other types of trauma include motor vehicle collisions, falls from more than 2 metres, and intentional injury^3,15. Hip fracture is the most serious consequence of falls among adults¹⁶, and older adults with hip fracture are at increased risk of NCDs. A systematic review of observational studies found that 19% of people with hip fracture meet formal diagnostic criteria for dementia, and 42% have cognitive impairment¹⁷. It is expected that the number of people living with dementia will double worldwide to 75 million by 2030 and 131.5 million in 2050¹⁸. Therefore, the number of people with cognitive impairment or dementia and hip fracture will increase over the next 25 years¹⁹. Older adults with cognitive impairment or dementia are likely to have a complex hip fracture care pathway, due to their greater risk of poor outcomes and need for person-centred, tailored support¹⁹. The high incidence of delirium (up to 30%) among hip fracture patients impacts on cognitive screening^5,20,21. Delirium itself can result in increased length of stay, long-term cognitive and functional decline, increased risk of discharge to long-term care, higher mortality, and patient and carer distress. Although management of delirium is an essential facet of acute geriatric care, older adults with pre-existing cognitive impairment without delirium are at high risk of being undiagnosed²².

Given the breadth of available cognition and delirium screening tools, the variability in recommended cut-off thresholds, and the adverse outcomes for older adults with NCDs post-trauma in acute care, it is timely to conduct a diagnostic accuracy review of screening tests for cognitive impairment and delirium in older adults. The findings will contribute greatly to clinical practice and policy making by identifying the 1) delirium and 2) cognitive/dementia screening tools with the best diagnostic test accuracy for use with older adults post-trauma in acute care to enhance the probability of patients with NCDs receiving appropriate care and management.

Protocol

Study design

A systematic review of studies that describe the accuracy of screening tools for 1) delirium and 2) cognitive impairment or dementia in older adults admitted to acute care following physical trauma compared with a clinical reference standard based on the Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria (version IV or V)^7,23, the International Classification of Diseases (ICD)²⁴, or a validated tool will be conducted. This protocol follows the PRISMA-P Reporting Guidelines²⁵ (see Extended data²⁶). The systematic review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 11 March 2024 (https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42024518730). If applicable, important protocol updates will be registered via PROSPERO. The review will be carried out and reported in accordance with two guidelines: the Standards for Reporting of Diagnostic Accuracy Studies (STARD 2015)²⁷ and the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) checklist²⁸.

Selection criteria

Studies will be included if:

Population consists of older adults aged 60 years or above or has median age above 70 years in an acute care setting following physical trauma as the main study group or as a clearly defined subgroup. Trauma is defined as a physical injury of sudden onset and severity which requires immediate medical attention⁶.
The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.
The study aims to diagnose 1) delirium or 2) cognitive impairment or dementia using a clinical decision, based on standardised diagnostic criteria or a validated tool.
The study design is a cross-sectional, prospective, or a retrospective cohort study. Control arms of randomised controlled trials will also be considered for inclusion where relevant.
They are published in peer-reviewed journal articles.
They are reported in any language.
The study provides sufficient data to construct a two-by-two table.

Studies will be excluded if any of the following apply:

The study focuses only on younger people (under 60 years).
The population does not consist of physical trauma patients.
Case reports or case-control design.
Studies in which participants are identified in environments other than acute care (e.g., primary care, memory clinic, outpatient, rehabilitation settings, or community settings) will be excluded.

Search strategy

A comprehensive search strategy will be developed in collaboration with a librarian and a search of electronic databases will be conducted, including MEDLINE (Ovid), Embase (Elsevier), PsycInfo (EBSCO), CINAHL (EBSCO), and the Cochrane Library (Wiley). Search terms related to neurocognitive disorders, delirium and cognitive screening tools, and diagnostic test accuracy²⁹ will be included. No date or language restrictions will be applied. Citation lists of previous systematic reviews and included studies will be hand-searched. See Extended data for full search strategy for all sources³⁰.

Study selection

All citations identified from the search strategy will be exported to EndNote (Version X20) for reference management, where duplicates will be identified and removed. Using Covidence Systematic Review Software³¹, two reviewers will independently review the titles and abstracts of the remaining citations to identify those for full-text review. The full texts will be obtained and independently evaluated by two reviewers, applying the defined inclusion and exclusion criteria. Where disagreements occur, discussions will be held to reach consensus and where necessary, a third reviewer will be involved. Citations excluded during the full-text review stage will be documented alongside the reasoning for their exclusion and included in the PRISMA flow diagram³². Study authors will be contacted if further information is required to determine eligibility.

Data extraction

Data extraction will be performed by one reviewer and checked for accuracy and omissions by a second. Where disagreements occur, discussions will be held to reach consensus and where necessary, a third reviewer will be involved. Data will be extracted on: 1) title, first author, year and country; ii) type of study; iii) setting; iv) study population; v) comorbid illness or illness severity, if reported; vi) patient demographics; vii) number of patients in total, that is a) n with delirium, n without delirium and b) n with mild cognitive impairment or dementia, n without mild cognitive impairment or dementia; viii) details of screening tool administration (timing, assessor, etc.) and the condition reference standard process; ix) details of screening tool implementation (acceptability, inter-rater reliability, training, etc.); x) test accuracy data, extracted to a two-by-two table (number of true positives, false positives, true negatives and false negatives); xi) statistics used, including adjustments made; xii) conclusion of study; and, xiii) comments on study quality.

Risk of bias in individual studies

Risk of bias assessment will be conducted using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool³³. Narrative summaries will be provided describing risk of bias (high, low, or unclear) and concerns regarding applicability. All included studies will be appraised independently by two designated authors. Disagreements will be resolved through discussion and a third author will be involved if necessary.

Data synthesis

Characteristics and results of all studies will be narratively synthesised and presented in tables. Quantitative synthesis of the findings from the included studies will be conducted. From the data extracted from the included studies, 2 × 2 tables will be constructed for presence or absence of delirium or dementia/cognitive impairment. Summary estimates of sensitivity and specificity with 95% confidence intervals will be calculated. Meta-analysis will be conducted if two studies reporting on the same screening instrument are included. A random effects bivariate meta-analysis of sensitivity and specificity will be conducted using Diagnostic Test Accuracy Meta-Analysis software, version v2.1.1 (https://crsu.shinyapps.io/MetaDTA/)^34,35. A sensitivity analysis will be performed on studies which were deemed to have an overall low risk of bias. Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma) and on type of reference standard (i.e., standardised diagnostic criteria or a validated tool).

Study status

Pilot searches were conducted in February 2024. Main searches were conducted in March 2024. Title and abstract screening and final study selection will be completed from March to July 2024. Data extraction and quality appraisal will be completed from August to October 2024.

Conclusions

This systematic review will aim to identify, critically appraise, and synthesise the diagnostic accuracy of available screening tools for detection of 1) delirium and 2) cognitive impairment or dementia in adults aged ≥60 in acute care settings following physical trauma. The findings will inform clinical practice and policy making to enhance the probability of patients with NCDs receiving appropriate care and management.

Data availability

Underlying data

No data are associated with this article.

Extended data

Zenodo: Extended Data for ‘Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review’. https://doi.org/10.5281/zenodo.10978778³⁰.

This project contains the following extended data:

- Full Search Strategy

Data is available under the terms of Creative Commons Attribution 4.0 International (CC by 4.0)

Reporting guidelines

Zenodo: PRISMA-P Checklist for ‘Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review’. https://doi.org/10.5281/zenodo.10978882²⁶.

Contributions

NAM is the guarantor and drafted the initial manuscript. All authors contributed to the development of the search strategy, the selection criteria, and the risk of bias assessment strategy. All authors read, provided feedback, and approved the final manuscript.

Faculty Opinions recommended

References

1. Reynish EL, Hapca SM, De Souza N, et al.: Epidemiology and outcomes of people with dementia, delirium, and unspecified cognitive impairment in the general hospital: prospective cohort study of 10,014 admissions. BMC Med. 2017; 15(1): 140. PubMed Abstract | Publisher Full Text | Free Full Text
2. Pedone C, Ercolani S, Catani M, et al.: Elderly patients with cognitive impairment have a high risk for functional decline during hospitalization: the GIFA study. J Gerontol A Biol Sci Med Sci. 2005; 60(12): 1576–80. PubMed Abstract | Publisher Full Text
3. Lenartowicz M, Parkovnick M, McFarlan A, et al.: An evaluation of a proactive geriatric trauma consultation Service. Ann Surg. 2012; 256(6): 1098–101. PubMed Abstract | Publisher Full Text
4. National Office of Clinical Audit: Irish hip fracture database national report 2020. Dublin, Ireland, 2021. Reference Source
5. National Office of Clinical Audit: Irish hip fracture database national report 2017 - 2021. Dublin, Ireland, 2022. Reference Source
6. Government of Ireland: A trauma system for Ireland: report of the trauma steering group. 2018; [updated 26 October 2020]. Reference Source
7. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Washington, DC: American Psychiatric Association, 2013. Publisher Full Text
8. Lezak MD, Howieson DB, Bigler ED, et al.: Neuropsychological assessment. 5th ed. Oxford; New York: Oxford University Press, 2012. Reference Source
9. Petersen RC, Lopez O, Armstrong MJ, et al.: Practice guideline update summary: mild cognitive impairment: report of the guideline development, dissemination, and implementation subcommittee of the American Academy of Neurology. Neurology. 2018; 90(3): 126–35. PubMed Abstract | Publisher Full Text | Free Full Text
10. World Health Organization: WHO dementia factsheet. World Health Organization, 2023; [updated 15 March 2023]. Reference Source
11. Jackson TA, Gladman JRF, Harwood RH, et al.: Challenges and opportunities in understanding dementia and delirium in the acute hospital. PLoS Med. 2017; 14(3): e1002247. PubMed Abstract | Publisher Full Text | Free Full Text
12. Nowroozpoor A, Dussetschleger J, Perry W, et al.: Detecting cognitive impairment and dementia in the emergency department: a scoping review. J Am Med Dir Assoc. 2022; 23(8): 1314.e31–1314.e88. PubMed Abstract | Publisher Full Text | Free Full Text
13. Fogg C, Griffiths P, Meredith P, et al.: Hospital outcomes of older people with cognitive impairment: an integrative review. Int J Geriatr Psychiatry. 2018; 33(9): 1177–97. PubMed Abstract | Publisher Full Text | Free Full Text
14. Tran J, Nimojan T, Saripella A, et al.: Rapid cognitive assessment tools for screening of mild cognitive impairment in the preoperative setting: a systematic review and meta-analysis. J Clin Anesth. 2022; 78: 110682. PubMed Abstract | Publisher Full Text
15. National Office of Clinical Audit: Major trauma audit national report 2021. Dublin, Ireland, 2023. Reference Source
16. Smith T, Hameed Y, Cross J, et al.: Assessment of people with cognitive impairment and hip fracture: a systematic review and meta-analysis. Arch Gerontol Geriatr. 2013; 57(2): 117–26. PubMed Abstract | Publisher Full Text
17. Seitz DP, Adunuri N, Gill SS, et al.: Prevalence of dementia and cognitive impairment among older adults with hip fractures. J Am Med Dir Assoc. 2011; 12(8): 556–64. PubMed Abstract | Publisher Full Text
18. Alzheimer's Disease International: Dementia statistics. Alzheimer's Disease International, 2020. Reference Source
19. Smith TO, Gilbert AW, Sreekanta A, et al.: Enhanced rehabilitation and care models for adults with dementia following hip fracture surgery. Cochrane Database Syst Rev. 2020; 2(2): CD010569. PubMed Abstract | Publisher Full Text | Free Full Text
20. Johansen A, Golding D, Brent L, et al.: Using national hip fracture registries and audit databases to develop an international perspective. Injury. 2017; 48(10): 2174–2179. PubMed Abstract | Publisher Full Text
21. Smith TO, Cooper A, Peryer G, et al.: Factors predicting incidence of post-operative delirium in older people following hip fracture surgery: a systematic review and meta-analysis. Int J Geriatr Psychiatry. 2017; 32(4): 386–96. PubMed Abstract | Publisher Full Text
22. de Moraes GS, Curiati PK, Avelino-Silva TJ, et al.: Aren't you forgetting something? Cognitive screening beyond delirium in the emergency department. J Am Geriatr Soc. 2023; 71(3): 987–91. PubMed Abstract | Publisher Full Text
23. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-IV. Washington DC: American Psychiatric Association, 1994. Reference Source
24. World Health Organization: International Classification of Diseases (ICD). World Health Organization, 2022; [updated 1 January 2022]. Reference Source
25. Moher D, Shamseer L, Clarke M, et al.: Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text
26. Merriman NA, Walsh ME, O'Regan N, et al.: PRISMA-P Checklist for ‘Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: a protocol for a systematic review’. Zenodo. 2024.
27. Cohen JF, Korevaar DA, Altman DG, et al.: STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016; 6(11): e012799. PubMed Abstract | Publisher Full Text | Free Full Text
28. McInnes MDF, Moher D, Thombs BD, et al.: Preferred Reporting Items for a Systematic review and Meta-analysis of Diagnostic Test Accuracy studies: the PRISMA-DTA statement. JAMA. 2018; 319(4): 388–96. PubMed Abstract | Publisher Full Text
29. Scottish Intercollegiate Guidelines Network: Search filters. Reference Source
30. Merriman NA, Walsh ME, O'Regan N, et al.: Extended data for 'Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: a protocol for a systematic review'. Zenodo. 2024.
31. Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation.
32. Moher D, Liberati A, Tetzlaff J, et al.: Preferred Reporting Items for Systematic reviews and Meta-Analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text
33. Whiting PF, Rutjes AWS, Westwood ME, et al.: QUADAS-2: a revised tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann Intern Med. 2011; 155(8): 529–36. PubMed Abstract | Publisher Full Text
34. Freeman SC, Kerby CR, Patel A, et al.: Development of an interactive web-based tool to conduct and interrogate Meta-analysis of Diagnostic Test Accuracy studies: MetaDTA. BMC Med Res Methodol. 2019; 19(1): 81. PubMed Abstract | Publisher Full Text | Free Full Text
35. Patel A, Cooper N, Freeman S, et al.: Graphical enhancements to summary receiver operating characteristic plots to facilitate the analysis and reporting of Meta-analysis of Diagnostic Test Accuracy data. Res Synth Methods. 2021; 12(1): 34–44. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 04 Jun 2024

Author details Author details

Niamh A. Merriman
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Mary E. Walsh
Roles: Conceptualization, Methodology, Writing – Review & Editing

Niamh O'Regan
Roles: Conceptualization, Methodology, Writing – Review & Editing

Marie Carrigan
Roles: Conceptualization, Methodology, Writing – Review & Editing

Pamela Hickey
Roles: Conceptualization, Methodology, Writing – Review & Editing

Louise Brent
Roles: Conceptualization, Methodology, Writing – Review & Editing

Catherine Blake
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

This review is funded by the Health Research Board Ireland through the Applying Research into Policy and Practice fellowship (ARPP-2023-002) as part of the larger project entitled “The HipCog Study: evaluating the impact of cognitive impairment on outcomes for
older adults with hip fracture”. The funder will have no input on the interpretation or publication of the study results.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 03 Sep 2024, 7:33

https://doi.org/10.12688/hrbopenres.13894.2

version 1

Published: 04 Jun 2024, 7:33

https://doi.org/10.12688/hrbopenres.13894.1

© 2024 Merriman NA et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

383

downloads

Citations

SEE MORE DETAILS

CITE

how to cite this article

Merriman NA, Walsh ME, O'Regan N et al. Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review [version 2; peer review: 1 approved, 2 approved with reservations]. HRB Open Res 2024, 7:33 (https://doi.org/10.12688/hrbopenres.13894.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 03 Sep 2024

Revised

Views

Reviewer Report 27 Sep 2024

Carole Fogg, University of Southampton, Southampton, England, UK

Approved

https://doi.org/10.21956/hrbopenres.15335.r42249

Many thanks for the helpful responses. I ... Continue reading

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 25 Sep 2024

Joyce Chan, School of Public Health and Primary Care, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, Hong Kong

Approved with Reservations

https://doi.org/10.21956/hrbopenres.15335.r42258

This protocol aims to evaluate the performance of screening tools for the identification of NCDs in older persons with acute trauma. My comments are as follows:

In the Introduction, the authors may consider adding relevant studies or findings about the current assessment tools, or the relevant systematic reviews related to this topic.

In the methods, I suggest separating the presentation of "cognitive impairment" and "dementia". It is because the definition and diagnosis of "cognitive impairment" and "dementia" should be different.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Research Interests:mild cognitive impairment, dementia, depression, cogntive assessment, psychosocial interventions for older adults

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 04 Jun 2024

Views

Reviewer Report 05 Jul 2024

Carole Fogg, University of Southampton, Southampton, England, UK

Approved with Reservations

https://doi.org/10.21956/hrbopenres.15237.r40571

This systematic review protocol provides a clear overview of the study methods and background to the study.
My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.
Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Older adults health and care, epidemiology.

CITE

Report a concern

Author Response 03 Sep 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

03 Sep 2024

Author Response
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear ... Continue reading
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear overview of the study methods and background to the study.

Response: Thank you for reviewing this protocol. We appreciate the feedback on clarity of the description of the methodology and study background.

My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.

Response: Thank you for this comment. The condition of interest in this instance is a contemporaneous diagnosis of cognitive impairment or dementia. Similar to previous reviews (https://doi.org/10.1177/23969873211042192), it is not our intention to conduct separate comparisons based on whether the reference standard is a clinical dementia diagnosis or clinical diagnosis of cognitive impairment. We will however conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Response: We agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to: “Studies in which participants are identified in environments other than acute care”
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear overview of the study methods and background to the study.

Response: Thank you for reviewing this protocol. We appreciate the feedback on clarity of the description of the methodology and study background.

My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.

Response: Thank you for this comment. The condition of interest in this instance is a contemporaneous diagnosis of cognitive impairment or dementia. Similar to previous reviews (https://doi.org/10.1177/23969873211042192), it is not our intention to conduct separate comparisons based on whether the reference standard is a clinical dementia diagnosis or clinical diagnosis of cognitive impairment. We will however conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Response: We agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to: “Studies in which participants are identified in environments other than acute care”
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 03 Sep 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

03 Sep 2024

Author Response
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear ... Continue reading
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear overview of the study methods and background to the study.

Response: Thank you for reviewing this protocol. We appreciate the feedback on clarity of the description of the methodology and study background.

My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.

Response: Thank you for this comment. The condition of interest in this instance is a contemporaneous diagnosis of cognitive impairment or dementia. Similar to previous reviews (https://doi.org/10.1177/23969873211042192), it is not our intention to conduct separate comparisons based on whether the reference standard is a clinical dementia diagnosis or clinical diagnosis of cognitive impairment. We will however conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Response: We agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to: “Studies in which participants are identified in environments other than acute care”
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear overview of the study methods and background to the study.

Response: Thank you for reviewing this protocol. We appreciate the feedback on clarity of the description of the methodology and study background.

My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.

Response: Thank you for this comment. The condition of interest in this instance is a contemporaneous diagnosis of cognitive impairment or dementia. Similar to previous reviews (https://doi.org/10.1177/23969873211042192), it is not our intention to conduct separate comparisons based on whether the reference standard is a clinical dementia diagnosis or clinical diagnosis of cognitive impairment. We will however conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Response: We agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to: “Studies in which participants are identified in environments other than acute care”
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 28 Jun 2024

Parichita Choudhury, Banner Sun Health Research Institute, Sun City, AZ, USA

Andrew Ho, Banner Sun Health Research Institute (Ringgold ID: 89008), Sun City, Arizona, USA

Approved with Reservations

https://doi.org/10.21956/hrbopenres.15237.r40568

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Other considerations:

1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Is the rationale for, and objectives of, the study clearly described?

Partly
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Dementia

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

CITE

Report a concern

Author Response 30 Aug 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

30 Aug 2024

Author Response
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy ... Continue reading
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

Response: Thank you for reviewing this protocol. We appreciate the feedback on appropriateness of the study methodology.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Response: While screening may occur in the acute setting, we agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to reflect this. “Studies in which participants are identified in environments other than acute care.”

Other considerations:
1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

Response: Thank you for this comment. There is no restriction on tools based on who administers them. Tools administered by any healthcare professional and tools for self-reporting of cognitive function are eligible for inclusion. We have rephrased the study selection criterion to: “The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.”

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

Response: Thank you for this comment. Studies reported in a language other than English will not be excluded. Our search strategy is comprehensive and will be translated across five databases to ensure maximum identification of relevant studies.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

Response: We agree that there may be high variability in diagnostic accuracy as a function of the reference standard used. For that reason, we plan to conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

Response: We agree that patients with and without head trauma should be analysed separately. We have rephrased the following sentence in the data synthesis section to: “Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma)”

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Response: Thank you for this comment. We appreciate the feedback on the importance of the research question
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

Response: Thank you for reviewing this protocol. We appreciate the feedback on appropriateness of the study methodology.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Response: While screening may occur in the acute setting, we agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to reflect this. “Studies in which participants are identified in environments other than acute care.”

Other considerations:
1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

Response: Thank you for this comment. There is no restriction on tools based on who administers them. Tools administered by any healthcare professional and tools for self-reporting of cognitive function are eligible for inclusion. We have rephrased the study selection criterion to: “The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.”

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

Response: Thank you for this comment. Studies reported in a language other than English will not be excluded. Our search strategy is comprehensive and will be translated across five databases to ensure maximum identification of relevant studies.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

Response: We agree that there may be high variability in diagnostic accuracy as a function of the reference standard used. For that reason, we plan to conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

Response: We agree that patients with and without head trauma should be analysed separately. We have rephrased the following sentence in the data synthesis section to: “Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma)”

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Response: Thank you for this comment. We appreciate the feedback on the importance of the research question
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 30 Aug 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

30 Aug 2024

Author Response
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy ... Continue reading
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

Response: Thank you for reviewing this protocol. We appreciate the feedback on appropriateness of the study methodology.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Response: While screening may occur in the acute setting, we agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to reflect this. “Studies in which participants are identified in environments other than acute care.”

Other considerations:
1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

Response: Thank you for this comment. There is no restriction on tools based on who administers them. Tools administered by any healthcare professional and tools for self-reporting of cognitive function are eligible for inclusion. We have rephrased the study selection criterion to: “The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.”

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

Response: Thank you for this comment. Studies reported in a language other than English will not be excluded. Our search strategy is comprehensive and will be translated across five databases to ensure maximum identification of relevant studies.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

Response: We agree that there may be high variability in diagnostic accuracy as a function of the reference standard used. For that reason, we plan to conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

Response: We agree that patients with and without head trauma should be analysed separately. We have rephrased the following sentence in the data synthesis section to: “Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma)”

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Response: Thank you for this comment. We appreciate the feedback on the importance of the research question
We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

Response: Thank you for reviewing this protocol. We appreciate the feedback on appropriateness of the study methodology.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Response: While screening may occur in the acute setting, we agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to reflect this. “Studies in which participants are identified in environments other than acute care.”

Other considerations:
1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

Response: Thank you for this comment. There is no restriction on tools based on who administers them. Tools administered by any healthcare professional and tools for self-reporting of cognitive function are eligible for inclusion. We have rephrased the study selection criterion to: “The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.”

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

Response: Thank you for this comment. Studies reported in a language other than English will not be excluded. Our search strategy is comprehensive and will be translated across five databases to ensure maximum identification of relevant studies.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

Response: We agree that there may be high variability in diagnostic accuracy as a function of the reference standard used. For that reason, we plan to conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

Response: We agree that patients with and without head trauma should be analysed separately. We have rephrased the following sentence in the data synthesis section to: “Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma)”

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Response: Thank you for this comment. We appreciate the feedback on the importance of the research question
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 04 Jun 2024

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 03 Sep 24		read	read
Version 1 04 Jun 24	read	read

Parichita Choudhury, Banner Sun Health Research Institute, Sun City, USA

Andrew Ho, Banner Sun Health Research Institute (Ringgold ID: 89008), Sun City, USA
Carole Fogg, University of Southampton, Southampton, UK
Joyce Chan, The Chinese University of Hong Kong, Hong Kong, Hong Kong

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

3 Views

27 Sep 2024 | for Version 2

Carole Fogg, University of Southampton, Southampton, England, UK

3 Views Cite this report Responses(0)

Approved

Many thanks for the helpful responses. I have no further comments and happy to approve.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Older adults health and care, epidemiology.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

5 Views

25 Sep 2024 | for Version 2

Joyce Chan, School of Public Health and Primary Care, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, Hong Kong

5 Views Cite this report Responses(0)

Approved With Reservations

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Research Interests:mild cognitive impairment, dementia, depression, cogntive assessment, psychosocial interventions for older adults

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

13 Views

05 Jul 2024 | for Version 1

Carole Fogg, University of Southampton, Southampton, England, UK

13 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Older adults health and care, epidemiology.

Respond to this report

Responses (1)

Author Response

03 Sep 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This systematic review protocol provides a clear overview of the study methods and background to the study.

Response: Thank you for reviewing this protocol. We appreciate the feedback on clarity of the description of the methodology and study background.

My main query is about the grouping of “cognitive impairment/dementia” – from the perspective that screening tools which assess cognitive function may then trigger a series of further assessments which may result in a diagnosis of dementia. It is unclear from the way in which this is grouped (if indeed that is the intention) as to what the ‘outcome’ in the case of such screening tools is – will there be a comparison for those screening for cognitive impairment vs clinically confirmed diagnosis of cognitive impairment, and a separate comparison for a screening tool for cognitive impairment which has led to a diagnosis of dementia? If the authors could please clarify what they had intended by combining these, this would be helpful.

Response: Thank you for this comment. The condition of interest in this instance is a contemporaneous diagnosis of cognitive impairment or dementia. Similar to previous reviews (https://doi.org/10.1177/23969873211042192), it is not our intention to conduct separate comparisons based on whether the reference standard is a clinical dementia diagnosis or clinical diagnosis of cognitive impairment. We will however conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

Related to this, the exclusion criteria of ‘studies conducted in environments other than acute care’ may be difficult to apply in situations where although screening occurs in acute care, subsequent follow-up/assessments take place in another environment. The authors could consider altering this to something like ‘studies in which participants are identified in environments other than acute care’, which would allow collection of diagnostic data across settings.

Response: We agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to: “Studies in which participants are identified in environments other than acute care”

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

15 Views

28 Jun 2024 | for Version 1

Parichita Choudhury, Banner Sun Health Research Institute, Sun City, AZ, USA

Andrew Ho, Banner Sun Health Research Institute (Ringgold ID: 89008), Sun City, Arizona, USA

15 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for, and objectives of, the study clearly described?

Partly
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Dementia

Respond to this report

Responses (1)

Author Response

30 Aug 2024

Niamh A. Merriman, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland

We thank the reviewer for the constructive feedback on this protocol. We now address each point in responses below the review comment.

This study aims to validate the accuracy of different screening tools in detecting different neurocognitive disorders in the setting of an acute traumatic event. They wish to do a systematic review utilizing well-established systems to assess the accuracy of studies such as PRISMA-DTA. The inclusion and exclusion criteria are clear and concise to achieve the desired search outcomes. The study utilizes the recommended minimum of two reviewers with a third to act as a tiebreaker to ensure reproducibility, and to present the dataset in 2x2 table to define specificity and sensitivity of each tool is a usable and accessible format.

Response: Thank you for reviewing this protocol. We appreciate the feedback on appropriateness of the study methodology.

One of the primary concerns lies within the study question itself.
The diagnosis of dementia or other cognitive impairment in acute setting is challenging and recommended against given confounding factors of delirium and other medical comorbidities. There are some tools for detecting delirium, but to my knowledge, there are no good tools to diagnose or detect mild cognitive impairment or dementia in an acute clinical environment. Accurate diagnosis of these latter two conditions require testing under nonacute conditions which would require evaluation prior to hospitalization or a trusted care partner at bedside. An initial diagnosis within an acute setting invites significant heterogeneity of conflicting differential diagnoses, and authors should be mindful of this as they conduct this review. This would make diagnostic accuracy of all tools poorer. Just because there is a good question does not always ensure that there will be a good answer.

Response: While screening may occur in the acute setting, we agree that a contemporaneous diagnosis of cognitive impairment or dementia is likely to occur in a setting other than acute. We have rephrased the study selection criterion to reflect this. “Studies in which participants are identified in environments other than acute care.”

Other considerations:
1. Would the tools included be: patient reported? administered by any healthcare professional? administered by professionals with specific training?

Response: Thank you for this comment. There is no restriction on tools based on who administers them. Tools administered by any healthcare professional and tools for self-reporting of cognitive function are eligible for inclusion. We have rephrased the study selection criterion to: “The study aims to validate a screening tool for detection of 1) delirium or 2) cognitive impairment, or dementia at any timepoint in acute care. Tools can be self-report or administered by any healthcare professional.”

2. If reports of any language is included, please be mindful of dearth of validation of screening tools in other languages and cultures.

Response: Thank you for this comment. Studies reported in a language other than English will not be excluded. Our search strategy is comprehensive and will be translated across five databases to ensure maximum identification of relevant studies.

3. Gold standards between studies will vary considerably, and therefore diagnostic accuracy may also have high variability.

Response: We agree that there may be high variability in diagnostic accuracy as a function of the reference standard used. For that reason, we plan to conduct a sub-group analysis based on the type of reference standard used (i.e., standardised diagnostic criteria or a validated tool).

4. The authors should consider separating out head trauma vs. no head trauma data as this may significantly impact the accuracy of results - detection of delirium.

Response: We agree that patients with and without head trauma should be analysed separately. We have rephrased the following sentence in the data synthesis section to: “Sub-group analyses will be conducted based on study population (i.e., post-hip fracture, post-surgical, head trauma, other physical trauma)”

The scientific question is an important one and if there are no good tests, then perhaps an answer of "no good tests" may encourage the development of a new tool, which is worthwhile in itself.

Response: Thank you for this comment. We appreciate the feedback on the importance of the research question

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Reynish EL, Hapca SM, De Souza N, et al.: Epidemiology and outcomes of people with dementia, delirium, and unspecified cognitive impairment in the general hospital: prospective cohort study of 10,014 admissions. BMC Med. 2017; 15(1): 140. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Pedone C, Ercolani S, Catani M, et al.: Elderly patients with cognitive impairment have a high risk for functional decline during hospitalization: the GIFA study. J Gerontol A Biol Sci Med Sci. 2005; 60(12): 1576–80. PubMed Abstract | Publisher Full Text

[3] 3. Lenartowicz M, Parkovnick M, McFarlan A, et al.: An evaluation of a proactive geriatric trauma consultation Service. Ann Surg. 2012; 256(6): 1098–101. PubMed Abstract | Publisher Full Text

[4] 4. National Office of Clinical Audit: Irish hip fracture database national report 2020. Dublin, Ireland, 2021. Reference Source

[5] 5. National Office of Clinical Audit: Irish hip fracture database national report 2017 - 2021. Dublin, Ireland, 2022. Reference Source

[6] 6. Government of Ireland: A trauma system for Ireland: report of the trauma steering group. 2018; [updated 26 October 2020]. Reference Source

[7] 7. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Washington, DC: American Psychiatric Association, 2013. Publisher Full Text

[8] 8. Lezak MD, Howieson DB, Bigler ED, et al.: Neuropsychological assessment. 5th ed. Oxford; New York: Oxford University Press, 2012. Reference Source

[9] 9. Petersen RC, Lopez O, Armstrong MJ, et al.: Practice guideline update summary: mild cognitive impairment: report of the guideline development, dissemination, and implementation subcommittee of the American Academy of Neurology. Neurology. 2018; 90(3): 126–35. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. World Health Organization: WHO dementia factsheet. World Health Organization, 2023; [updated 15 March 2023]. Reference Source

[11] 11. Jackson TA, Gladman JRF, Harwood RH, et al.: Challenges and opportunities in understanding dementia and delirium in the acute hospital. PLoS Med. 2017; 14(3): e1002247. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Nowroozpoor A, Dussetschleger J, Perry W, et al.: Detecting cognitive impairment and dementia in the emergency department: a scoping review. J Am Med Dir Assoc. 2022; 23(8): 1314.e31–1314.e88. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Fogg C, Griffiths P, Meredith P, et al.: Hospital outcomes of older people with cognitive impairment: an integrative review. Int J Geriatr Psychiatry. 2018; 33(9): 1177–97. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Tran J, Nimojan T, Saripella A, et al.: Rapid cognitive assessment tools for screening of mild cognitive impairment in the preoperative setting: a systematic review and meta-analysis. J Clin Anesth. 2022; 78: 110682. PubMed Abstract | Publisher Full Text

[15] 15. National Office of Clinical Audit: Major trauma audit national report 2021. Dublin, Ireland, 2023. Reference Source

[16] 16. Smith T, Hameed Y, Cross J, et al.: Assessment of people with cognitive impairment and hip fracture: a systematic review and meta-analysis. Arch Gerontol Geriatr. 2013; 57(2): 117–26. PubMed Abstract | Publisher Full Text

[17] 17. Seitz DP, Adunuri N, Gill SS, et al.: Prevalence of dementia and cognitive impairment among older adults with hip fractures. J Am Med Dir Assoc. 2011; 12(8): 556–64. PubMed Abstract | Publisher Full Text

[18] 18. Alzheimer's Disease International: Dementia statistics. Alzheimer's Disease International, 2020. Reference Source

[19] 19. Smith TO, Gilbert AW, Sreekanta A, et al.: Enhanced rehabilitation and care models for adults with dementia following hip fracture surgery. Cochrane Database Syst Rev. 2020; 2(2): CD010569. PubMed Abstract | Publisher Full Text | Free Full Text

[20] 20. Johansen A, Golding D, Brent L, et al.: Using national hip fracture registries and audit databases to develop an international perspective. Injury. 2017; 48(10): 2174–2179. PubMed Abstract | Publisher Full Text

[21] 21. Smith TO, Cooper A, Peryer G, et al.: Factors predicting incidence of post-operative delirium in older people following hip fracture surgery: a systematic review and meta-analysis. Int J Geriatr Psychiatry. 2017; 32(4): 386–96. PubMed Abstract | Publisher Full Text

[22] 22. de Moraes GS, Curiati PK, Avelino-Silva TJ, et al.: Aren't you forgetting something? Cognitive screening beyond delirium in the emergency department. J Am Geriatr Soc. 2023; 71(3): 987–91. PubMed Abstract | Publisher Full Text

[23] 23. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-IV. Washington DC: American Psychiatric Association, 1994. Reference Source

[24] 24. World Health Organization: International Classification of Diseases (ICD). World Health Organization, 2022; [updated 1 January 2022]. Reference Source

[25] 25. Moher D, Shamseer L, Clarke M, et al.: Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text

[26] 26. Merriman NA, Walsh ME, O'Regan N, et al.: PRISMA-P Checklist for ‘Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: a protocol for a systematic review’. Zenodo. 2024.

[27] 27. Cohen JF, Korevaar DA, Altman DG, et al.: STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016; 6(11): e012799. PubMed Abstract | Publisher Full Text | Free Full Text

[28] 28. McInnes MDF, Moher D, Thombs BD, et al.: Preferred Reporting Items for a Systematic review and Meta-analysis of Diagnostic Test Accuracy studies: the PRISMA-DTA statement. JAMA. 2018; 319(4): 388–96. PubMed Abstract | Publisher Full Text

[29] 29. Scottish Intercollegiate Guidelines Network: Search filters. Reference Source

[30] 30. Merriman NA, Walsh ME, O'Regan N, et al.: Extended data for 'Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: a protocol for a systematic review'. Zenodo. 2024.

[31] 31. Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation.

[32] 32. Moher D, Liberati A, Tetzlaff J, et al.: Preferred Reporting Items for Systematic reviews and Meta-Analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text

[33] 33. Whiting PF, Rutjes AWS, Westwood ME, et al.: QUADAS-2: a revised tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann Intern Med. 2011; 155(8): 529–36. PubMed Abstract | Publisher Full Text

[34] 34. Freeman SC, Kerby CR, Patel A, et al.: Development of an interactive web-based tool to conduct and interrogate Meta-analysis of Diagnostic Test Accuracy studies: MetaDTA. BMC Med Res Methodol. 2019; 19(1): 81. PubMed Abstract | Publisher Full Text | Free Full Text

[35] 35. Patel A, Cooper N, Freeman S, et al.: Graphical enhancements to summary receiver operating characteristic plots to facilitate the analysis and reporting of Meta-analysis of Diagnostic Test Accuracy data. Res Synth Methods. 2021; 12(1): 34–44. PubMed Abstract | Publisher Full Text

Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review

Abstract

Background

Objective

Design

Literature search

Study selection criteria

Data synthesis

Conclusion

Registration

Keywords

Revised Amendments from Version 1

Introduction

Protocol

Study design

Selection criteria

Search strategy

Study selection

Data extraction

Risk of bias in individual studies

Data synthesis

Study status

Conclusions

Data availability

Underlying data

Extended data

Reporting guidelines

Contributions

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

Diagnostic test accuracy of screening tools for the detection of neurocognitive disorders in older adults post-trauma: A protocol for a systematic review

Abstract

Background

Objective

Design

Literature search

Study selection criteria

Data synthesis

Conclusion

Registration

Keywords

Revised Amendments from Version 1

Introduction

Protocol

Study design

Selection criteria

Search strategy

Study selection

Data extraction

Risk of bias in individual studies

Data synthesis

Study status

Conclusions

Data availability

Underlying data

Extended data

Reporting guidelines

Contributions

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!