Prehospital characteristics that identify major trauma patients: A hybrid systematic review protocol

Reviewers report
The protocol “Prehospital characteristics that identify major trauma patients: A hybrid systematic review protocol” by Donnelly et al. is a concise and informative protocol and an interesting teaser for the review study that aims at identifying prehospital characteristics associated with major trauma. The protocol is overall very clear and well-written. Below is our overall impression of each part of the protocol, followed by detailed suggestions for improvements.

Abstract: The abstract is good overall. After addressing our remarks (below) some details will need to be updated.

Introduction: Overall the introduction appropriately explains the burden of trauma and the challenges with designing an effective trauma care system, including the importance of trauma triage tools. However, it can be improved to more accurately support the chosen objective of the article, please see suggestions below.

Method: Relatively clear overall, and the extended files provide additional detail and add value to the protocol. While the method appears to align well with the established objective, enhancing comprehensibility could be achieved by offering clarification and motivation for specific choices (see remarks below for detail).
Conclusion: The conclusion connects well to the introduction, objective and method, is clear and concise, and states the expected impact of the contribution.

References: The reference list is pertinent; however, some important studies are missed. Please see remarks below.

We have some general and specific recommendations for improving the paper. We will clarify our concerns and recommendations point-by-point below. We recommend the authors to address the following remarks, where major remarks are numbered with Arabic numerals and minor remarks with Roman numerals.

We would like to thank the reviewers for the comprehensive and positive comments.

Major remarks:

1. The primary outcome is defined as ISS>15. This is probably still the most widely used definition, although criteria using New Injury Severity Score (NISS) are becoming increasingly common, please see the article by Tohira et al. (2012¹). Moreover, different thresholds of ISS and NISS may be used to identify major trauma, please see the article by Palmer et al. (2016²). The authors should clarify how papers using NISS as criteria, and different thresholds for major trauma, will be handled.
We have re-worded the section on secondary outcomes to clarify this. This section now reads:
“Sensitivity analyses will examine alternative definitions of major trauma. This will include: intensive care admission, death in the Emergency Department, patients transferred in/out of hospital for specialist care (1), and the New Injury Severity Score (NISS) (2). We will also examine different thresholds of ISS and NISS to identify major trauma (3). To support this, the definition(s) of major trauma applied in each paper will be noted as part of the data extraction criteria. Where a paper applies more than one definition, both will be recorded and analysed separately in the analysis.”

2. Although the reference list is pertinent, there is a need for revision. There are several information items that are insufficiently supported by references and instances where it could be suitable to replace or complement references with others. For example, reference 2 by van Breugel et al mainly studies how mortality related to polytrauma has changed, and could be complemented with for instance WHO’s report "Injuries and Violence: The Facts 2014". There also exist different definitions of undertriage and overtriage, thus, please add a supporting reference for the selected definitions. Furthermore, no references are used to support the claim that integrated inclusive trauma networks improve the trauma care process and outcomes. The word choice of “consistently” (both in the abstract and introduction) may be inappropriate, please be aware of opposing views or challenges with trauma networks and consider rephrasing the paragraph to include this information. For instance, the reference by Pigneri et al. (2017³) could be used to point on challenges, e.g., loss of autonomy and cost of meeting requirements. Furthermore, please add supporting references when referring to the literature or statistics (e.g., “… Nearly 6 million people die each year because of traumatic injuries. …”, “… International evidence has …”, “… There is considerable international literature …”, and similar statements). These are some suggestions for changes concerning the protocol’s references and usage, but please review the whole protocol for similar changes.
We have re-worded the sections referred to, incorporated the suggested references and included additional references to support the text.

3. We recommend clarifying what types of studies that will be included, and whether their quality will be assessed and if so how. For example, in “Types of studies" scoping reviews are not explicitly mentioned. Scoping reviews can also be an important source of information, and sometimes grey literature is also examined. We recommend clarifying whether scoping reviews and grey literature will be included, and explain rationales for the choices made. Please provide a more detailed description of how the utilization of the Newcastle Ottawa Quality Assessment form aligns with the research objective and how this may influence your findings. For instance, consider how high-quality studies may carry more weight in your analysis, while studies falling below a certain quality threshold could be excluded from your study. Furthermore, please clarify what you define as “outside prehospital setting”.
As we are interested in publications that have undergone peer review, grey literature will not be included in the review. Given resource constraints, it will not be possible for us to include scoping reviews. Potentially, this may result in the omission of some trauma triage tools. However, these tools should not have been missed by systematic reviews conducted to date. Therefore, we do not anticipate the omission of scoping reviews would significantly impact the study findings. We have acknowledged the exclusion of scoping reviews in the limitations section of the protocol. This paragraph reads:
“In tandem with this, also given resource constraints, it will also not be possible for us to include scoping reviews. Potentially, this may result in the omission of trauma triage tools. However, these tools should not have been missed by systematic reviews conducted to date. Therefore, we do not anticipate the omission of scoping reviews would significantly impact the study findings.”

We will assess study quality with the Newcastle Ottawa Quality Assessment Scale. We have selected this tool as it is a recommended assessment tool for observational studies by the Cochrane Collaboration (4). To clarify, the objective of the review is to examine the prehospital characteristics applied in the international literature, with the aim of developing as complete and comprehensive list of characteristics as is possible. Excluding studies which fall below any threshold in quality would undermine this aim. Rather, the Newcastle Ottawa Quality Assessment Scale will enable us to examine the degree of agreement in prehospital characteristics in better quality studies. ‘Outside prehospital setting’ refers to studies that examine patient characteristics measured in differing contexts. For example, studies that measure patients’ characteristics from within the Hospital Emergency Department or an acute care ward.

4. Search methods – Databases: Relatively few databases are searched. Please provide a rationale for omitting databases such as Scopus and Google Scholar.
The search strategy was developed with the support of an Information Specialist in RCSI. As part of developing the search strategy, we reviewed the range of possible databases included in recent systematic reviews in the area  ADDIN EN.CITE.DATA (5-7). In each of these reviews the databases employed were: Pubmed (Ovid MEDLINE), Embase, Cochrane Library of Systematic Reviews and Cochrane Central Register of Clinical Trials. Given other reviews in the area had included these databases, it was felt that databases selected would provide a sufficient degree of specificity and sensitivity to the search. Google Scholar will be used for the ‘cited by’ function, to identify articles which have cited the included articles, further ensuring comprehensive coverage.

5. Data collection – Data extraction: We recommend to also extract the limitations the original authors highlight with their studies.
Thank you for this suggestion. We will include the limitation the original authors highlight with their studies in the data extraction template.

6. Data collection – Data synthesis: We recommend to extract the accuracy (over and under triage, etc) explicitly in a separate table, to allow readers to quickly get an overview of current tools and their performance.
Thank you for this suggestion we are currently including the estimated over and under triage rates reported in a separate table as part of the data extraction protocol.

7. Please consider adding a section addressing the limitations of the study. This helps ensure research transparency and provides a more complete understanding of the study findings.
We have included a comprehensive section addressing limitations in the protocol. This reads:
“Limitations
Given resource constraints, it will not be possible for us to include non-English language publications. Potentially, this may result in the omission of some non-English tools. However, based on previous reviews in the area, it would appear that tools from non-English speaking countries (such as France, the Netherlands and Norway) have been translated and published in English (8-10). Thus, we do not anticipate the omission of many non-English language publications.
In tandem with this, also given resource constraints, it will also not be possible for us to include scoping reviews. Potentially, this may result in the omission of trauma triage tools. However, these tools should not have been missed by systematic reviews conducted to date. Therefore, we do not anticipate the omission of scoping reviews would significantly impact the study findings.
As the study focuses on the prehospital characteristics applied in trauma triage tools, the search concerned triage tools only. Studies that analyse the characteristics related to major trauma but did not develop a triage tool were excluded. Potentially this may result in the omission of some prehospital characteristics. However, given the decades of research in the area, it is unlikely that pertinent variables have been omitted from the trauma triage tools developed to date. Therefore, we do not anticipate a significant impact on the study findings given the omission of such studies.
In developing the search strategy, we ran exploratory searches and found the use of the terms ‘assessment’ or ‘evaluation’ led to too broad a search, with many thousands of extra irrelevant hits, which we do not have the resources to countenance. This may result in the omission of potentially relevant articles. However, given our hybrid approach of searching for systematic reviews, supplemented with the search for individual studies from more recent literature, it is unlikely that important articles will have been omitted. 
In searching for systematic reviews, there is always a risk that a search strategy applied in those systematic reviews will omit a relevant article, or that reviewers will incorrectly classify a paper as irrelevant. However, this risk is mitigated by reviewing the reference lists of articles that are included, and the ‘cited by’ function in Google Scholar for these articles also. It is very unlikely that using these methods along with our combination of searching for systematic reviews, supplemented with individual studies from more recent literature, will omit important articles. This is especially the case as the more recent articles will most likely cite other recent literature.
Finally, we have excluded tools that do not apply standardised approaches for assessment. Potentially, this could result in the omission of tools that are based on modern technology that have yet to establish a standardised approach to assessment. We have excluded tools with non-standardised approaches to assessment as the review is being conducted to inform the development of future trauma triage tools to be used by paramedics to identify major trauma patients. As the tool would be used by a range of practitioners in a range of contexts, it is important that standardised approaches for assessment are included. Should the approach not be standardised, there is a potential that this could expose patients to sub-optimal clinical decision-making. These could be explored in further research in the future.”

8. We find the objective to be somewhat vague. The term "prehospital characteristics" is not properly explained in the introduction, and the objective may then not be comprehended by the readers. At the end of the introduction, two different gaps seem to be presented: 1) there is no optimal triage tool, and 2) there is no minimal criteria for prehospital identification of major trauma. These two gaps may result in different aims, and it is not clear which one this review will focus on. Furthermore, the presented gaps may be associated with other objectives, such as finding the best method for triaging (e.g., traditional statistics versus artificial intelligence) rather than finding characteristics for major trauma. We therefore recommend that you clarify the introduction in terms of the gap which this study will focus on, and why the selected objective is chosen.
We have amended the fourth paragraph in the introduction to clarify the term ‘prehospital characteristics’. This sentence reads: “prehospital characteristics, these are patient characteristics measured before reaching a hospital setting.”
 
There are two gaps presented at the end of the introduction as one follows the other. There is no consensus on the optimal triage tool. As a triage tool presents a minimum set of criteria to identify a major trauma patient, there is no consensus on the minimum criteria to identify a major trauma patient in a prehospital context. In order to identify a comprehensive list of prehospital characteristics applied to support paramedics in determining if a patient is a major trauma patient, we need to identify the range of trauma triage tools developed in the international literature which utilise characteristics with predictive validity. We have also amended the introduction to clarify that examining and synthesising the prehospital characteristics applied to identify major trauma patients will inform the development of future trauma triage tools. While we appreciate that finding the best methods for triaging a patient is an interesting question, it is nonetheless a separate question and therefore outside the scope of this review.

Minor remarks:

i. In “Types of Participants", could the PICO process be used for increased clarification of the literature search strategy? (please see https://en.wikipedia.org/wiki/PICO_process ). We also suggest that the second sentence about paediatrics and geriatrics is clarified regarding whether these groups will be included or not.
We thank the reviewers for suggesting applying the PICO process to clarify the literature search strategy. We refer the reviewers to the International Prospective Register of Systematic Reviews (PROSPERO) database registration of the review (Protocol Number: CRD42023393094). This cites the Participants/population, Interventions/exposures, Comparators and Outcomes. These are also specified in the protocol, under the section ‘Criteria for Selecting Studies’.
We have amended the section ‘types of participants’ to clarify:
“Studies that are confined to specific population (paediatrics or geriatrics) will be included in the review. If a study has been confined to a paediatric or geriatric population, it will be noted as part of the data extraction criteria”.

ii. Data collection – Study selection: The authors can consider to add a method for calculating the level of agreement between the reviewers, such as Cohen’s kappa value.
We are not applying Cohen’s Kappa to measure the degree of agreement between reviews as it is not recommended by Cochrane for measuring agreement (4).  Given that the implication of the disagreement for the selection of a study will be study dependent (large high-quality studies vs small low quality studies), Cochrane recommend the reasons for and implications of any disagreement should be explored rather than applying an arbitrary cut off.

iii. Method: No estimated time frame for completion of the review is provided, we recommend to add this information.
We have amended the section on study status to clarify that we anticipate the review will be completed by March 2024.

iv. Conclusion: The authors state that “The findings of this review will inform the development of a national clinical prediction rule for triage of major trauma patients”. We would recommend to stress that the findings also can be valuable for other stakeholders, such as clinicians and the research community within trauma care, etc.
We have amended the conclusion to include the following sentence:- “The findings of this review will therefore be of value to a wide range of trauma care stakeholders, including prehospital emergency care providers, hospital clinicians and the wider trauma care research community”.

v. Extended File 2 - Please clarify what the rightmost column in Table 1 (Search 1 – Search for Systematic Reviews of Trauma Triage Tools) represents.
The rightmost column represents the number of search results for each of the terms applied (in the middle column). We have amended Extended File 2 to include column titles to add clarity to the tables.

vi. The scope of HRB Open Research is broad and readers might not have a background knowledge of how trauma care is organized. We therefore suggest that a short description of the prehospital trauma care is provided in the introduction, before the second paragraph, so that readers can understand why transportation to hospital is relevant. Additional parts that could benefit from a description are "fragmented trauma care" and "trauma networks".
We have amended the introduction to now read:
“Trauma care in Ireland has traditionally been based on the practice of bringing trauma patients to the nearest hospital in the first instance (11). This has resulted in fragmentation of trauma care as patients often go to hospitals that cannot provide the specialties required. Consequently, a considerable proportion of trauma patients are transferred to another hospital to cater for their injuries. For example, in 2018, 20% of trauma patients were subsequently transferred to a second hospital (12). The publication of ‘A Trauma system for Ireland’ signalled the need to reform the delivery of trauma care in Ireland. The report recommended the establishment of trauma networks (13). The trauma care system in Ireland is currently undergoing a process of re-configuration to enable such networks. These networks will be made up of major trauma centres (for severe injuries) at the network hub, and Trauma Units, Local Emergency Hospitals (for less severe injuries) forming the spokes. This is to ensure patients are brought quickly and seamlessly to the most appropriate hospital to manage their injuries.”

vii. In the introduction, information is given about patients taken to hospitals that cannot provide the required specialties (“Patients often go to hospitals that cannot provide the specialties required.”). Please clarify if this refers to the transportation by prehospital resources, or self-transportation by the patients. Furthermore, to emphasise the consequences of this, it would be valuable to add information about mortality and disability.
The sentence “Patients often go to hospitals that cannot provide the specialties required” refers to all trauma patients, irrespective of the mode of arrival (car/walking/ambulance). Unfortunately, due to data limitations, it would not be possible to add information on mortality and disability for Irish patients who are transferred. Data from the UK suggests that introduction of nationally inclusive trauma networks with tiered hospitals and ambulance by-pass protocols, was associated with a 19% increase in the odds of survival following major trauma (14). From an Irish perspective, a study that is currently ongoing, the HRB TRAUMA study, aims to develop a database of the trauma patient’s journey from incident to discharge. This would include estimates of a range of outcomes for trauma patients transferred during the process of care (see https://traumastudy.eu/).

viii. It is unclear if the last sentence of the fifth paragraph in the introduction (“Bypass to a major trauma centre without clinical need also adds undue distress to the patient-family dyad given longer distances to travel to visit loved-ones in hospital.”) is a further elaboration on cons with overtriage, or critique for the concept of bypassing the closest hospital. Please clarify and if needed, move to a more suitable place in the introduction.
We have edited this sentence to clarify that this is a further elaboration on the cons of over triage. The sentence refers to scenarios where paramedics may bypass local hospitals and bring patients to a major trauma centre without clinical need. The absence of clinical need for the bypass is key here as patients are further away from their families, who have to travel longer distances, when there would not be a clinical benefit to the patient in bringing them to a major trauma centre.  This sentence now reads:
“In addition, where patient’s are over triaged, bypassed to a major trauma centre without clinical need, this can also add undue distress to the patient-family dyad given longer distances to travel to visit loved-ones in hospital.”

ix. When explaining the deliverable of the review in the last paragraph of the introduction, it seems that the review will primarily focus on finding tools, and then analyse what characteristics have been used. There is a concern that the search for tools might exclude studies that primarily focus on finding characteristics related to major trauma, without calling it a tool. Please consider if an alternative approach is more suitable, which includes both options.
The reviewers make an interesting point. However, our study is being conducted to examine the prehospital characteristics that have been applied in international trauma triage tools only. It is unlikely that pertinent variables have been omitted from the trauma triage tools developed to date given the decades of research in the area. However, we have added this to the limitations section. This paragraph reads:
“As the study focuses on the prehospital characteristics applied in trauma triage tools, the search concerned triage tools only. Studies that analyse the characteristics related to major trauma but did not develop a triage tool were excluded. Potentially this may result in the omission of some prehospital characteristics. However, given the decades of research in the area, it is unlikely that pertinent variables have been omitted from the trauma triage tools developed to date. Therefore, we do not anticipate a significant impact on the study findings given the omission of such studies.”

x. In "types of exposure", tools that do not apply standardised approaches for assessment will be excluded. This raises the concern that the review will miss new tools that are based on modern technology. These might not yet be established because of the time and resources needed to implement solutions, but may still be of interest. We therefore recommend a reconsideration of this exclusion criterion.
The review is being conducted to inform the development of future trauma triage tools to be used by paramedics to identify major trauma patients. As the tool would be used by a range of practitioners in a range of contexts, it is important that standardised approaches for assessment are included. Should the approach not be standardised, there is a potential that this could expose patients to sub-optimal clinical decision-making. It is difficult to envision modern technology that does not use a standardised approach to patient assessment. Tools that do not apply a standardised approach are unlikely to be useful for paramedics, especially should they result in variances in clinical guidance. These could be explored in further research in the future. We have added this to the limitations section of the protocol. This paragraph reads:
“Finally, we have excluded tools that do not apply standardised approaches for assessment. Potentially, this could result in the omission of tools that are based on modern technology that have yet to establish a standardised approach to assessment. We have excluded tools with non-standardised approaches to assessment as the review is being conducted to inform the development of future trauma triage tools to be used by paramedics to identify major trauma patients. As the tool would be used by a range of practitioners in a range of contexts, it is important that standardised approaches for assessment are included. Should the approach not be standardised, there is a potential that this could expose patients to sub-optimal clinical decision-making. These could be explored in further research in the future.”

xi. The search terms may not capture some studies due to the variation in terminology. For instance, triage may be implied by using the terms "assessment" or "evaluation", and trauma may be implied by using the term "injury". Please reconsider additional search terms to capture these and other related terminology variations.
The search strategy was developed with the support of an Information Specialist in RCSI. We ran exploratory searches when developing the search strategy and found the use of the terms ‘assessment’ or ‘evaluation’ led to too broad a search, with many tens of thousands of extra irrelevant hits, which we do not have the resources to countenance. Therefore, we retain the search strategy as indicated, but add this as a limitation. This paragraph reads:
“In developing the search strategy, we ran exploratory searches and found the use of the terms ‘assessment’ or ‘evaluation’ led to too broad a search, with many thousands of extra irrelevant hits, which we do not have the resources to countenance. This may result in the omission of potentially relevant articles. However, given our hybrid approach of searching for systematic reviews, supplemented with the search for individual studies from more recent literature, it is unlikely that important articles will have been omitted.” 

xii. Is there a risk that the systematic review covering original articles up to November 2019 is not capturing everything relevant? This is possibly a limitation of the chosen strategy, and something the authors may elaborate on.
There is always a risk that a search strategy will omit a relevant article, or that reviewers will incorrectly classify a paper as irrelevant. However, this risk is mitigated by reviewing the reference lists of articles that are included, and the ‘cited by’ function in Google Scholar for these articles also. It is very unlikely that using these methods along with our combination of searching for systematic reviews, supplemented with individual studies from more recent literature, will omit important articles. This is especially the case as the more recent articles will most likely cite other recent literature. Nonetheless, we have added this as a limitation of the hybrid review approach which may interest readers more broadly. This paragraph reads:
“In searching for systematic reviews, there is always a risk that a search strategy applied in those systematic reviews will omit a relevant article, or that reviewers will incorrectly classify a paper as irrelevant. However, this risk is mitigated by reviewing the reference lists of articles that are included, and the ‘cited by’ function in Google Scholar for these articles also. It is very unlikely that using these methods along with our combination of searching for systematic reviews, supplemented with individual studies from more recent literature, will omit important articles. This is especially the case as the more recent articles will most likely cite other recent literature.”

xiii. There seems to be missing an “on” in the sentence “The accuracy of a triage tool is based the tool’s ability to correctly identify those with and without severe injuries.” in the introduction. Also, please remove the word "all" in the method (e.g., "... We will find all systematic reviews ...", "... We will include all trauma...") since it will be based on the study design and limitations. Please also explain the abbreviation "RCSI", as it may be unfamiliar to readers.
We have made the suggested changes to add clarity to the publication.

References
1.            Waydhas C, Bieler D, Hamsen U, Baacke M, Lefering R, TraumaRegister DGU. ISS alone, is not sufficient to correctly assign patients post hoc to trauma team requirement. Eur J Trauma Emerg Surg. 2022;48(1):383-92.
2.            Tohira H, Jacobs I, Mountain D, Gibson N, Yeo A. Systematic review of predictive performance of injury severity scoring tools. Scand J Trauma Resusc Emerg Med. 2012;20:63.
3.            Palmer CS, Gabbe BJ, Cameron PA. Defining major trauma using the 2008 Abbreviated Injury Scale. Injury. 2016;47(1):109-15.
4.            Higgins; JP, Green;. S. Cochrane Handbook for Systematic Reviews of Interventions online: THE CHOCHRANE COLLABORATION; 2011 [updated Updated March 2011. Version 5.1.0:[Available from: https://handbook-5-1.cochrane.org/.
5.            Lupton JR, Davis-O'Reilly C, Jungbauer RM, Newgard CD, Fallat ME, Brown JB, et al. Under-triage and Over-triage Using the Field Triage Guidelines for Injured Patients: A Systematic Review. Prehospital emergency care : official journal of the National Association of EMS Physicians and the National Association of State EMS Directors. 2022:1-11.
6.            van Rein EAJ, Houwert RM, Gunning AC, Lichtveld RA, Leenen LPH, van Heijl M. Accuracy of prehospital triage protocols in selecting severely injured patients: A systematic review. J Trauma Acute Care Surg. 2017;83(2):328-39.
7.            Gianola S, Castellini G, Biffi A, Porcu G, Fabbri A, Ruggieri MP, et al. Accuracy of pre-hospital triage tools for major trauma: a systematic review with meta-analysis and net clinical benefit. World Journal of Emergency Surgery : WJES [Internet]. 2021; 16(1):[31 p.]. Available from: https://www.embase.com/search/results?subaction=viewrecord&id=L635312589&from=export
http://dx.doi.org/10.1186/s13017-021-00372-1.
8.            Voskens FJ, van Rein EAJ, van der Sluijs R, Houwert RM, Lichtveld RA, Verleisdonk EJ, et al. Accuracy of Prehospital Triage in Selecting Severely Injured Trauma Patients. JAMA Surgery. 2018;153(4):322-7.
9.            Vinjevoll OP, Uleberg O, Cole E. Evaluating the ability of a trauma team activation tool to identify severe injury: a multicentre cohort study. Scand J Trauma Resusc Emerg Med. 2018;26(1):63.
10.         Bouzat P, Ageron F-X, Brun J, Levrat A, Berthet M, Rancurel E, et al. A regional trauma system to optimize the pre-hospital triage of trauma patients. Critical Care. 2015;19(1):111.
11.         Irish Association for Emergency Medicine. An Integrated trauma system for Ireland. Dublin: RCSI,; 2014.
12.         National Office of Clinical Audit. Major Trauma Audit National Report 2018. Dublin: National Office of Clinical Audit; 2020.
13.         Report of the Trauma Steering Group. A Trauma System for Ireland. Dublin: Department of Health,; 2018.
14.         Moran CG, Lecky F, Bouamra O, Lawrence T, Edwards A, Woodford M, et al. Changing the System - Major Trauma Patients and Their Outcomes in the NHS (England) 2008-17. EClinicalMedicine. 2018;2-3:13-21.