Abstract
Background There is a growing trend toward verification of trauma centres, but its impact remains unclear. This systematic review aimed to synthesize available evidence on the effectiveness of trauma centre verification.
Methods We conducted a systematic search of the CINAHL, Embase, HealthStar, MEDLINE and ProQuest databases, as well as the websites of key injury organizations for grey literature, from inception to June 2019, without language restrictions. Our population consisted of injured patients treated at trauma centres. The intervention was trauma centre verification. Comparison groups comprised nonverified trauma centres, or the same centre before it was first verified or re-verified. The primary outcome was in-hospital mortality; secondary outcomes included adverse events, resource use and processes of care. We computed pooled summary estimates using random-effects meta-analysis.
Results Of 5125 citations identified, 29, all conducted in the United States, satisfied our inclusion criteria. Mortality was the most frequently investigated outcome (n = 20), followed by processes of care (n = 12), resource use (n = 12) and adverse events (n = 7). The risk of bias was serious to critical in 22 studies. We observed an imprecise association between verification and decreased mortality (relative risk 0.74, 95% confidence interval 0.52 to 1.06) in severely injured patients.
Conclusion Our review showed mixed and inconsistent associations between verification and processes of care or patient outcomes. The validity of the published literature is limited by the lack of robust controls, as well as any evidence from outside the US, which precludes extrapolation to other health care jurisdictions. Quasiexperimental studies are needed to assess the impact of trauma centre verification.
Systematic reviews registration PROSPERO no. CRD42018107083
The introduction of trauma systems, defined as an organized and multidisciplinary response to injury from prehospital care to rehabilitation and community integration, has led to important reductions in injury burdens in many high-income countries.1,2 Essential to the development of a trauma system is the designation of trauma centres according to levels of care (levels I–V for adults, and I or II for pediatric centres), which is commonly the role of states or provinces.3 Trauma centres are acute care hospitals where resources are prioritized to ensure that injured patients receive appropriate and timely care.4,5 Injury organizations, including the American College of Surgeons, have established trauma facility standard guidelines.3 These guidelines have been used to develop trauma centre verification or accreditation processes, aimed to determine whether trauma centres are fulfilling the criteria for optimal care. “Accreditation” and “verification” of trauma centres refer to the same process; hereafter we use the term verification to refer to both.5,6
The terms “verification” and “designation” are sometimes used interchangeably despite having different meanings.7–10 Designation is conducted by regional health authorities at the local or state stage, where centres are categorized in levels (I–V for adults, and I or II for pediatric centres), whereas verification (or accreditation) is generally an optional program to verify that a facility is performing as a trauma centre and meets the criteria for its designation level.5,9,11 For example, verification is offered by the American College of Surgeons in the United States3,12,13 and accreditation by Accreditation Canada;6 these organizations are not responsible for designation.11 A centre can be designated at a particular level without having received verification.9,10 In some US states or Canadian provinces, regulatory agencies may require regular verification for a trauma centre to maintain designation within their systems. Verification allows for standardization of personnel and equipment and a facility’s commitment to trauma care.9 Perceived advantages of verification include commitment to trauma care, and identification of opportunities and priorities for improvement.14 Verification is, however, an expensive and resource-intensive process.15,16 It generally requires a centre to submit a prereview questionnaire and have an on-site visit by an experienced peer review team.3 A summary of verification modalities in different countries is presented in Table 1.
Although verification has become a common practice,14,17 the evidence of its effectiveness on patient outcomes has not been systematically assessed and synthesized. It is essential to know whether the allocation of financial and human resources used in the verification process has its intended effect.17,18 This systematic review aims to synthesize available evidence on trauma centre verification to evaluate whether verification reduces in-hospital mortality, adverse events and resource use and improves processes of care.
Methods
The protocol of this review was registered in the PROSPERO database (record CRD42018107083) and published.19 The review was conducted in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.20 We received ethics approval for this project from the McGill University Faculty of Medicine Research Ethics Committee.
Literature search and study selection
We conducted a systematic search of the CINAHL, Embase, HealthStar, MEDLINE and ProQuest databases, as well as the websites of key injury organizations for grey literature from inception to June 2019, without language restrictions. Manual searches for additional eligible studies were performed by reviewing the reference lists of included studies. The search strategy is available in Appendix 1 (available at canjsurg.ca/016219-a1). Conference abstracts were included unless they were subsequently published as full articles.
Study population and intervention
Our study population consisted of injured patients treated at trauma centres. The intervention under evaluation was trauma centre verification. Comparison groups consisted of nonverified centres, or the same centre before it was first verified or re-verified. We considered all study designs; however, narrative studies without a quantitative estimate of the association between verification and the investigated outcomes were excluded.
Outcomes
Our primary outcome was in-hospital mortality. Secondary outcomes included population-based injury-related mortality, adverse events (e.g., complications), resource use (e.g., length of stay [LOS] and costs) and adherence to evidence-based processes of care (e.g., nonsurgical management of splenic injuries).
Data collection and extraction
After duplicates were removed from the search results,21 titles and abstracts were independently screened by 2 authors (B.B. and M.C.) using a Web and mobile app for systematic reviews.22 In case of disagreement or uncertainty, full papers were retrieved and discussed with a third author (L.M.). Full texts of selected studies were retrieved and examined to determine eligibility by 2 authors (B.B. and M.C.), who also independently extracted the data using standardized forms. When available, data recorded included country of the study, the number of centres, study design, patient demographic characteristics and outcome. Efforts were made to contact the corresponding author for further information when needed. Descriptive statistics and measures of associations were extracted directly from the studies or computed if enough information was provided.23–25
We assessed the risk of bias using the Risk Of Bias In Non-randomised Studies – of Interventions (ROBINS-I) assessment tool.26 We evaluated the quality of the collective evidence and strength of recommendations using Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group methodology.27
Statistical analysis
We summarized included studies descriptively. Owing to the diverse types of measures of association used and missing standard errors (SEs) or confidence intervals (CIs), some studies were not included in the meta-analysis. These studies were summarized narratively.
For studies included in the meta-analysis, we calculated the overall summary estimates, including relative risks (RRs), odds ratios (ORs) and weighted mean difference using random-effects meta-analysis.28 We quantified heterogeneity with the I2 statistic.29 We also produced funnel plots to examine the potential for publication bias. Sensitivity analysis according to the risk of bias was planned but could not be done owing to the small number of studies included. All analyses were performed with admetan and metafunnel packages in Stata 15 (Statacorp).30
Results
A total of 5125 citations were initially identified by the search strategy after de-duplication. Among them, 102 articles were selected for full-text review, of which 29 satisfied our inclusion criteria (Fig. 1 and Appendix 1, Table S1).
All included studies assessed American College of Surgeons verification in the US and were observational, including 18 cross-sectional,7,9,10,14,16,31–43 10 pre–post8,44–52 and 1 time-series53 design. Mortality was the most commonly investigated outcome (n = 20), followed by processes of care (n = 12), resource use (n = 12) and adverse events (n = 7). A summary of study characteristics is presented in Table 2. It was not possible to compute CIs for the measure of associations in 7 studies (24%). Almost half of included studies (13 [45%]) did not adjust for centre or patient case mix characteristics,8,35,38,39,40,44,46–52 and only one-third (6/18 [33%]) of multicentre studies considered the clustered nature of the data in their analyses. The risk of bias was serious to critical in 22 studies, and moderate in 7 (Table 3).
Outcomes
Mortality
Twenty studies7–10,14,16,31,33,34,37,42–45,47–51,53 investigated mortality, of which 18 looked at in-hospital mortality. Seven of the 20 (35%) focused on pediatric injured patients, 4 (20%) on adults, and 9 (45%) on both. Most studies had cross-sectional (11) or pre–post (8) designs, with only 1 time-series study. Half of the studies presented only crude estimates.
We excluded 5 studies from the meta-analysis owing to the different effect measure scales used, as well as missing SEs (Table 2). Jenkins and colleagues43 found that mortality increased during surgery conferences compared to non-conference dates in trauma centres lacking verification (OR 1.2, 95% CI 1.1 to 1.4). However, among verified trauma centres, no association was observed. Piontek and colleagues45 found a 22% reduction in standardized mortality ratio following verification of a level II trauma centre. The study by Schubert and colleagues,9 one of the few studies that accounted for time-varying verification status during the study period, showed a protective association in lower-level centres (RR 0.84, 95% CI 0.72 to 0.99 for level III and RR 0.25, 95% CI 0.12 to 0.54 for level IV). Notrica and colleagues37 found that population-based pediatric injury mortality rates in US states with verified level I pediatric trauma centres were 37% lower than those in states without a verified pediatric trauma centre. The only time-series analysis showed that the lagged crude rate of verified level I pediatric trauma centres was protective and contributed to a decline (12%, 95% CI 4% to 18%) in the rate of change in adolescent injury mortality in the US between 1999 and 2015.53 A similar, but smaller, protective effect was observed for combined adult and pediatric verified level I trauma centres.
We included 15 studies in the meta-analysis of mortality.7,8,10,14,16,31,33,34,42,44,47–51 Analysis of crude RR (n = 11) showed that verification was generally associated with decreased mortality (Fig. 2A). This association was also observed in analysis of adjusted estimates (n = 7), except among severely injured patients (Injury Severity Score > 24) (OR 1.1, 95% CI 1.05 to 1.30) (Fig. 2B).
Funnel plots indicated a certain degree of asymmetry, which was more pronounced among studies providing crude estimates (Fig. 3A and Fig. 3B), which suggests publication bias. These figures also showed substantial variability among studies with larger samples. The GRADE results suggest that the quality of evidence is very low (Appendix 1, Supplemental Table S2).
Resource use
Length of stay, including overall and intensive care unit, was the most studied outcome in the resource use category (10 of 12 studies).7,8,34,45,47–52 Other outcomes examined were blood products transfused, hospital charges, mechanical ventilation, bedside use of ultrasonography and use of recombinant factor VIIa (rFVIIa).36,40 Six of 12 studies focused on injured pediatric patients. Only 4 studies adjusted for at least 1 potential confounder.7,34,36,45
We conducted a meta-analysis of 7 studies assessing LOS. Three studies described the distribution of LOS using the median and interquartile range,49–51 and 4 presented the mean and standard deviation.8,34,45–48 We used a well-established technique to combine results reported on log-transformed or raw scales24,25 to conduct the analysis. Because of the skewed distribution of LOS, we computed only weighted mean differences of the log-transformed LOS (which can be interpreted as the geometric mean ratio when exponentiated). Our results suggest that American College of Surgeons verification was associated with longer intensive care unit LOS but not hospital LOS (Fig. 4).
Funnel plots displayed asymmetry in favour of studies with increased LOS among verified centres (Fig. 5). Our GRADE assessment suggested that the evidence was of very low quality (Appendix 1, Table S2).
Studies excluded from meta-analysis showed mixed and inconsistent results concerning the association between verification and use of various resources. For instance, Alexander and colleagues50 found that pediatric verification was associated with a decrease in the average number of blood products transfused per patient (7.2 units v. 2.4 units, 95% CI −10.1 to 0.6) and in professional charges (US–$16 171, 95% CI −$30 898 to −$1362). However, Piontek and colleagues45 reported that, after verification of a level II trauma centre, there was an increase in ventilation use (RR 1.30, 95% CI 1.12 to 1.51). Horton and colleagues36 surveyed level I and II trauma centres and found that American College of Surgeons verification was a predictor of rFVIIa use (OR 3.74, 95% CI 1.53 to 9.09) (Table 2).
Adverse events
Of the 7 studies that reported adverse events,7–10,14,45,50 3 adjusted for potential confounders.7,9,14 Four studies were cross-sectional, and 3 were pre–post. Two studies focused on injured pediatric patients, 1 study focused on adults, and 1 study focused on both pediatric and adult patients. Investigated outcomes included a wide range of complications such as pneumonia, pulmonary emboli, unplanned intubation, unplanned return to the operating room and unplanned readmission.
We did not conduct a meta-analysis of results for adverse events owing to the diversity of outcomes investigated; rather, we report them narratively. After risk adjustment, Agrawal and colleagues7 found lower odds of complications in verified centres than in state-designated centres (OR 0.88, 95% CI 0.87 to 0.90). Schubert and colleagues9 found a positive association between verified centres and unplanned intubation, especially among level I trauma centres (RR 1.53, 95% CI 1.11 to 1.65), after adjusting for hospital and patient characteristics. They did not observe an association between verification and unplanned return to the operating room. Piontek and colleagues45 found low evidence for changes in the incidence of complications (RR 1.27, 95% CI 0.86 to 1.89) or unplanned 30-day readmission (RR 0.91, 95% CI 0.77 to 1.08) after verification of a level II trauma centre. Likewise, Alexander and colleagues50 did not find an association between re-verification of a pediatric trauma centre and unplanned 30-day readmission at an already verified adult level I centre. The small number of patients (126) and readmissions (2), however, limit the interpretation of their findings. Conversely, Choi and colleagues8 found a decrease in unplanned hospital readmissions (RR 0.36, 95% CI 0.15 to 0.87) and hospital-acquired pneumonia (RR 0.41, 95% CI 0.17 to 0.99) 2 years after verification of a level I trauma centre. Grossman and colleagues,14 using a representative sample (n = 94) of US trauma centres, found that verified centres had a lower incidence of major complications (based on the National Trauma Data Bank definition54) than non-verified centres. This association was higher among older adults (OR 0.40, 95% CI 0.27 to 0.60) and children with an Injury Severity Score greater than 24 (OR 0.23, 95% CI 0.12 to 0.47). Finally, Smith and colleagues10 observed fewer cases of acute respiratory distress syndrome in verified level I trauma centres than in state-designated centres (RR 0.91, 95% CI 0.84 to 0.99) (Table 2).
Processes of care
In 4 of the 12 included studies, the authors adjusted for at least 1 potential confounder.34,35,41,52 Six studies focused on pediatric patients,39,41,46,49,50,52 2 on adults,34,37 and 4 on both pediatric and adult patients.34,36,38,44 There were 7 cross-sectional studies and 5 pre–post designs.
We did not conduct a meta-analysis owing to the diversity of outcomes investigated. In the pediatric population, a reduction in the incidence of splenectomy was found by Murphy and colleagues49 (2.7% among verified trauma centres v. 11% among nonverified centres) and Alarhayem and colleagues39 (6% among verified trauma centres v. 13% among nonverified centres) following verification of a pediatric level 1 centre. Alexander and colleagues50 observed a decrease in splenic interventions (i.e., splenectomy, splenorrhaphy or embolization) among children with blunt splenic injuries following pediatric verification (RR 0.36, 95% CI 0.13 to 0.99). Ehrlich and colleagues46 reported an improvement in pediatric trauma patient evaluation (including radiology) and time to emergency department discharge (< 120 min) following verification of a combined adult/pediatric level I trauma centre. Finally, after adjusting for Injury Severity Score, Bogumil and colleagues41 observed a higher prevalence of nonaccidental trauma in verified pediatric centres than in nonverified centres (prevalence ratio 1.81, 95% CI 1.73 to 1.90). This association was higher in level I centres (prevalence ratio 1.89, 95% CI 1.80 to 1.98) than level II centres (prevalence ratio 1.62, 95% CI 1.51 to 1.75).
Surveys of all state-designated US trauma centres in 2000 and 2006 showed that verified centres had a higher likelihood of full compliance with published guidelines for the management of severe traumatic brain injury for both 2000 (OR 5.1, 95% CI 1.1 to 23) and 2006 (OR 1.55, 95% CI 1.00 to 2.40).32,35 Similarly, Theologis and colleagues38 reported that verified level I trauma centres had a higher proportion of compliance with cervical spine clearance protocols than nonverified centres (75% v. 54%). Kim34 did not find any association between verification and time to surgery in patients with head injuries. Finally, Richardson and colleagues44 found that verification of a level III trauma centre was associated with an increase in the proportion of admissions of transferred patients into a referent level I centre (RR 1.19, 95% CI 1.03 to 1.36) (Table 2).
Discussion
In this systematic review and meta-analysis, we found mixed and inconsistent results for the association between trauma centre verification by the American College of Surgeons and all outcomes studied (in-hospital mortality, adverse events, resource use and processes of care). Nonetheless, verification was imprecisely associated with decreased mortality and longer LOS. Some evidence pointed to positive associations between verification and some processes of care, including adherence to published guidelines and reductions in the occurrence of complications. These findings, however, should be interpreted with caution given serious methodologic concerns about the quality of the empirical evidence.
First, inference of the obtained estimates is limited by the unclear nature of the control group in each study. For instance, in cross-sectional studies (18/29), it was not possible to distinguish those that failed during the verification process from those that never applied among nonverified centres. In addition, a quarter of multicentre studies (5/18) combined centres that had no trauma designation and state-designated centres as nonverified centres. If we placed ourselves in a trial framework, the results obtained from these studies would be neither intention-to-treat nor per-protocol estimates. This issue leads to selection and healthy-user biases, in the sense that high-performing centres may be more willing to seek verification than low-performing centres. This may affect the validity and generalizability of the observed associations. It is important to note that not all included studies assessed the impact of verification as the primary objective.
Although pre–post studies are less vulnerable to the biases mentioned above, they cannot account for the underlying trend in the measured outcomes before verification,55 which can bias estimates in either direction. An interesting alternative to assess verification benefits would be the use of quasiexperimental designs such as difference-in-differences and interrupted time-series, which are frequently used to assess the impact of policy and other population-level interventions in health research. These methods can account for unobservable or unmeasured variables that are fixed over time, and for secular trends in outcomes.56,57
Second, preparation for verification visits may lead to improvements in measured outcomes and therefore bias estimates of associations. Only 3 studies8,46,51 accounted for this, by removing the period just before verification in the analysis or via stratification. Of the 73 articles excluded from our review, 6 were excluded because the authors assessed only the preparation for verification visits.58–63
Finally, issues related to analytical methods may have biased the results. For instance, 45% of studies did not adjust for centre-level or patient-level risk factors. The latter is necessary to account for the changing epidemiologic features of trauma populations (e.g., due to population aging and possible change in referral patterns generally attributed to increased marketability).8 In addition, several papers reported ORs as a measure of association, but ORs are known to overestimate RRs, especially when the outcome is common.64 Only one-third (6/18) of multicentre studies accounted for this in their analysis, which may have led to type I errors and CIs that were too narrow.31 Also, the competing risk of death was not considered when LOS was assessed, and missing data were rarely handled appropriately (Table 3).
Our findings are similar to those of previous systematic reviews assessing verification in other health care fields65–69 that showed that many of the studies were heterogeneous and highly vulnerable to confounding, and added little clarity or guidance. They also highlighted major methodologic challenges such as self-selection and lack of robust controls, which limit their inference.
Limitations
Although the uptake of trauma centre verification is rising worldwide, all included articles were from the US. Pediatric patients were overrepresented. The inclusion of multiple study designs provided a more comprehensive assessment of the relevant literature; however, it introduces substantial heterogeneity, which, in turn, affects the robustness of meta-analysis estimates. Our choice of random-effects meta-analysis was based on the assumption that there might not be a common RR or OR applicable to all trauma populations.70 The small number of studies included in our meta-analysis made it difficult to properly summarize estimates and interpret funnel plots. Nonetheless, publication bias seems to be more likely in crude than in adjusted analyses. We also noted that several large studies fell outside the projected lines of the funnel plots, which indicates substantial variability among studies with small SEs.71 Since trauma-verification standards have evolved with time, we were unable to stratify our results by time, which may have introduced a bias.3 Studies were excluded from meta-analyses because of missing CIs or SEs and the scale of effect measure used, despite our efforts to compute desired statistics when raw data were available. Finally, the quality and strength of the cumulative evidence (as assessed with the GRADE framework) was very low.72
Conclusion
Our review illustrates the inability to extrapolate or infer causality on the effectiveness of trauma centre verification from the published literature owing to significant methodologic challenges, such as the lack of robust controls and the concentration of all the available studies in the US. Considering the prevalence and spread of trauma verification globally, this systematic review and meta-analysis underscores the need for quasiexperimental studies that assess the impact of trauma centre verification on changes in clinical processes of care and outcomes. Such studies may provide solid evidence to guide policy-making and individual hospitals’ decisions to seek verification.
Acknowledgement
The authors acknowledge librarian Andrea Quaiattini for her help in refining the research question, keywords and MeSH terms for the preliminary search strategy.
Footnotes
Competing interests: None declared.
Contributions: B. Batomen, L. Moore, A. Nandi, M. Carabali, P.-A. Tardif and H. Champion designed the study. B. Batomen and M. Carabali acquired the data extraction, which B. Batomen, L. Moore, M. Carabali, H. Champion and A. Nandi analyzed. B. Batomen and M. Carabali drafted the manuscript, which A. Nandi, P.-A. Tardif, L. Moore, M. Carabali, H. Champion and B. Batomen critically revised. All authors gave final approval of the article to be published.
- Accepted February 19, 2020.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/