Early identification of the need for major intervention in patients with traumatic hemorrhage: development and internal validation of a simple bleeding score
=============================================================================================================================================================

* Alexandre Tran
* Monica Taljaard
* Kasim E. Abdulaziz
* Maher Matar
* Jacinthe Lampron
* Ewout W. Steyerberg
* Christian Vaillancourt

## Abstract

**Background:** Failure to rapidly identify bleeding in trauma patients leads to substantial morbidity and mortality. We aimed to develop and validate a simple bedside score for identifying bleeding patients requiring escalation of care beyond initial resuscitation.

**Methods:** We included patients with major blunt or penetrating trauma, defined as those with an Injury Severity Score greater than 12 or requiring trauma team activation, at The Ottawa Hospital from September 2014 to September 2017. We used logistic regression for derivation. The primary outcome was a composite of the need for massive transfusion, embolization or surgery for hemostasis. We prespecified clinical, laboratory and imaging predictors using findings from our prior systematic review and survey of Canadian traumatologists. We used an AIC-based stepdown procedure based on the Akaike information criterion and regression coefficients to create a 5-variable score for bedside application. We used bootstrap internal validation to assess optimism-corrected performance.

**Results:** We included 890 patients, of whom 133 required a major intervention. The main model comprised systolic blood pressure, clinical examination findings suggestive of hemorrhage, lactate level, focused assessment with sonography in trauma (FAST) and computed tomographic imaging. The C statistic was 0.95, optimism-corrected to 0.94. A simplified Canadian Bleeding (CAN-BLEED) score was devised. A score cut-off of 2 points yielded sensitivity of 97.7% (95% confidence interval [CI] 93.6 to 99.5) and specificity 73.2% (95% CI 69.9 to 76.3). An alternative version that included mechanism of injury rather than CT had lower discriminative ability (C statistic = 0.89).

**Conclusion:** A simple yet promising bleeding score is proposed to identify high-risk patients in need of major intervention for traumatic bleeding and determine the appropriateness of early transfer to specialized trauma centres. Further research is needed to evaluate the performance of the score in other settings, define interrater reliability and evaluate the potential for reduction of time to intervention.

Catastrophic hemorrhage is the second leading cause of mortality following a traumatic injury and is responsible for nearly half of all deaths within the first 24 hours.1,2 Uncontrolled bleeding is the most common cause of preventable mortality following both military and civilian trauma.3 Delay in recognizing substantial bleeding results in larger volumes of blood loss, higher resuscitation requirements and severe physiologic derangements. 2 Major interventions to address ongoing hemorrhage may include massive transfusion, hemostatic surgery or angioembolization. The early identification of hemorrhagic injury is a critical component of modern trauma care,4 and an objective, evidence-based approach to bleeding assessment would be invaluable for triage decisions and provision of life-saving interventions.3

The widely adopted Advanced Trauma Life Support (ATLS) guidelines provide an algorithmic approach to initial assessment of the trauma patient including a 4-class system for grading severity and proposed corresponding management of hemorrhage based on clinical examination and disturbances in vital signs.5 However, several studies evaluating the ATLS classification have raised concerns regarding its reliability and usefulness. A large database validation of the Trauma Audit and Research Network in the United Kingdom showed that the classification overestimated the degree of tachycardia and hypotension associated with blood loss.6 Similarly, another multicentre validation by the Trauma Registry of the German Society for Trauma Surgery demonstrated that fewer than 1 in 10 patients could be accurately classified according to the ATLS system.5 In other words, substantial hemorrhage may occur insidiously before obvious disturbances in vital signs appear. In an international survey, only 10.9% of ATLS course directors and instructors considered the ATLS classification of hemorrhagic shock to be a “good guide for fluid resuscitation and blood product transfusion.”7

Although some patients arrive at the hospital with obvious evidence of substantial blood loss and clear indication for intervention, there remains a subset of the trauma population for whom the classical clinical and biochemical signs of hemorrhage may not be as clear. Such patients are at risk of clinical deterioration from ongoing insidious bleeding and these delays in identifying major hemorrhage are responsible for up to 16% of preventable deaths in this setting.4

In this study, we sought to derive and internally validate a clinical prediction model capable of identifying the need for major intervention in patients with traumatic hemorrhage early in the clinical assessment.

## Methods

### Population

This prediction rule study is based on data from the Ottawa Trauma Registry, a prospectively collected database for The Ottawa Hospital, from September 2014 to September 2017. This time frame allowed for to-the-minute electronic blood transfusions, which was established at our institution in 2014. The Ottawa Hospital is the designated level 1 trauma centre for the Champlain Local Health Integration Network (LHIN) in eastern Ontario. Cases included in the database were required to be classified as a major trauma. Major trauma was defined as Injury Severity Score (ISS) greater than 12 or requiring trauma team activation, a standardized definition used across the province of Ontario. We included all patients who arrived alive directly from the scene of the injury or were transferred to the study hospital from another receiving hospital within 3 hours of injury; we excluded patients with a delayed presentation and those who were dead on arrival. We included only patients with blunt or penetrating mechanisms of injury; we excluded those with nonhemorrhagic mechanisms such as burn injury, drowning and electrical injury. We also excluded patients with isolated head injury, defined as patients with an Abbreviated Injury Scale (AIS) score greater than 2 and who did not have injuries of the thorax, abdomen or pelvis. This study was approved by the Ottawa Health Science Network Research Ethics Board.

### Outcomes

The primary outcome was the occurrence of clinically significant bleeding requiring corrective intervention(s) within the first 24 hours of arrival, defined as a composite of the following interventions: massive transfusion (≥ 10 units of packed red blood cells), an operation for hemostasis or angiography with embolization. To classify surgical outcomes, we reviewed all operative reports for eligible patients and included those requiring thoracotomy, laparotomy, pelvic fixation or vascular surgery with a hemostatic intervention. Patients who underwent an exploratory operation without a therapeutic hemostatic procedure within the operative report were placed in the nonevent category. Patients receiving angiography without embolization were similarly placed in the nonevent category. Time from arrival in the emergency department (ED) to occurrence of the first major intervention was determined for all outcomes. Massive transfusion, as assessed by blood product administration records, was considered to have occurred upon the initiation of the 10th unit of packed red blood cells. To account for survivorship bias, we performed a preplanned sensitivity analysis wherein we included hemorrhagic death within the first 24 hours as part of the composite outcome.

### General approach to analysis

We developed our data analysis protocol in accordance with guidelines8,9 for clinical prediction modelling. We prioritized full prespecification of the predictors, use of flexible functions for continuous predictors, and multiple imputation. Analyses were conducted using the rms10 packages in R version 3.4 and SAS version 9.4 software.

### Prespecification of predictors

Model prespecification, based on clinical knowledge and evidence from the literature, minimizes the risk of overfitting in small data sets and improves performance at external validation.9 We conducted a systematic review to identify predictors in the existing literature.11,12 We conducted a national survey of Canadian traumatologists13 and asked respondents to place in rank order a list of potential predictors on the basis of their importance in the evaluation of a patient’s bleeding risk. An adjudication committee of local trauma surgeons (including M.M. and J.L.) reviewed the results of these studies and selected the most appropriate predictors for our prespecified model. Challenges and limitations for predictor selection included clinical relevance and ease of use, availability of institutional data variables and sample size considerations (and available degrees of freedom).

### Data coding, missing values and collinearity

Data collection was completed by a single reviewer (A.T.) using a prepiloted data extraction form. Data points of interest included patient demographic characteristics, initial vital signs and physical examination findings suggestive of hemorrhage, complete blood count, blood gas levels, results of coagulation testing, findings from focused assessment with sonography in trauma (FAST) and computed tomography, as well as occurrence and timing of outcomes (massive transfusion, hemostatic surgery or embolization). Restricted cubic splines were used to flexibly model continuous variables.

From an exploratory analysis of missing data, we identified 2 variables for which values were missing for more than 25% of cases: FAST findings and lactate level. For all other predictors, values were missing for fewer than 5% of cases. Following consultation with clinical experts in trauma and laboratory medicine, we determined that the high prevalence of missing values for the FAST and lactate variables was likely a result of low-risk patients who have values within a normal clinical range. To evaluate the appropriateness of this assumption, we used deterministic imputation. We categorized these variables as either abnormal or not (test was performed and result was normal or test was not performed and result was assumed to be normal), using prespecified thresholds of known clinical importance with all missing values assumed to be normal. We considered findings to be abnormal if FAST had been performed and the result was positive and if lactate level had been measured and the result was greater than 5 mmol/L. Otherwise, these data were considered negative or less than 5 mmol/L, respectively (normal findings). We generated imputed values using the aregImpute function in the Hmisc library.10 This procedure uses flexible parametric additive regression spline models fit to bootstrap resamples to predict the missing values under an assumption of missing at random. Before modelling, target variables are first optimally transformed. Actual values used for imputation are obtained using predictive mean matching, by using donor observations from complete cases matched to the predicted values for missing cases. The imputation model consisted of the full list of predictor variables, outcomes and available ancillary variables (age, sex and time to hospital). We determined that the imputed values for FAST and lactate were in the study-defined normal range for more than 90% of cases and that the prediction model did not change, thus satisfying the appropriateness of our previous assumption. Variance inflation factors were used to examine multicollinearity, and we reduced collinear relationships (pH/base excess, hematocrit/hemoglobin) to single variables (base excess, hemoglobin) using clinical judgment.

### Derivation of the full model and simplification

The adjudication committee prespecified predictors for the model to ensure a minimum a 5–10 events per predictor.9 An initial logistic regression model was fit with continuous predictors modelled using restricted cubic splines. An initial plot of partial associations corrected for the number of degrees of freedom, referred to as an analysis of variance (ANOVA) plot, was generated to place in rank order the predictors in terms of their strengths of association with the outcome. The plot was used to inform the decision about how many degrees of freedom to allocate to each variable in the final model: strong associations were modelled with greater complexity than weak associations. Associations were expressed using odds ratio (ORs) and 95% confidence intervals (CIs). For continuous predictors, ORs were expressed comparing the 75th to 25th percentile values. To derive an approximation of the full model excluding predictors with low prognostic value, we used a stepdown approach,8 setting a rule for selection based on the Akaike information criterion. Stepdown models improve practicality with fewer predictors for use in clinical practice.

### Model performance and internal validation

Model discrimination was assessed by the concordance index (C statistic). Internal validation was performed to minimize the risk of overfitting, which describes the phenomenon when a new model describes the derivation population well but fails to provide accurate predictions for new subjects.9 We therefore internally validated the model using the bootstrap method, which maximizes sample size while minimizing potential overfitting, as opposed to using split or cross validation, which is considered less efficient.9 We validated the model using 1000 bootstrap samples to provide a more realistic measure of performance in external populations using the optimism-corrected C statistic. We assessed model discrimination using box plots of predicted values to ensure separation of groups with and without the outcome of interest. Model calibration was assessed graphically using plots that compared the observed and predicted probabilities, using a smoothing technique.14

### Creation of a scoring system

From the stepdown model, we generated a nomogram using the built-in nomogram function in the Hmisc package in R,10 and we used it to determine the number of points associated with each predictor. For continuous predictors, the points were determined to correspond to midpoints of clinically meaningful intervals of values. We calculated the total number of points for each patient in the analysis and assessed sensitivity, specificity, positive predictive value and negative predictive value corresponding to different cut points for the total number of points. The C statistic for the simplified model was compared with that of the full model and the reduced model to ensure minimal change in performance.

## Results

### Patient characteristics and outcomes

A total of 890 patients were included in the analysis, of whom 133 had the event of interest (at least 1 of the 3 pre-determined composite outcomes representing major intervention for hemorrhage). Patient characteristics and outcomes are presented in Table 1. The most common mechanism of injury was high-velocity blunt injury (572 patients, 64.3%). There were 182 patients (20.5%) transferred from peripheral hospitals with a median transfer time of 1.0 hours (interquartile range [IQR] 0.5 to 2.0 h). FAST was performed in 545 patients (61.2%), of whom 92 (16.9%) had positive results. CT imaging was performed in 778 patients (88.5%); for 219 of these patients (28.1%) the results demonstrated free fluid and for 74 (9.4%) the results demonstrated contrast extravasation. There were 63 (7.1%) patients requiring only hemostatic surgery, 22 (2.5%) patients requiring only embolization and 6 (0.7%) patients requiring only massive transfusion; the remaining 42 (5.4%) patients required 2 or more major interventions. Fourteen (1.6%) patients died from hemorrhage within the first 24 hours. A preplanned sensitivity analysis including these hemorrhagic deaths as part of the composite outcome did not significantly alter the model.

View this table:
[Table 1](http://canjsurg.ca/content/63/5/E422/T1)

Table 1 
Patient characteristics and outcomes

### Fully prespecified and reduced models

The full prespecified model is presented in Appendix 1 (available at canjsurg.ca/010619-a1). The reduced multivariable model, after the stepdown procedure was used, included 5 predictors: clinical examination findings suggestive of hemorrhage (OR 17, 95% CI 7.8 to 38), FAST (OR 2.4, 95% CI 1.2 to 4.7), lactate level greater than 5 mmol/L (OR 6.2, 95% CI 2.6 to 15), free fluid or contrast extravasation on CT (OR 21, 95% CI 9.2 to 50) and systolic blood pressure (OR 4.9, 95% CI 2.6 to 9.0, comparing patients with systolic blood pressure of 80 v. 120 mm Hg) (Table 2). The modelled association with blood pressure is presented graphically in Appendix 1. Given that an infinite number of odds ratios can be generated for systolic blood pressure depending on the 2 points chosen for comparison, a comparison of 80 versus 120 mm Hg is provided to illustrate an example odds ratio based on values above and below 90.

View this table:
[Table 2](http://canjsurg.ca/content/63/5/E422/T2)

Table 2 
Final multivariable model

The C statistic for the full prespecified model was 0.96; the reduced model after the stepdown procedure was used had a naïve C statistic of 0.94 (95% CI 0.93 to 0.96), corrected for optimism to 0.94, demonstrating that there was little overfitting in the model. The calibration plot for the full model demonstrated excellent calibration with a calibration slope of 1 (95% CI 0.84 to 1.16, Appendix 1). The reduced model also showed an acceptable calibration slope (0.94, 95% CI 0.85 to 1.15, Appendix 1).

### Scoring system

The resulting scoring system, called the Canadian bleeding (CAN-BLEED) score, has a total of 8 points and is presented in Table 3. Clinically meaningful categorization thresholds of 90 and 120 mm Hg were chosen to assign scores for systolic blood pressure on the basis of the results of a previous survey.8 The remaining categorical variables require simple yes or no answers. The C statistic for the model with the CAN-BLEED score as the only predictor was 0.94 (95% CI 0.87 to 0.98). The calibration plot (Appendix 1) showed excellent agreement between observed and predicted probabilities across the entire spectrum of risk (calibration slope 1, 95% CI 0.85 to 1.15).

View this table:
[Table 3](http://canjsurg.ca/content/63/5/E422/T3)

Table 3 
Canadian Bleeding (CAN-BLEED) score: main model

The classification performance of the 8-point CAN-BLEED score is presented in Table 4, organized by 1-point increments. A score cut-off of 2 points yielded a sensitivity of 97.7% (95% CI 93.6 to 99.5), a specificity of 73.2% (95% CI 69.9 to 76.3), a positive predictive value of 39.0% (95% CI 33.8 to 44.5) and a negative predictive value of 99.5% (95% CI 98.5 to 99.9).

View this table:
[Table 4](http://canjsurg.ca/content/63/5/E422/T4)

Table 4 
Classification table for the CAN-BLEED score: main model (with CT)

### Time to first major intervention

The median time to first major intervention for all 133 patients was 2.00 hours (IQR 0.88 to 4.00 h) (Table 5). Of these patients, 49 (36.8%) received their first intervention more than 3 hours after arrival to the ED, 20 (15.0%) after more than 6 hours and 6 (4.5%) after more than 12 hours. The patients requiring only hemostatic surgery generally received an intervention the most quickly, with a median time of 1.50 hours (IQR 0.50 to 3.00 h).

View this table:
[Table 5](http://canjsurg.ca/content/63/5/E422/T5)

Table 5 
Time to first major intervention

### Supplementary analyses

The CAN-BLEED score includes CT findings, which may not be available in some settings. To accommodate traumatic hemorrhage evaluation in lower resource settings, we conducted additional analyses to derive an alternative score that excludes CT. We repeated the model derivation process but excluded CT as a candidate predictor. The stepdown procedure selected the same variables as for the CAN-BLEED score with the addition of mechanism of injury. To preserve simplicity and promote the usefulness of the alternative version, we retained the original number of points allocated to each variable in CAN-BLEED but replaced the 2 points allocated to CT with 1 point allocated to penetrating mechanism of injury. We found no significant difference in C statistic performance between the full no-CT model (C = 0.91, 95% CI 0.84 to 0.98) and the simplified no-CT model (C = 0.89, 95% CI 0.82 to 0.93). The alternative 7-point version that excludes CT is referred to as the CAN-BLEED (no CT) model (Table 6).

View this table:
[Table 6](http://canjsurg.ca/content/63/5/E422/T6)

Table 6 
Canadian Bleeding (CAN-BLEED) score: no-CT model

For the CAN-BLEED (no CT) model, the calibration slope was 1.00 (95% CI 0.85 to 1.15), and the classification performance is presented in Table 7. A score cut-off of 1 point yields 95.5% sensitivity (95% CI 90.4 to 98.3), 48.9% specificity (95% CI 45.3 to 52.5), 24.7% positive predictive value (95% CI 21.0 to 28.7) and 98.4% negative predictive value (95% CI 96.6 to 99.4)

View this table:
[Table 7](http://canjsurg.ca/content/63/5/E422/T7)

Table 7 
Classification table for the CAN-BLEED score: no-CT model

Noting the potential for competing risk and survivorship bias, we conducted a sensitivity analysis with a 4-outcome composite that additionally included death from hemorrhagic causes within 24 hours. There was no change to the main CAN-BLEED model.

## Discussion

We developed and internally validated a simple bleeding score for early identification of patients requiring major intervention for traumatic hemorrhage. We created the CAN-BLEED score to include CT, which is available at more specialized tertiary care centres. We created an alternate version for other settings, comprised of predictors for which data are typically available to the clinician during the first hour of trauma resuscitation: mechanism, systolic blood pressure, clinical examination findings suggestive of hemorrhage, FAST results and lactate level. Evaluation of both scores demonstrated excellent overall performance with a C statistic of 0.94 for CAN-BLEED and 0.89 for the no-CT version. Most importantly, a score cut-off of 1 point for the no-CT model and 2 points for the main model demonstrated sensitivities greater than 95% and negative predictive values greater than 98% while maintaining moderate to good specificity, suggesting that these models offer considerable potential as a screening tool.

Despite well-established principles emphasizing minimization of time to bleeding control and resuscitation,15 challenges remain in terms of accurately identifying or ruling out hemorrhagic shock.3 Despite an abundance of available diagnostic tests, clinical gestalt for identification of bleeding trauma patients requiring major intervention continues to perform quite poorly, demonstrating 65.6% sensitivity, 63.8% specificity and a C statistic of 0.63.16 Within our study cohort, the time to intervention was more 3 hours for 1 in 3 patients and more 6 hours for 1 in 6 patients, which suggests the possibility of delayed recognition of insidious bleeding or systemic issues in timely protocol activation. With the trauma community recognizing the fallibility of clinical judgment and the need for validated, objective approaches to prognostication for traumatic hemorrhage,3 clinical prediction models have grown in popularity over the last few years.12 However, the methodologic quality of the traumatic hemorrhage prediction literature has generally failed to meet guideline-recommended standards for development and validation.12 In addition, many of the existing modelling studies focus on massive transfusion as a sole outcome, a concept shown to be vulnerable to competing risk and survivorship bias.17–19

Our score shares many of the same elements as the previously developed ABC and TASH scores for prediction of massive transfusion, notably mechanism,20 systolic blood pressure,20,21 FAST ultrasound20,21 and pelvic instability. 21 The 6 components (clinical examination findings suggestive of hemorrhage, systolic blood pressure, lactate level, positive FAST results, mechanism and free fluid or contrast extravasation on CT) of our proposed scores all received grade 1 recommendations by the European guideline on management of major bleeding for inclusion in the standard assessment for hemorrhage. 15 Our proposed score offers the advantage of having been explicitly derived and validated for a composite bleeding outcome that better captures patients with clinically significant bleeding than massive transfusion alone.19 A direct comparison of performance with the ABC and TASH scores is not within the scope of this study given that those scores were derived for and are applied for a different target population and purpose (massive transfusion protocol activation only).

The proposed score, with and without CT, allows for earlier identification of high-risk patients and may potentially reduce the time to intervention or time to interhospital transfer to specialized trauma centres. The “time to request,” which is the time between initial hospital arrival and the decision by the emergency physician to request a transfer, can often extend well beyond 2 hours.22,23 Importantly, up to 8% of transferred patients may require critical interventions within the first 15 minutes of arrival at the trauma centre; this cohort has a 52% mortality rate.23 Previous work has highlighted the effectiveness of objective triage protocols and checklists in reducing transfer times for trauma patients arriving at nontrauma centres.23 These considerations highlight trauma patients arriving at hospitals without definitive trauma care as a potentially vulnerable group that would benefit from targeted early prognostication and transfer. With regard to Canadian trauma care specifically, previous work has demonstrated substantial variability in the degree of trauma system development across the country. 24 In particular, fewer than half of the provinces had an inclusive trauma system model or interfacility transfer agreements, which suggests that there may be opportunities to implement procedures and processes that would improve the inclusiveness of care. This score offers considerable potential as a screening tool that would enable clinicians in nontrauma centres to quickly and effectively assess the need to transfer patients who may be bleeding to level 1 trauma centres for definitive management.

Strengths of this study include the rigorous adherence to prediction modelling methodology standards25 and the comprehensive process of model prespecification using the best available literature and clinical expertise. In addition, we sought to maintain practicality and clinical functionality during model development, incorporating clinical expertise and resource availability for determination of model composition and a final simple scoring system corresponding to clinically meaningful thresholds.

### Limitations

Our model development is based on data from a Canadian tertiary care centre with specialized expertise and resources, and thus the model needs to be independently validated across a variety of settings. In addition, we acknowledge that interpretations of pelvic stability, visualization of active bleeding and FAST and CT imaging may be subject to variable interrater reliability. However, our pragmatic approach to data collection, which was based on the documented findings in the radiologists’ reports and the traumatologist consultants’ notes rather than reinterpretation of primary data by a single reviewer, accounts for subjective decision-making. Given these considerations, explicit validation within an interhospital transfer population to account for differences in setting, resources and clinical experience with trauma patients is warranted.

## Conclusion

In this study we adopted robust methodology for clinical prediction modelling and synthesized the best available clinical and evidence-based knowledge to develop and internally validate the Canadian Bleeding (CAN-BLEED) score for early identification of patients requiring major intervention following traumatic hemorrhage. We have proposed simple and extended models that are based on predictors available to the clinician within the first hour of assessment: systolic blood pressure, clinical examination, FAST, lactate level and CT imaging. These scores demonstrate excellent performance, offering considerable promise as a screening tool for early risk stratification of trauma patients who are bleeding and determination of the appropriateness of early interhospital transfer to specialized trauma centres. Future studies are required to evaluate the performance of the scores in other settings, define interrater reliability and evaluate the potential for reduction of time to intervention.

## Footnotes

*   Presented at the 2017 Canadian Surgery Forum, Sept. 14–16, 2017, Victoria, B.C.; the 2018 American College of Surgeons’ Committee on Trauma Residents Trauma Papers Competition, Mar. 8, 2018, San Antonio, Tex.; and the 2018 Trauma Association of Canada Annual Scientific Meeting, Feb. 22–23, 2018, Toronto, Ont.

*   **Competing interests:** None declared.

*   **Contributors:** A. Tran, M. Taljaard, M. Matar, J. Lampron and C. Vaillancourt designed the study. A. Tran, M. Matar acquired the data, which A. Tran, M. Taljaard, K. Abulzaziz, J. Lampron, E. Steyerberg and C. Vaillancourt analyzed. A. Tran and K. Abulzaziz wrote the manuscript, which all authors critically revised. All authors gave final approval of the version of the article to be published.

*   Accepted December 16, 2019.

## References

1.  Cotton BA, Faz G, Hatch QM, et al.Rapid thrombelastography delivers real-time results that predict transfusion within 1 hour of admission.J Trauma 2011;71:407–14.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/TA.0b013e31821e1bf0&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=21825945&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

2.  Kauvar DS, Lefering R, Wade CE.Impact of hemorrhage on trauma outcome: an overview of epidemiology, clinical presentations and therapeutic considerations.J Trauma 2006;60SupplS3–11.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.ta.0000199961.02677.19&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=16763478&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000238477600002&link_type=ISI) 

3.  Shackelford SA, Colton K, Stansbury LG, et al.Early identification of uncontrolled hemorrhage after trauma: current status and future direction.J Trauma Acute Care Surg 2014;77Suppl 2S222–7.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/TA.0000000000000198&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=24770559&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

4.  Tien HC, Spencer F, Tremblay LN, et al.Preventable deaths from hemorrhage at a level 1 Canadian trauma center.J Trauma 2007;62:142–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.ta.0000251558.38388.47&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=17215745&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000243490100027&link_type=ISI) 

5.  Mutschler M, Nienaber U, Brockamp T, et al.A critical reappraisal of the ATLS classification of hypovolaemic shock: Does it really reflect clinical reality?.Resuscitation 2013;84:309–13.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.resuscitation.2012.07.012&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=22835498&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000318164200017&link_type=ISI) 

6.  Guly HR, Bouamra O, Spiers M, et al.Vital signs and estimated blood loss in patients with major trauma: testing the validity of the ATLS classification of hypovolaemic shock.Resuscitation 2011;82:556–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.resuscitation.2011.01.013&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=21349628&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000291340600013&link_type=ISI) 

7.  Mutschler M, Hoffmann M, Wolfl C, et al.Is the ATLS classification of hypovolaemic shock appreciated in daily trauma care? An online-survey among 383 ATLS course directors and instructors.Emerg Med J 2015;32:134–7.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzMi8yLzEzNCI7czo0OiJhdG9tIjtzOjE5OiIvY2pzLzYzLzUvRTQyMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  Harrell FE (2001) Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis (Springer, New York (NY)), 2nd ed.
    
    

9.  Steyerberg EW (2009) Clinical prediction models: a practical approach to development, validation and updating (Springer, New York (NY)).
    
    

10. Harrell FE (May, 2018) Package ‘Hmisc’ version 5.2, Available: [https://cran.r-project.org/web/packages/rms/rms.pdf](https://cran.r-project.org/web/packages/rms/rms.pdf). accessed 2019 Jan 15.
    
    

11. Tran A, Matar M, Steyerberg EW, et al.Early identification of patients requiring massive transfusion, embolization or hemostatic surgery for traumatic hemorrhage: a systematic review protocol.Syst Rev 2017;6:80
    
    

12. Tran A, Matar M, Lampron J, et al.Early identification of patients requiring massive transfusion, embolization or hemostatic surgery for traumatic hemorrhage: a systematic review and meta-analysis.J Trauma Acute Care Surg 2018;84:505–16.
    
    

13. Tran A, Matar M, Lampron J, et al.Outcome variation among Canadian trauma centres: toward a clinical prediction rule for standardizing approaches to clinical assessment of hemorrhage.Can J Surg 2017;60:E3–4.
    
    

14. Van Calster B, Nieboer D, Vergouwe Y, et al.A calibration hierarchy for risk models was defined: from utopia to empirical data.J Clin Epidemiol 2016;74:167–76.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2015.12.005&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=26772608&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

15. Rossaint R, Bouillon B, Cerny V, et al.The European guideline on management of major bleeding and coagulopathy following trauma: fourth edition.Crit Care 2016;20:100
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1186/s13054-016-1265-x&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=27072503&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

16. Pommerening MJ, Goodman MD, Holcomb JB, et al.PROMTT Study GroupClinical gestalt and the prediction of massive transfusion after trauma.Injury 2015;46:807–13.
    
    

17. Ho AM, Dion PW, Yeung JH, et al.Simulation of survivorship bias in observational studies on plasma to red blood cell ratios in massive transfusion for trauma.Br J Surg 2012;99:132–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1002/bjs.7732&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=22441868&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

18. Savage SA, Zarzaur BL, Croce MA, et al.Redefining massive transfusion when every second counts.J Trauma Acute Care Surg 2013;74:396–400.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/TA.0b013e31827a3639&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=23354230&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 

19. Tran A, Nemnom MJ, Lampron J, et al.Accuracy of massive transfusion as a surrogate for significant traumatic bleeding in health administrative datasets.Injury 2019;50:318–23.
    
    

20. Nunez TC, Voskresensky IV, Dossett LA, et al.Early prediction of massive transfusion in trauma: simple as ABC (assessment of blood consumption)?.J Trauma 2009;66:346–52.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/TA.0b013e3181961c35&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=19204506&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000263442800008&link_type=ISI) 

21. Yucel N, Lefering R, Maegele M, et al.the Polytrauma Study Group of the German Trauma SocietyTrauma Associated Severe Hemorrhage (TASH)-Score: probability of mass transfusion as surrogate for life threatening hemorrhage after multiple trauma.J Trauma 2006;60:1228–36.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.ta.0000220386.84012.bf&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=16766965&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000238328700018&link_type=ISI) 

22. Utter GH, Victorino GP, Wisner DH.Interhospital transfer of acute trauma patients: How long does it take and how is the time spent?.Clin Med Trauma Intensive Care Med 2008Sept
    
    

23. Harrington DT, Connolly M, Biffl WL, et al.Transfer times to definitive care facilities are too long: a consequence of an immature trauma system.Ann Surg 2005;241:961–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.sla.0000164178.62726.f1&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=15912045&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000229349300025&link_type=ISI) 

24. Evans CC, Tallon JM, Bridge J, et al.An inventory of Canadian trauma systems: opportunities for improving access to trauma care.CJEM 2014;16:207–13.
    
    

25. Steyerberg EW, Moons KG, van der Windt DA, et al.PROGRESS Study GroupPrognosis Research Strategy (PROGRESS) 3: prognostic model research.PLoS Med 2013;10:e1001381
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1371/journal.pmed.1001381&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=23393430&link_type=MED&atom=%2Fcjs%2F63%2F5%2FE422.atom)