Summary
Medical error is common during trauma resuscitations. Most errors are nontechnical, stemming from ineffective team leadership, nonstandardized communication among team members, lack of global situational awareness, poor use of resources and inappropriate triage and prioritization. We developed an interprofessional, simulation-based trauma team training curriculum for Canadian surgical trainees. Here we discuss its piloting and evaluation.
Effective trauma resuscitation requires the coordinated effort of an interdisciplinary team. Medical error is common during trauma resuscitations, occurring even in well-resourced settings with experienced practitioners. The majority of errors are nontechnical in nature, stemming from ineffective team leadership, nonstandardized communication among team members, lack of global situational awareness, poor use of resources and inappropriate triage and prioritization. Lessons from high-hazard, high-reliability industries have therefore informed the development of medical and surgical crisis resource management (CRM) principles and team training applications to address local and domain-specific needs. There is currently no formally derived or systematically evaluated nontechnical skills curriculum for Canadian surgical trainees. Simulation-based CRM training provides an opportunity for focused instruction, deliberate practice, feedback and assessment on the communicator, collaborator and manager roles espoused by the CanMEDS framework in a manner that is consistent and reproducible and poses no threat to patients. We describe the development, piloting and multilevel evaluation of a novel, interprofessional, simulation-based trauma team training curriculum for Canadian surgical trainees: Trauma Non-Technical Training (TNT-2).
The TNT-2 currirulum was piloted as a prospective, 2-phase, single blinded education study and was informed by a Best Evidence Medical Education systematic review1 and a focused literature review by the authors.2 The strength of the TNT-2 study stems from the multilevel curriculum assessment, with objective assessment of team performance as the primary study outcome.
The course was held at the Allen Waters Family Simulation Centre at St. Michael’s Hospital in Toronto, Ont., and used SimMan (Laerdal), an operator-controlled, full-sized human patient simulator. The “health” of SimMan can be interactively manipulated on the basis of management decisions. The simulation theatre was set up to resemble a trauma room, and teams had access to equipment and diagnostic test results in real time. Two confederates were trained to respond in a standardized fashion according to scenario scripts. A single experienced operator controlled all scenarios, and teams were video-recorded for subsequent blinded evaluation.
All second-year trainees from the General Surgery program at the University of Toronto were invited to participate. In addition, 4 experienced trauma nurses were recruited to participate. Nineteen individuals participated in the first phase, and 9 (6 trainees and 3 nurses) elected to participate in the follow-up phase.
On the training day, participants were randomly divided into teams comprising 3 trainees, 2 nurses and a facilitator with extensive trauma and education experience. Teams received a standard orientation to the simulator environment, followed by a 20-minute prebriefing course, which included a question and answer period with the facilitator and a viewing of a segment of the film The Deadliest Plane Crash about the Tenerife aviation catastrophe.
Teams participated in 3 peer-reviewed standardized trauma resuscitation scenarios lasting 15 minutes each that were designed to highlight 1 or 2 nontechnical, team-based challenges known to encumber effective trauma team performance. The manner in which participants responded to and negotiated these nontechnical challenges then formed the basis for subsequent debriefings. Debriefings consisted of 30 minutes of feedback by the facilitators and focused on scenario-specific learning objectives, such as managing authority gradients, recognizing and avoiding fixation errors, resource utilization, problem solving and team communication skills.
Six months after their initial training, participants voluntarily returned to the simulation centre to participate in the second phase using new scenarios with similar training goals, objectives and levels of difficulty. Participants were rerandomized into teams and facilitated through 3 scenario–debriefing couplets in a manner similar to the first phase.
The impact of TNT-2 was evaluated on multiple levels. Our primary outcome, team performance, was assessed using the Mayo High Performance Team Scale (MHPTS), which rates team performance based on the consistency of 16 different criteria and is designed to evaluate global team performance.3 We used the Ottawa Global Rating Scale (OGRS) to evaluate the performance of the team leader; the scale consists of 5 nontechnical criteria and a global assessment score.4 The OGRS and MHPTS were completed by the team facilitator immediately after each scenario and before the debriefing. Two trained blinded reviewers subsequently analyzed recorded data and rated performance on the MHPTS and OGRS scales.
To measure attitudinal shifts occurring as a result of training, participants completed the Human Factors Attitudes Survey (HFAS) before and after each scenario on each training day. Positive changes in HFAS scores reflect improved attitudes toward team-based behaviours.5 We quantified participants’ perceptions of training using the Participant Evaluation of Training Quality (PETQ) tool. The PETQ questionnaire measures the face validity of the simulation environment, the perceptions on the usefulness of the intervention and perceived personal benefit. The HFAS and PETQ scores were secondary study outcomes.
From our preliminary analysis, the TNT-2 curriculum was feasible and well received, and it provided a strategy for teaching and evaluating key nontechnical behaviours pertinent to team-based trauma resuscitation. The MHPTS and OGRS scores improved but did not reach statistical significance. Considering the small number of participants, this result was not surprising. Decay in CRM skills by trainees did not occur, and the 6-month interval between phases 1 and 2 is the longest reported to date for this type of training, suggesting that biannual instruction may be sufficient for nontechnical skill maintenance. Moreover, to our knowledge, TNT-2 is the first resident study to use team performance, as evaluated by a validated behavioural rating scale, as the primary outcome.
Significant, positive changes in HFAS scores suggest that an important attitudinal shift regarding team-based behaviours occurred as a result of TNT-2 training. Several observations on the PETQ, including scenario realism and satisfaction with high-fidelity simulation training, averaged greater than 5 on a 6-point Likert scale. In addition, there was no change in PETQ scores across all scenarios in both phases, which suggests that standardized, realistic trauma team training can be created and maintained even after initial training and exposure.
Finally, there was excellent agreement between the 2 blinded reviewers and between blinded assessment and unblinded, real-time raters (facilitators) using the MHPTS and the OGRS. This result informs the argument that behavioural rating scales may be reliable for real-time assessment and feedback.
Footnotes
This work was presented at the 2013 Canadian Association of Emergency Physicians Annual Conference, Vancouver BC, June 5, 2013.
Competing interests: This work was supported in part by funds from a University of Toronto Education Development Fund grant (N. Ahmed and C.M. Hicks.), a St. Michael’s Hospital Keenan Research Centre Scholarship and a Queen’s University Research Studentship (A.G. Doumouras).
Contributors: All authors contributed substantially to writing and revising and to the conception and design of the manuscript and approved the final version submitted for publication.
- Accepted January 27, 2014.