Is there an association between implementation of a medical team training program and surgical mortality?

Commentary

The development of Crew Resource Management, the team training curriculum used in aviation, was led by human factors expert, Robert Helmreich. As early as 1993, Helmreich recognized that the principles of communication and team coordination that improved the performance of flight crews were applicable to medicine.1 Nearly 2 decades later, these concepts of teamwork are acknowledged as important. However, the penetrance into health care institutions has been relatively limited. The stated goals of team training programs include improved patient safety and staff satisfaction. An early challenge in the implementation of these programs has been to identify metrics by which the goals can be measured.

Because airplane accidents are rare, demonstrating improved safety with Crew Resource Management proved challenging. Helmreich developed the Safety Attitudes Questionnaire as a way to measure perceived team-work as an indirect quality measure. This questionnaire was subsequently adapted for health care purposes to measure the attitudes of operating room team members. Early reports on team training used the Safety Attitudes Questionnaire as a measure of effectiveness.2 More recently, investigators have been able to link team training programs to improved team performance in terms of efficiency and compliance with performance measures, such as preoperative antibiotic administration.3 This article by Neily and colleagues adds to their previous report that demonstrated improved patient outcomes following implementation of a team training curriculum in the VHA system.4

In the present study, the authors examined a retrospective health services cohort using a contemporaneous control group. At VHA hospitals where team training was implemented, investigators compared surgical mortality following the intervention with the centre’s own baseline surgical mortality and with that of facilities where no intervention had been implemented. Of the 108 facilities included in the study, 74 had participated in team training between 2006 and 2008, and 34 had not yet received the training. The intervention involved training clinicians to work as a team; challenge each other when they identify safety risks; conduct checklist-guided preoperative briefings and postoperative debriefings; and implement other communication strategies, such as recognizing red flags, establishing rules of conduct for communication, stepping back to reassess a situation, and communicating effectively during care transitions between clinicians. Data on the covariates and outcomes were collected from the VASQIP data. The main outcome measures were observed and risk-adjusted 30-day mortality. Observed mortality was defined as any-cause death in the 30 days after a major noncardiac surgery performed at a VA medical centre. Risk-adjusted mortality was defined as observed mortality adjusted for each patient’s risk profile (surgical complexity, patient comorbidity and socio-demographic characteristics). Patients in the intervention versus the nonintervention groups were stratified using propensity scores and matched to assess the effect of the intervention/exposure.

Neily and colleagues noted that, when comparing the before and after data at the 74 trained facilities, there was an 18% decrease in the annual observed mortality (RR 0.82, p = 0.01). At the 34 untrained facilities there was a 7% decrease in mortality (RR 0.93, p = 0.59). Propensity score matched analysis showed a 50% greater decrease in annual mortality in the trained (RR 1.49, p = 0.01) compared with the nontrained group. Interestingly, the investigators found a dose-dependent relationship for increasing quarters of training. Mortality decreased by 0.5 deaths per 1000 procedures for every quarter (p = 0.001).

Two other before and after studies evaluating the effect of using a surgical safety checklist on postoperative mortality found that use of a checklist alone could reduce surgical mortality by 50%.5,6 By comparison, Neily and colleagues showed that medical team training resulted in an 18% reduction in postoperative mortality in the trained cohort. This reduction in postoperative mortality was 50% greater than that in the nontrained cohort in the propensity-matched analyses. While different studies need to be compared cautiously, it is interesting to note that the much more simple, quick and inexpensive intervention of a surgical safety checklist appears to be as or more effective at reducing postoperative deaths than the multidimensional intervention team-training evaluated in the present study. As such, decision-makers may be less likely to be persuaded to adopt medical team training as a method to improve surgical outcomes.

A major shortcoming of the present study was its retrospective nature. No randomized controlled trial has demonstrated a benefit of perioperative communication tools (e.g., surgical safety checklists) or behavioural interventions (e.g., medical team training). The evidence on the effectiveness of these interventions comes solely from before and after studies, which are subject to selection bias and confounding. The gold standard for ensuring that 2 groups are similar in a comparative study is randomization. The rationale behind randomization is that identical confounding variables should be similar in both the intervention and the control group, thus allowing the investigators to conclude that an observed difference is due to the intervention. In most real-life studies, randomization is not possible. Propensity score matching is a method that provides investigators of a nonrandomized study a certain degree of confidence that the intervention and control arms are comparable at baseline. Although propensity-score matching can control for many potential confounders, it cannot remove all potential bias; therefore, the training assignment in the present study cannot truly be considered random.

Notwithstanding these limitations, the results are provocative and may be clinically important. Lower short-term mortality for major surgical procedures has been an important issue in patient safety. Concrete evidence to support suitable and focused interventions toward this goal has been slow to come by. Importantly, the study by Neily and colleagues was able to demonstrate reduction in mortality across a nationwide system of health care facilities. These data seemingly provide evidence to support the implementation of such training and follow-up activities at surgical facilities across the United States. It is important to note, however, that the study was conducted exclusively in the US Veterans Affairs system, which differs substantially from many other health care environments. Most patients in the VA system are men, older and may have more comorbidities than the average patient. The VA hospitals also have different organizational structures and operating room staffing models than non-VA hospitals. In many hospitals, health care personnel do not work with the same people on a consistent basis, limiting their ability to develop team behaviours since there really may be no “team.” Furthermore, the intervention tested in the study included a 1-day training session using lectures, group interactions and videos. Although it may not seem like a major logistical feat, in many typical hospital settings it would be difficult to organize a full-day intervention involving all the appropriate health care providers. As such, while the results of the study may potentially be generalizable to non-VA hospitals, it remains unclear if such an intervention has broader applicability.

Furthermore, explanations of the mechanisms of quality improvement from the team training intervention were not particularly compelling. For example, stories shared by surgical teams in the study about how medical team training improved surgical performance included anecdotes about discovering during a briefing that a patient was anticoagulated or required cardiac clearance. It is not clear why a team briefing would identify these potential problems any better than a routine preoperative assessment by a health care provider. Until there is experimental evidence supporting the effectiveness of perioperative behavioural interventions, it cannot be assumed that they will result in the improvements in perioperative outcomes suggested by the existing literature.

One of the most important learning points in this article is the months of preparation and planning that occurred before the actual team training and the 1-year follow-up to provide support and coaching. Hamman appropriately cautions that medical team training “is not a one-time immunization given in isolation to a professional in a 2-day seminar.7” The success of a team training program is dependent on an institution’s long-term commitment to the principles covered, such as using checklists, flattening hierarchy and adopting rules of conduct for communication. These programs require extensive institutional investment of resources for curriculum infrastructure and time away from work for the participants. In addition, ongoing coaching, encouragement of constructive behaviour and repercussions for individuals who refuse to participate are needed. Armour and colleagues reported that the improved outcomes after team training disappeared after several months, reflecting cultural deterioration.8

In theory, the new behaviours taught in team training courses could become so embedded in the culture of an institution that a separate team training curriculum would no longer be required. The principles of respect, appropriate assertiveness and mutual support would be woven into the education of trainees at all levels. More importantly, more junior members of the team would learn by having exemplary role models. At this time, additional studies that demonstrate improved outcomes with team training are warranted to encourage more institutions to develop such programs.