Use of a surgical safety checklist to reduce morbidity and mortality
====================================================================

* Steve Latosinsky
* Richard Thirlby
* David Urbach

The term “evidence-based medicine” was first coined by Sackett and colleagues as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”1 The key to practising evidence-based medicine is applying the best current knowledge to decisions in individual patients. Medical knowledge is continually and rapidly expanding. For clinicians to practise evidence-based medicine, they must have the skills to read and interpret the medical literature so that they can determine the validity, reliability, credibility and utility of individual articles. These skills are known as critical appraisal skills, and they require some knowledge of biostatistics, clinical epidemiology, decision analysis and economics, and clinical knowledge.

Evidence Based Reviews in Surgery (EBRS) is a program jointly sponsored by the Canadian Association of General Surgeons (CAGS) and the American College of Surgeons (ACS) and is supported by an educational grant from ETHICON and ETHICON ENDO-SURGERY, both units of Johnson & Johnson Medical Products, a division of Johnson & Johnson and ETHICON Inc. and ETHICON ENDO-SURGERY Inc., divisions of Johnson & Johnson Inc. The primary objective of EBRS is to help practising surgeons improve their critical appraisal skills. During the academic year, 8 clinical articles are chosen for review and discussion. They are selected for their clinical relevance to general surgeons and because they cover a spectrum of issues important to surgeons, including causation or risk factors for disease, natural history or prognosis of disease, how to quantify disease, diagnostic tests, early diagnosis and the effectiveness of treatment. A methodological article guides the reader in critical appraisal of the clinical article. Methodological and clinical reviews of the article are performed by experts in the relevant areas and posted on the EBRS website, where they are archived indefinitely. In addition, a listserv allows participants to discuss the monthly article. Surgeons who participate in the monthly packages can obtain Royal College of Physicians and Surgeons of Canada Maintenance of Certification credits and/or continuing medical education credits for the current article only by reading the monthly articles, participating in the listserv discussion, reading the methodological and clinical reviews and completing the monthly online evaluation and multiple choice questions.

We hope readers will find EBRS useful in improving their critical appraisal skills and in keeping abreast of new developments in general surgery. Four reviews are published in condensed versions in the *Canadian Journal of Surgery* and 4 are published in the *Journal of the American College of Surgeons*. For further information about EBRS, please refer to the CAGS or ACS websites. Questions and comments can be directed to the program administrator, Marg McKenzie, at mmckenzie{at}mtsinai.on.ca.

#### Reference

1.  Evidence-Based Medicine Working GroupEvidence-based medicine.JAMA 1992;268:2420–5.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1992.03490170092032&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=1404801&link_type=MED&atom=%2Fcjs%2F53%2F1%2F64.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1992JV69400030&link_type=ISI) 

## Selected article

 Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population N Engl J Med 2009 360 491 9.[CrossRef](http://canjsurg.ca/lookup/external-ref?access\_num=10.1056/NEJMsa0810119&link\_type=DOI)[PubMed](http://canjsurg.ca/lookup/external-ref?access\_num=19144931&link_type=MED&atom=%2Fcjs%2F53%2F1%2F64.atom)[Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000262812400008&link_type=ISI)

## Abstract

**Question:** Would the implementation of a surgical safety checklist designed to improve team communication and consistency of care reduce major complications and deaths associated with adult noncardiac surgery? **Design:** Prospective before–after study. **Setting:** Eight hospitals in 8 cities representing a variety of geographic and economic circumstances and diverse populations. **Patients:** Patients who underwent noncardiac surgery. **Methods:** The intervention involved implementation of a 2-step checklist program. Various process and outcome data were collected prospectively before (3733 patients) and after (3955 patients) implantation of the checklist. **Outcomes:** Surgical site infections, unplanned returns to the operating room, pneumonia, death and any complications. **Results:** The rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (*p* = 0.003). Complications occurred in 11.0% of inpatients at baseline and in 7.0% after introduction of the checklist (*p* < 0.001). **Conclusion:** Implementation of the checklist was associated with concomitant reductions in the rates of death and complications among patients who underwent noncardiac surgery at a diverse group of hospitals.

## Commentary

To reduce the rate of major complications and deaths related to noncardiac surgery, Haynes and colleagues1 assessed the effect of a surgical safety checklist in the operating room. They hypothesized that the use of the checklist would improve outcomes by improving team communication and consistency of care. Implementation of the checklist in study hospitals throughout the world was associated with significant reductions in the rate of death and complications. Despite some controversy with regards to the before–after study design, the intervention was easy to implement, unlikely to cause harm and inexpensive. Uptake of the checklist will likely be widespread.

What are the concerns with the design of this study? The investigators stated correctly that randomly assigning individual patients to the use of the checklist would have risked false-negative results because of contamination, and they thus used a before–after study design. They failed, however, to discuss other options such as a cluster randomized trial in which randomization is done at one level (organization or professional) and data are collected at a different level (patient). This design, although it requires a much larger sample size, would have alleviated the concerns about contamination as well as concerns about biases inherent in before–after studies.

A before–after study is a variant of a cohort study. In cohort studies, patients are not randomly assigned to treatment. In their simplest form, cohort studies track forward in time a group exposed to a factor and a nonexposed group and compare outcomes between the groups. The nonrandom assignment of the patients can lead to selection bias. Selection bias is the existence of differences in measured or unmeasured baseline characteristics in the treatment and control groups due to the method of group assignment. If the characteristic that differs affect the outcome, this bias can lead to confounding in which incorrect conclusions are made about the relation between the exposure of interest and the outcome. Although the measured baseline characteristics of the 2 groups in this study appeared similar, unmeasured characteristics could still lead to confounding because of the nonrandom design.

Typically, cohort studies examine both groups from the same point in time. Before–after studies examine the groups sequentially; first the control group and then the treatment group, which can lead to additional biases. The first possible bias is the possibility of a learning curve in the process of collecting the data; for example, what was considered a major wound infection in the first group may not be the same in the second group because of more experience in classifying wound infections. It is not certain that this bias, if present, would only result in a decreased effect size as the authors claimed. The second problem is the influence of temporal or secular (long-term) trends. Factors such as seasonal variations, changes in staffing (residents), differences in equipment or techniques could result in differences in outcomes due to the sequential enrolment of the groups. Finally, the Hawthorne effect — the very act of observing a system will affect that system — could produce better outcomes after introduction of the checklist, not because of the checklist but because the operating room team knows they are being evaluated. As the authors concede, the contribution of the Hawthorne effect is difficult to disentangle in this study.

Simple before–after studies are probably never sufficient to support widespread introduction of quality-improvement initiatives.2,3 This is because of concerns about the risk of expending tremendous resources without obtaining a true benefit and the possibility of introducing new problems. Despite these considerations, a randomized controlled trial will probably not be done because the use of a surgical checklist has been rapidly adopted as a standard of care in many hospitals. Surgeons may not believe that there is enough uncertainty to justify a randomized trial, and they might also believe that omission of a surgical checklist would lead to needless morbidity and mortality. The items included on the checklist would already be considered standard of care, or at least best practice, in most operating rooms, are easy to implement, are unlikely to cause harm and are not costly.

The results of this study were both clinically important and statistically significant. The overall rate of death during the postoperative hospitalization period (up to 30 days) was 1.5% before the checklist was introduced and declined to 0.8% afterward (*p* = 0.003). Overall complications occurred in 11.0% of inpatients at baseline and in 7.0% after introduction of the checklist (*p* < 0.001). The most common complication, which accounted for most of the differences in the complications seen before and after introduction of the safety checklist, was surgical site infection; unplanned return to the operating room and pneumonia were the next most common complications.

At low-income sites, there was a reduction in the death rate from 2.1% to 1.0% (*p* = 0.006), and the complication rate decreased from 11.7% to 6.8% (*p* < 0.001) after introduction of the checklist. At high-income sites, the death rate went from 0.9% to 0.6% (*p* = 0.18). There was, however, a moderate decrease in the complication rate from 10.3% to 7.1% (*p* < 0.001). The mechanism for improvement in the rate of surgical site infections is not clear in the high-income sites; appropriate antibiotic use in high-income hospitals only changed markedly (20%) in 1 hospital. Surprisingly, that hospital had the smallest change in surgical site infections, from 2.0% to 1.7%. Results were robust to additional analyses that included multivariate analyses of factors such as the presence of a data collector, case mix, cross-validation of site and income.

Although this study showed that there was a reduction in the rates of major complications and deaths in patients who underwent noncardiac surgery, the methods of the study were weak, leading to uncertainty about the validity of the conclusions. While a checklist may be considered standard of care and some form of it is being adopted in many operating rooms because it is easy to implement and unlikely to cause harm, adopters should be aware of the limitations of this study.

## Footnotes

*   * The CAGS/ACS Evidence Based Reviews in Surgery Group comprises Drs. N.N. Baxter, K.J. Brasel, C.J. Brown, P. Chaudhury, C.S. Cutter, C. Divino, E. Dixon, L. Dubois, G.W.N. Fitzgerald, H.J.A. Henteleff, A.W. Kirkpatrick, S. Latosinsky, A. MacLean, T.M. Mastracci, R.S. McLeod, A. Morris, L.A. Neumayer, L.R. Temple and Ms. M.E. McKenzie.

*   **Competing interests:** None declared.

## References

1.  Haynes AB, Weiser TG, Berry WR, et al.A surgical safety checklist to reduce morbidity and mortality in a global population.N Engl J Med 2009;360:491–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJMsa0810119&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=19144931&link_type=MED&atom=%2Fcjs%2F53%2F1%2F64.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000262812400008&link_type=ISI) 

2.  Auerbach AD, Landefeld CS, Shojania KG.The tension between needing to improve care and knowing how to do it.N Engl J Med 2007;357:608–13.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJMsb070738&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=17687138&link_type=MED&atom=%2Fcjs%2F53%2F1%2F64.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000248584900016&link_type=ISI) 

3.  Eccles M, Grimshaw J, Campbell M, et al.Research designs for studies evaluating the effectiveness of change and improvement strategies.Qual Saf Health Care 2003;12:47–52.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjEyLzEvNDciO3M6NDoiYXRvbSI7czoxNzoiL2Nqcy81My8xLzY0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)