Improving patient safety through the systematic evaluation of patient outcomes
==============================================================================

* Alan J. Forster
* Geoff Dervin
* Claude Martin, Jr.
* Steven Papp

## Abstract

Despite increased advocacy for patient safety and several large-scale programs designed to reduce preventable harm, most notably surgical checklists, recent data evaluating entire health systems suggests that we are no further ahead in improving patient safety and that hospital complications are no less frequent now than in the 1990s. We suggest that the failure to systematically measure patient safety is the reason for our limited progress. In addition to defining patient safety outcomes and describing their financial and clinical impact, we argue why the failure to implement patient safety measurement systems has compromised the ability to move the agenda forward. We also present an overview of how patient safety can be assessed and the strengths and weaknesses of each method and comment on some of the consequences created by the absence of a systematic measurement system.

In 1999, the Institute of Medicine in the United States released a landmark report entitled *To err is human: building a safer health system*.1 This report compiled statistics from the published medical literature on the safety of health care. It concluded that up to 100 000 Americans were dying annually as a result of medical errors, which exceeded the number of deaths due to motor vehicle collisions, breast cancer or HIV.1 This report spurred a call to action to the health care community, which translated into millions of dollars of investment dedicated to improving patient safety. Several organizations, such as the Institute for Healthcare Improvement in the United States and the Canadian Patient Safety Institute, were created to address these concerns. These and other organizations have been very active trying to advocate for patient safety and have promoted several large-scale programs designed to reduce preventable harm.2 While some of these efforts, most notably surgical checklists,3,4 have been associated with a positive impact in the clinical trial setting, recent data evaluating entire health systems suggest that, for the most part, we are no further ahead in improving patient safety and that hospital complications are no less frequent now than they were in the 1990s.5

In this article, we suggest that the failure of the health care system to systematically measure patient safety is the fundamental reason why it seems to have made little progress on this important priority. We start by defining patient safety outcomes and providing data describing their financial and clinical impact. Then, we argue why the failure to implement patient safety measurement systems has been a fundamental problem compromising any ability to move the agenda forward. In this discussion, we present an overview of how patient safety can be assessed and the strengths and weaknesses of each method. We conclude by commenting on some of the consequences created by the absence of a systematic measurement system.

## Is patient safety a real problem?

Adverse events (AEs) are undesirable outcomes attributable to medical care rather than to the underlying disease process.1 They occur because of inherent risks related to therapies, such as surgery or medications. They also occur due to errors in health system design or individual error. The subgroup of AEs due to errors is considered preventable.” The major AE types include hospital-acquired infections, adverse drug events, surgical complications, system errors, diagnostic errors, treatment errors, obstetrical injuries, procedure complications and anesthesia-related injuries.6–12 It is important to define AE type, as strategies to prevent them will vary.

While most patients receiving treatment are generally safe, Canadian studies demonstrate that 7%–12% of hospital patients experience an AE and that 30%–40% of these events are preventable.6,7 In surgical populations, AEs appear to be more common than in other populations.6–12 For example, in the Harvard Medical Practice Study,8 the risk of AEs was greater for vascular surgery (16%), cardiac surgery (11%), neurosurgery (10%) and general surgery (7%) than for general medicine (4%), obstetrics (2%) and neonatology (0.5%). This means that iatrogenic illness is one of the most common conditions treated in hospital and in surgery. In comparison, diabetes, congestive heart failure and chronic pulmonary disease affect 15%, 10% and 9%, respectively, of hospital inpatients.6,13 Research in many countries indicates that AEs are a global problem.6–12,14,15

Existing studies suggest that AEs have a major impact on costs and patient health. With respect to costs, there have been several studies of different AE types. Zhan and Miller16 evaluated administrative claims data using a nationwide cohort design to determine the associated costs of different types of AE. They found that most AE types were associated with substantially increased costs and length of stay in hospital. These increases were most dramatic for in-hospital sepsis, which accounted for an average of 10 additional days in hospital and US$ 57 000 in charges. The authors found that other complications, such as medical errors and procedural injuries, had a similar impact. Eber and colleagues17 performed a similar analysis focusing only on hospital-acquired infections and found comparable results.

Bates and colleagues18 and Classen and colleagues19 used clinical surveillance to identify patients who experienced adverse drug events. Using case–control methods, they found that, on average, an adverse drug event was associated with 2 additional days in hospital and US$ 2000–5000 in charges, depending on event preventability. Using these data and information on the incidence of adverse drug events, Bates and colleagues18 estimated the annual cost of adverse drug events for a typical 700-bed hospital to be US$ 5 million or 5% of its total expenditures.18

From a Canadian perspective, Baker and colleages,6 using a cohort design with medical record review, found that AEs had a differential impact on length of stay in hospital, depending on the hospital type. For teaching hospitals, large community hospitals and small hospitals, AEs were judged to cause 6, 4 and 8 additional days in hospital, respectively. There were no cost estimates on charges reported in this study, but based on average daily costs, this translates to up to $10 000 per event.

With respect to impact on patient health, while there are ample studies of the short-term effects of AEs, there are no studies assessing the long-term health outcomes. Prior studies usually grade the severity of the event based on an assessment of the in-hospital course. Specifically, most of the large hospital AE studies assessed medical records and rated whether an AE caused temporary disability, permanent disability or death. These studies have determined that about 5% of AEs will lead to temporary disability expected to last for at least 6 months, 5% will lead to permanent disability, and 15% will lead to death.6–12 Although the remaining 75% of AEs are less severe, they also cause temporary disability (< 6 months’ duration), extend the patients’ hospital stay and cause increased suffering.

While on an individual AE basis these costs and health impacts are large, collectively, if one considers the high prevalence of AEs, they are astounding. The risk of AE for a Canadian adult who is admitted to an acute care hospital for a medical or surgical diagnosis is 10%. Therefore, for a typical acute care hospital that will admit 20 000 patients annually, 2000 patients will experience an AE. Based on the studies mentioned previously, this will translate to up to $20 million in direct incremental charges, 1600 additional days in hospital and 300 deaths. Furthermore, by extrapolating the number of hospital admissions nationally, it has been estimated that up to 100 000 Americans and 24 000 Canadians die annually owing to AEs. These data suggest a strong imperative for action.

## Measuring harm — getting the fundamentals right

The inability of health systems to adequately detect AEs and monitor their prevalence has been identified as a major factor in their persistence.20–22 We agree with this position. In this section, we demonstrate why sound measurement is fundamental to the task of improvement.

It should be self-evident that measurement is an important starting point. First, without measuring the specific types of AEs or their frequency, it is impossible to specify patient safety priorities. Second, failing to correctly classify AEs impedes our understanding of their causes and reduces our ability to design effective solutions. Finally, a lack of measurement interferes with our ability to track progress or compare performance among peers.

There are also some studies supporting an association between systematic measurement and improved health outcomes. The best known example of such a system-wide approach is the National Surgical Quality Improvement Program (NSQIP) in the United States.23–25 There are several examples of institutions and health systems using the results of the NSQIP to modify care processes with resulting improvements in patient outcomes. Systematic monitoring of AEs is also the basis of infection control programs, which were demonstrated to be effective in the mid-1970s.26 Although all these studies were observational, they lend credence to our premise. Furthermore, it is unlikely that an experimental study could be conducted to test the hypothesis.

Finally, it is relevant to consider some clinical examples. Consider 3 different patients readmitted to hospital within 28 days of discharge after elective arthroplasty (Box 1). To prevent these distinct problems, different strategies, with varying likelihood of success, are required. Systematic measurement of adverse events and classification of their causes will therefore help us select the appropriate intervention and estimate its ability to improve outcomes. For example, if one were to evaluate the causes of readmissions and found patient 3 (Box 1) to be most representative, then improving anticoagulant management might be helpful; however, if patients 1 and 2 were most representative, it is unlikely that any intervention would work. Only systematic measurement of undesireable outcomes and investigation of their causes can inform clinicians of the appropriate corrective response.

Box 1 
### Examples of undesirable outcomes following elective hip arthroplasty

Consider 3 patients who required a readmission within 30 days of elective hip arthroplasty:

**Patient 1 readmitted because of a massive pulmonary embolism despite adequate prophylactic anticoagulation.** This case represents an adverse event, as the outcome (pulmonary embolism) was caused by health care management (hip arthroplasty). The event was not preventable, as it was not due to a system error — the patient received appropriate prophylaxis.

**Patient 2 readmitted because of an upper gastrointestinal bleed resulting from a duodenal ulcer.** This case does not represent an adverse event, as the outcome (upper gastrointestinal bleed) was caused by underlying disease (the duodenal ulcer) not health care management. Note that this patient was not on anticoagulants when the bleed occurred.

**Patient 3 readmitted because of a subdural bleed resulting from an INR of 6.0 (the patient was prescribed extended prophylaxis with warfarin).** This case represents an adverse event, as the outcome (subdural bleed) was caused by health care management (warfarin). The event was preventable, as it was due to a system error — the patient’s anticoagulation was not well controlled at the time of the bleed.

INR = international normalized ratio.

## How are AEs detected?

To detect AEs, health systems generally use 2 broad categories of AE detection strategies: voluntary reporting and proactive surveillance (Table 1).20

View this table:
[Table 1](http://canjsurg.ca/content/55/6/418/T1)

Table 1 
Summary of adverse event detection methods

### Voluntary reporting

Voluntary reporting, also called incident reporting, is the method currently used in most hospitals. It involves health providers voluntarily reporting “incidents” using standardized forms that are either paper-based or electronic. Despite voluntary reporting being the most common method, it is largely ineffective. Voluntary incident reporting systems identify fewer than 10% of AEs,27 as providers rarely report meaningful complications and often identify insignificant incidents.27–30

### Proactive surveillance

Proactive surveillance involves actively monitoring for AEs, and there are several different methods. In general, patients are monitored to identify events suggesting poor outcomes. Depending on the surveillance method, these outcomes undergo peer review to determine their most likely cause. If the outcome was deemed to be caused by medical care, then it is considered an AE; if it was deemed to be due to an error, then it is considered a preventable AE. The methods of surveillance include medical record review, administrative surveillance, clinical surveillance and eTrigger surveillance.

#### Medical record review

Medical record review has been used in epidemiologic studies of AEs.6–12,14 It involves a nurse performing an initial chart review to flag patients with poor outcomes and a physician reviewing the flagged charts. Medical record review has greater validity than voluntary reporting. However, the method is retrospective and can be limited by poor documentation in clinical records.31,32

#### Administrative surveillance

Administrative surveillance is commonly used for the purposes of public reporting.33–35 In this method, hospital billing data are screened for encounters containing International Classification of Diseases (ICD) codes indicating hospital complications. Peer review is usually not involved in this method. Administrative surveillance is currently used for interfacility comparisons by the Agency for Healthcare Research and Quality and the Canadian Institute for Health Information and for international comparisons by the Organisation for Economic Co-operation and Development. Administrative surveillance is, by far, the least expensive method to implement, as it simply tracks codes collected for other purposes. This method is limited: there are very high proportions of false-positive and false-negative cases35,36 resulting from coding inaccuracy in the billing data, and the method does not track all AE types, as codes are not currently adapted to capture diagnostic, system or management errors, which collectively account for about 30% of AEs.

#### Clinical observation

Clinical observation involves a trained observer directly monitoring patients and providers28,37–39 rather than relying on documentation (as they would in the 2-stage chart review). When a patient experiences a poor outcome, the observer collects prespecified information, which is then reviewed by a physician or physician panel. Clinical observation is the most accurate method to measure AEs. It has the added benefits of describing the AEs in great detail because the events are captured prospectively in real-time.26,28,37,38,40,41 Conversely, the need for an observer who understands clinical processes adds a cost not incurred in other methods. Clinical surveillance is the only method for which there are data correlating the systematic measurement of AEs with subsequent improvement in clinical outcomes.23–26,42,43

#### eTrigger surveillance

eTrigger surveillance involves detecting clinical events by scanning electronic data systems for prespecified data. When data indicate a potential problem, the relevant information is electronically sent to individuals to investigate cause and impact. eTrigger surveillance can lead directly to AE prevention strategies: often eTriggers identify outcomes about to happen rather than events that have already happened. As a result, eTriggers can be transformed into electronic messages to clinicians in the form of real-time clinical decision support. eTriggers generally have high sensitivity, but highly variable specificity, which means they identify most AEs but often have a high false-positive rate. The poor specificity results from immature information systems in most hospitals and can be improved by incorporating a peer review process.44–49 eTrigger detection is also limited by the need for robust clinical information systems, which most hospitals have not had to date.

#### Summary

Voluntary incident reporting, medical record review and administrative surveillance are the most commonly used AE detection methods, yet they have important flaws that greatly limit their ability to accurately detect AEs. Clinical observation is currently the most accurate method of AE detection but is not widely adopted, likely as a result of cost and a failure of organizations to recognize the deficiencies of existing methods. Finally, eTrigger surveillance is very promising, as it has the potential to greatly reduce the cost of accurate AE detection and because of its ability to be directly translated into actions to mitigate or prevent harm. Furthermore, with the large investments currently being made into health care information technology systems, the accuracy and generalizability of this approach will likely improve.

## What is impeding the adoption of a systematic approach to AE detection?

Systematic AE detection likely does not occur for at least 3 reasons. The first 2 reasons are primarily technical and can be overcome with improved technological solutions. The third reason is psychological and must be overcome with culture change. In this section, we describe these reasons and highlight how they might be overcome.

In our opinion, the most important reason obstructing the adoption of AE detection systems is the “invisible” nature of AEs. That is, because undesirable outcomes are most likely due to the natural course of illness, it is normal for patients and doctors to attribute them to illness rather than treatment. It is only through careful peer review that the cause of an outcome can be determined. For example, patients with diabetes often experience renal failure. Therefore, it is natural for patients and clinicians to consider diabetes, rather than an angiotensin converting enzyme inhibitor that was recently prescribed, to be the cause of renal failure. To untangle the contributing causes, an expert needs to review the case.

Even with peer review, there is often disagreement between professionals regarding true causative factors.50 Some physicians will state the cause to be an error, whereas others will disagree. This reliability issue is definitely a concern, but it can be overcome by increasing the number of reviewers and requiring a certain proportion to agree.51

An alternative approach is to ignore the causes of undesirable outcomes and simply compare outcome rates across providers or time. This approach works reasonably well for surgical complications, as it is relatively easy to attribute the outcomes to the surgery, especially when accurate risk adjustment models are used. Risk adjustment techniques incorporate baseline clinical risk factors to ensure that different outcome rates are not attributed to “sicker” patients. The success of the NSQIP program is largely a function of the ability to systematically identify the outcomes and compare them across institutions using accurate risk adjustment. However, the approach works less well for other types of AEs because there are often competing explanations for the outcome. For example, atrial fibrillation on day 2 after a nonemergent thoracotomy is by convention designated a postoperative complication, as it is unlikely that the patient would have experienced the event in the absence of the surgery. On the other hand, many potential factors could contribute to the same atrial fibrillation in a 75-year-old man with longstanding type 2 diabetes and known coronary artery disease who is admitted to hospital for septicemia. For this reason, we advocate the use of peer review to determine the causes of all undesirable outcomes. As the examples in Box 1 illustrate, even in the case of surgery, the outcomes can be explained by other factors that may not be accounted for in the risk adjustment. The technological solution to address this requirement is the facility to ensure that peer review can occur in a timely manner with minimal disruption to the physician’s daily routine. Specifically, the physician should not have to review the medical records; rather, the review should be completed online.

A second reason preventing AE detection is the distributed nature of AEs. Often, an AE manifests clinically days or months after the clinical encounter. For example, a missed diagnosis might not become apparent for months or even years, or a surgical site infection may not become apparent until long after discharge. For this reason, the clinician involved in a case may not observe the outcome. To overcome this barrier, detection and feedback systems must necessarily have a “systemic” view. That is, the AE detection system must not be limited to a particular silo within the health care system, such as a hospital or an out-patient facility. Rather, the detection system must follow the patient and identify the outcomes irrespective of where the patient is and when the outcome occurs. This requirement imposes a major challenge on designing the AE detection system, particularly in light of privacy concerns. Privacy considerations are relevant to both the patient and provider. From the patient perspective, there are legitimate concerns regarding the disclosure of their personal health information for purposes other than their direct care. From the provider perspective, there are concerns that they will be unfairly identified as having high complication rates. There are laws limiting the disclosure of personal health information, which can be seen as both a help and hindrance, depending on the perspective. While a full discussion of these issues lies outside the scope of this review, the laws were established to support both direct care provision and quality assurance. We are confident that policies and practices can be designed to be compliant with the intent of the law. Once these have been articulated, then it is reasonably straightforward to design the underpinning information systems to support the work.

The third and probably most important reason limiting AE detection is the way AEs make clinicians feel. Any time a patient experiences a bad outcome, a physician will feel it to be their personal failing. If the treating physician perceives that he or she made an error, then these feelings are greatly inflated (even if the perception of error is invalid).52 These feelings make it difficult to examine the outcome objectively and critically. Combined with the adversarial nature of our tort system for managing malpractice, it is not surprising that many physicians feel threatened when they are mandated to participate in systematic methods of AE detection. Overcoming this challenge is difficult, as it will require cultural change. There are obviously many other requirements to modify culture. Clearly, effective leadership is a basic need to ensure trust within and outside of the clinical group. In addition, there needs to be a supportive environment with adequate resources to accomplish tasks. However, even with these requirements, we argue that little progress will be made without specific attention to patient safety issues.

We suggest there are 2 important things physician leaders can do to induce meaningful progress relevant to patient safety culture. The first is to provide, support or mandate patient safety education. This will help physicians understand the importance of AEs in terms of both their frequency and impact. Also, it will highlight that prevailing theories on the causes of AEs suggest that systems, not individuals, are to blame. Finally, patient safety education will provide information on how to reduce the incidence of AEs. All 3 of these lessons will be liberating and empowering for physicians, as it will help them to understand that they are not solely to blame for particular AEs. More importantly, it will show them that without physician leadership, AEs are likely to persist.

There are many ways in which physicians can obtain patient safety education. There are some great online resources and text books to support learning in this area. There are also some courses and annual meetings. Finally, we recommend programs that offer some experiential component in which learners complete a practical project. This will ensure the learning is not completed in abstract. A second suggestion is to establish a regular forum to discuss AEs in a multidisciplinary format. Many groups already have morbidity and mortality rounds, which typically include attending staff and residents. These meetings provide a mechanism in which to learn from mistakes. Some have observed weaknesses in how the meetings are conducted —specifically, they often do not identify cases systematically, they are not multidisciplinary, and they do not incorporate “systems” theory in their discussions.53 However, the positive effects of education likely exceed these negative observations, especially if the rounds are thoughtfully designed and implemented.54 We suggest that for this activity to have maximal impact it would require some minor modifications. First, inviting at least 1 nurse and 1 pharmacist from the unit to the meetings will completely change the focus of discussion and will encourage a more collaborative approach to understanding the causes and solutions to problems. Second, developing a transparent and objective method for selecting the cases to discuss will ensure a tracking of the important problems. We recommend selecting some outcomes that are common and relatively minor because, as the stakes are a little lower, it makes it easier to discuss the underlying causes of the problem. Third, have a strict format for describing the cases and classifying their causes, including a focus on the system causes. Fourth, keep track of the results of the discussion and revisit the issues from time to time to see if anything is being done to prevent the problems from recurring. In addition to these suggestions, the tone used for leading discussions is very important. It is mandatory to keep the language nonaccusatory, respectful and compassionate. It is important to remember that some people in the room will feel very uncomfortable with some of the discussion points. It is very easy for people to feel they are being blamed for something, even when they are not. In our experience, this tends to happen when the physicians blame themselves. The leader of the discussion should be attuned to people’s reactions and should address overt anxiety or anger in an appropriate manner, usually 1-on-1 after the meeting.

As stated, there are many other actions that could be taken to support culture change. We have observed that once educated on patient safety and more comfortable discussing complications in a team environment, physicians will often lead these initiatives, and the change becomes self-sustaining.

## Conclusion

We have argued that patient safety is lacking and that improving it requires a disciplined approach to measurement. We have also reviewed the strengths and weakness of various methods for detecting AEs. Finally, we have reviewed some of the issues impeding the ability to detect AEs and provided some suggestions on how they can be overcome.

We would like to conclude by briefly describing 2 other salient issues that are preventing progress in patient safety, which we argue could be resolved through improved measurement. There is a perception that there has been a lack of physician leadership, engagement and accountability in organized patient safety activities.55 This perception has led some to conclude that physicians are part of the problem. Whether or not this is true is beyond the scope of this review; however, we agree that without physician leadership and engagement, it will be impossible for the health system to improve. We also suggest that if there is a lack of engagement, it is at least partially explained by a paucity of robust clinical data at the local level to motivate change.

Second, there has been a tendency for health systems to rush toward inadequately justified, “one size fits all” solutions.2 Such top–down approaches will rarely work, wasting resources and diminishing the credibility of health system leaders. On the other hand, in the absence of robust clinical data, what alternative do the system managers have? A response of some kind is required given the problem’s extent and the overall importance of the health system. If the managers had better data on which to base decisions, then it is likely these types of largely ineffective solutions would disappear.

We offer these perspectives with humility and without wishing to belittle previous and ongoing efforts. Improving patient safety and health system quality in general is very complicated. It is easy to stand at the sidelines and criticize efforts, especially when one is uninformed of all important perspectives. At the same time, we cannot ignore data suggesting the lack of meaningful progress. Thus, we feel compelled to make these observations and suggestions.

## Footnotes

*   **Competing interests:** A.J. Forster and S. Papp designed the review. C. Martin Jr. acquired the data, which G. Dervin analyzed. A.J. Forster wrote the article. G. Dervin, C. Martin Jr. and S. Papp reviewed it. All authors approved publication.

*   **Contributors:** None declared.

*   Accepted September 6, 2011.

## References

1.  The Institute of Medicine (2000) To err is human: building a safer health system (National Academy Press, Washington).
    
    

2.  Berwick DM, Calkins DR, McCannon CJ, et al.The 100,000 Lives Campaign: setting a goal and a deadline for improving health care quality.JAMA 2006;295:324–7.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.295.3.324&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=16418469&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000234684500027&link_type=ISI) 

3.  de Vries EN, Ramrattan MA, Smorenburg SM, et al.The incidence and nature of in-hospital adverse events: a systematic review.Qual Saf Health Care 2008;17:216–23.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjE3LzMvMjE2IjtzOjQ6ImF0b20iO3M6MTg6Ii9janMvNTUvNi80MTguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  Haynes AB, Weiser TG, Berry WR, et al.A surgical safety checklist to reduce morbidity and mortality in a global population.N Engl J Med 2009;360:491–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJMsa0810119&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=19144931&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000262812400008&link_type=ISI) 

5.  Landrigan CP, Parry GJ, Bones CB, et al.Temporal trends in rates of patient harm resulting from medical care.N Engl J Med 2010;363:2124–34.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJMsa1004404&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=21105794&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000284587800007&link_type=ISI) 

6.  Baker GR, Norton PG, Flintoft V, et al.The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada.CMAJ 2004;170:1678–86.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTcwLzExLzE2NzgiO3M6NDoiYXRvbSI7czoxODoiL2Nqcy81NS82LzQxOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  Forster AJ, Asmis TR, Clark HD, et al.Ottawa Hospital Patient Safety Study: incidence and timing of adverse events in patients admitted to a Canadian teaching hospital.CMAJ 2004;170:1235–40.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTcwLzgvMTIzNSI7czo0OiJhdG9tIjtzOjE4OiIvY2pzLzU1LzYvNDE4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

8.  Brennan TA, Leape LL, Laird NM, et al.Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I.N Engl J Med 1991;324:370–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJM199102073240604&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=1987460&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1991EW29600004&link_type=ISI) 

9.  Vincent C, Neale G, Woloshynowych M.Adverse events in British hospitals: preliminary retrospective record review.BMJ 2001;322:517–9.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjIvNzI4NS81MTciO3M6NDoiYXRvbSI7czoxODoiL2Nqcy81NS82LzQxOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

10. Wilson RM, Runciman WB, Gibberd RW, et al.The Quality in Australian Health Care Study.Med J Aust 1995;163:458–71.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.5694/j.1326-5377.1995.tb124691.x&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7476634&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995TM31000004&link_type=ISI) 

11. Thomas EJ, Studdert DM, Burstin HR, et al.Incidence and types of adverse events and negligent care in Utah and Colorado.Med Care 2000;38:261–71.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-200003000-00003&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10718351&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000085644100003&link_type=ISI) 

12. Davis P, Lay-Yee R, Briant R, et al.Adverse events in New Zealand public hospitals I: occurrence and impact.N Z Med J 2002;115:U271
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12552260&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

13. van Walraven C, Austin PC, Jennings A, et al.A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.Med Care 2009;47:626–33.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/MLR.0b013e31819432e5&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=19433995&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

14. Runciman WB, Webb RK, Helps SC, et al.A comparison of iatrogenic injury studies in Australia and the USA. II: reviewer behaviour and quality of care.Int J Qual Health Care 2000;12:379–88.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1093/intqhc/12.5.379&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=11079217&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000165211900006&link_type=ISI) 

15. Bates DW, Larizgoitia I, Prasopa-Plaizier N, et al.on behalf of the Research Priority Setting Working Group of the WHO World Alliance for Patient SafetyGlobal priorities for patient safety research.BMJ 2009;338:b1775
    
    [FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzgvbWF5MTRfMS9iMTc3NSI7czo0OiJhdG9tIjtzOjE4OiIvY2pzLzU1LzYvNDE4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

16. Zhan C, Miller MR.Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization.JAMA 2003;290:1868–74.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.290.14.1868&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=14532315&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000185752600027&link_type=ISI) 

17. Eber MR, Laxminarayan R, Perencevich EN, et al.Clinical and economic outcomes attributable to health care-associated sepsis and pneumonia.Arch Intern Med 2010;170:347–53.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/archinternmed.2009.509&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=20177037&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000274793900006&link_type=ISI) 

18. Bates DW, Spell N, Cullen DJ, et al.The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group.JAMA 1997;277:307–11.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1997.03540280045032&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9002493&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997WC45800030&link_type=ISI) 

19. Classen DCM, Pestotnik SLM, Evans RSP, et al.Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality.JAMA 1997;277:301–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1997.03540280039031&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9002492&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997WC45800029&link_type=ISI) 

20. Thomas EJ, Petersen LA.Measuring errors and adverse events in health care.J Gen Intern Med 2003;18:61–7.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1046/j.1525-1497.2003.20147.x&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12534766&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000180352600010&link_type=ISI) 

21. Brennan TA.The Institute of Medicine report on medical errors. Could it do harm?.N Engl J Med 2000;342:1123–5.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJM200004133421510&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10760315&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000086406700010&link_type=ISI) 

22. Pronovost PJ, Miller MR, Wachter RM.Tracking progress in patient safety: an elusive target.JAMA 2006;296:696–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.296.6.696&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=16896113&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000239603800032&link_type=ISI) 

23. Neumayer L, Mastin M, Vanderhoof L, et al.Using the Veterans Administration National Surgical Quality Improvement Program to improve patient outcomes.J Surg Res 2000;88:58–61.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1006/jsre.1999.5791&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10644468&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000085226100011&link_type=ISI) 

24. Khuri SF, Daley J, Henderson WG.The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs.Arch Surg 2002;137:20–7.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/archsurg.137.1.20&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=11772210&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000173300200003&link_type=ISI) 

25. Rowell KS, Turrentine FE, Hutter MM, et al.Use of national surgical quality improvement program data as a catalyst for quality improvement.J Am Coll Surg 2007;204:1293–300.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2007.03.024&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=17544087&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000247152700021&link_type=ISI) 

26. Haley RW, Culver DH, White JW, et al.The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals.Am J Epidemiol 1985;121:182–205.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1093/oxfordjournals.aje.a113990&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=4014115&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1985TZ80700003&link_type=ISI) 

27. Cullen DJ, Bates DW, Small SD, et al.The incident reporting system does not detect adverse drug events: a problem for quality improvement.Jt Comm J Qual Improv 1995;21:541–8.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8556111&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

28. Donchin Y, Gopher D, Olin M, et al.A look into the nature and causes of human errors in the intensive care unit. 1995.Qual Saf Health Care 2003;12:143–7discussion 147–8
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjEyLzIvMTQzIjtzOjQ6ImF0b20iO3M6MTg6Ii9janMvNTUvNi80MTguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. Beckmann U, Bohringer C, Carless R, et al.Evaluation of two methods for quality improvement in intensive care: facilitated incident monitoring and retrospective medical chart review.Crit Care Med 2003;31:1006–11.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.CCM.0000060016.21525.3C&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12682464&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000182411900002&link_type=ISI) 

30. Chaudhry SI, Olofinboba KA, Krumholz HM.Detection of errors by attending physicians on a general medicine service.J Gen Intern Med 2003;18:595–600.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1046/j.1525-1497.2003.20919.x&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12911640&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000184722500001&link_type=ISI) 

31. Brennan TA, Localio RJ, Laird NL.Reliability and validity of judgments concerning adverse events suffered by hospitalized patients.Med Care 1989;27:1148–58.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-198912000-00006&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=2593729&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1989CF34000006&link_type=ISI) 

32. Madsen JJ.Comparison of concurrent and retrospective methods of detecting adverse drug reactions.Am J Hosp Pharm 1993;50:2556–7.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8122693&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

33. Zhan C, Miller MR.Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization.JAMA 2003;290:1868–74.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.290.14.1868&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=14532315&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000185752600027&link_type=ISI) 

34. Bates DW, O’Neil AC, Petersen LA, et al.Evaluation of screening criteria for adverse events in medical patients.Med Care 1995;33:452–62.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-199505000-00002&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7739271&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995QW54400002&link_type=ISI) 

35. Weingart SN, Iezzoni LI, Davis RB, et al.Use of administrative data to find substandard care: validation of the complications screening program.Med Care 2000;38:796–806.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-200008000-00004&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10929992&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000088393900004&link_type=ISI) 

36. McCarthy EP, Iezzoni LI, Davis RB, et al.Does clinical evidence support ICD-9-CM diagnosis coding of complications?.Med Care 2000;38:868–76.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-200008000-00010&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10929998&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000088393900010&link_type=ISI) 

37. Forster AJ, Halil RB, Tierney MG.Pharmacist surveillance of adverse drug events.Am J Health Syst Pharm 2004;61:1466–72.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpocCI7czo1OiJyZXNpZCI7czoxMDoiNjEvMTQvMTQ2NiI7czo0OiJhdG9tIjtzOjE4OiIvY2pzLzU1LzYvNDE4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

38. Andrews LB, Stocking C, Krizek T, et al.An alternative strategy for studying adverse events in medical care.Lancet 1997;349:309–13.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0140-6736(96)08268-2&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9024373&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997WF79400009&link_type=ISI) 

39. Bates DW, Cullen DJ, Laird N, et al.Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group.JAMA 1995;274:29–34.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1995.03530010043033&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7791255&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995RG23300027&link_type=ISI) 

40. Bates DW, Leape LL, Petrycki S.Incidence and preventability of adverse drug events in hospitalized adults.J Gen Intern Med 1993;8:289–94.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1007/BF02600138&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8320571&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1993LG74300001&link_type=ISI) 

41. Forster AJ, Fung I, Caughey S, et al.Adverse events detected by clinical surveillance on an obstetric service.Obstet Gynecol 2006;108:1073–83.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/01.AOG.0000242565.28432.7c&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=17077227&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000246769500005&link_type=ISI) 

42. Wolff AM, Bourke J, Campbell IA, et al.Detecting and reducing hospital adverse events: outcomes of the Wimmera clinical risk management program.Med J Aust 2001;174:621–5.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=11480681&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000169421300004&link_type=ISI) 

43. Wolff AM, Bourke J.Detecting and reducing adverse events in an Australian rural base hospital emergency department using medical record screening and review.Emerg Med J 2002;19:35–40.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo3OiIxOS8xLzM1IjtzOjQ6ImF0b20iO3M6MTg6Ii9janMvNTUvNi80MTguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

44. Murff HJ, Patel VL, Hripcsak G, et al.Detecting adverse events for patient safety research: a review of current methodologies.J Biomed Inform 2003;36:131–43.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.jbi.2003.08.003&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=14552854&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000186072200013&link_type=ISI) 

45. Forster AJ, Andrade J, van Walraven C.Validation of a discharge summary term search method to detect adverse events.J Am Med Inform Assoc 2005;12:200–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1197/jamia.M1653&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=15561788&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

46. Murff HJ, Forster AJ, Peterson JF, et al.Electronically screening discharge summaries for adverse medical events.J Am Med Inform Assoc 2003;10:339–50.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1197/jamia.M1201&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12668691&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

47. Honigman B, Light P, Pulling RM, et al.A computerized method for identifying incidents associated with adverse drug events in out-patients.Int J Med Inform 2001;61:21–32.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S1386-5056(00)00131-3&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=11248601&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000169550100003&link_type=ISI) 

48. Hope C, Overhage JM, Seger A, et al.A tiered approach is more cost effective than traditional pharmacist-based review for classifying computer-detected signals as adverse drug events.J Biomed Inform 2003;36:92–8.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S1532-0464(03)00059-5&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=14552850&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000186072200009&link_type=ISI) 

49. Jha AK, Kuperman GJ, Teich JM, et al.Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report.J Am Med Inform Assoc 1998;5:305–14.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1136/jamia.1998.0050305&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9609500&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

50. Thomas EJ, Lipsitz SR, Studdert DM, et al.The reliability of medical record review for estimating adverse event rates.Ann Intern Med 2002;136:812–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.7326/0003-4819-136-11-200206040-00009&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=12044129&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000175964500004&link_type=ISI) 

51. Forster AJ, O’Rourke K, Shojania KG, et al.Combining ratings from multiple physician reviewers helped to overcome the uncertainty associated with adverse event classification.J Clin Epidemiol 2007;60:892–901.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2006.11.019&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=17689805&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000248931700005&link_type=ISI) 

52. Wu AW.Medical error: the second victim.BMJ 2000;320:726–7.
    
    [FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjAvNzIzNy83MjYiO3M6NDoiYXRvbSI7czoxODoiL2Nqcy81NS82LzQxOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

53. Pierluissi E, Fischer MA, Campbell AR, et al.Discussion of medical errors in morbidity and mortality conferences.JAMA 2003;290:2838–42.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.290.21.2838&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=14657068&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000186872800026&link_type=ISI) 

54. McDonnell C, Laxer RM, Roy WL.Redesigning a morbidity and mortality program in a university-affiliated pediatric anesthesia department.Jt Comm J Qual Patient Saf 2010;36:117–25.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=20235413&link_type=MED&atom=%2Fcjs%2F55%2F6%2F418.atom) 

55. Lewis S.CMA emerges dazed from cave, writes report.Essays 2010Available: [www.longwoods.com/content/21904](http://www.longwoods.com/content/21904)accessed 2012 Sept. 21