Abstract
At its 2009 annual symposium, chaired by Dr. William (Bill) Pollett, the Canadian Association of University Surgeons brought together speakers with expertise in surgery and medical education to discuss the role of surgical simulation for improving surgical training and safety. Dr. Daniel Jones, of Harvard University and the 2009 Charles Tator Lecturer, highlighted how simulation has been used to teach advanced laparoscopic surgery. He also outlined how the American College of Surgeons is moving toward competency assessments as a requirement before surgeons are permitted to perform laparoscopic surgery on patients. Dr. Teodor Grantcharov, from the University of Toronto, highlighted the role of virtual reality simulators in laparoscopic surgery as well as box trainers. Dr. Peter Brindley from the University of Alberta, although a strong proponent of simulation, cautioned against an overzealous adoption without addressing its current limitations. He also emphasized simulation’s value in team training and crisis resource management training. Dr. Chris de Gara, also from the University of Alberta, questioned to what extent simulators should be used to determine competency. He raised concerns that if technical skills are learned in isolation, they may become “decontextualized,” and therefore simulation might become counterproductive. He outlined how oversimplification can have an “enchanting” effect, including a false sense of security. As a result, simulation must be used appropriately and along the entire education continuum. Furthermore, far more needs to be done to realize its role in surgical safety.
Annual Charles Tator Lecture: Safer surgery through simulation
Dr. Daniel B. Jones
Today, laparoscopy is considered standard procedure.1 Patients experience a shortened hospital stay and a more rapid return to normal activity. However, when laparoscopy first gained popularity, there were questions about its legitimacy, particularly in regard to laparoscopic colectomy and tumour implantation at trocar sites.2 It has since been shown that, with expertise, laparoscopic colorectal surgery is safe.3 Accordingly, the number of laparoscopic surgeries has substantially increased in the last 10 years. More surgeries are done using this method, more courses are available and the procedures themselves have advanced.
To be considered an expert in any area it might be necessary to log 10 000 hours.4 However, since the Institute of Medicine in the United States placed restrictions on working hours, residents are falling short of the required number of cases and gaining less overall clinical experience.5 Residents may argue that the answer is to let them “see one, do one, teach one.” In this way, they can be allowed to make mistakes while faculty are there to correct and teach them. However, in 1999, the Institute of Medicine listed about 100 000 deaths per year owing to medical error and asked medical professionals to think of alternative methods for operative training.6
Ten years ago, it was substantially more challenging to teach laparoscopic surgery. For example, surgeons who were experienced in open surgery might have applied some concepts of an open operation to laparoscopic surgery. However, in reality, they rarely had adequate experience with all the required skills. It was also not possible to let a resident perform part of the operation, as this prevented the supervising surgeon adequate access to the operative field. There were substantial challenges in terms of operating time. For example, residents ought not to learn how to do something as rudimentary as tying knots in the operating room: it simply costs too much.7 Practising on animals and cadavers is also costly, and therefore many institutions simply do not have an adequate budget.8 This could mean that residents may have completed only 1 animal operation before moving into the clinical arena. Yet, when residents participated in only 1 weekend course and then performed a gastric bypass, fatalities occurred.5 In short, this model of training was inadequate.
Since that time, the relatively simple idea of training residents outside of the operating room has arisen. To investigate that model, one study divided second- and third-year residents and trained half with simple tasks, and the other half continued their residency with no particular training.9 Residents were tested on gallbladder surgery before and after the training in front of blinded evaluators. It was little surprise that the evaluation showed that, on tasks that had been practised, the residents had improved. The study also evaluated medical students doing the same tasks and found that whereas some medical students might start off slower than residents, after 25 or 30 repetitions, the medical students were as good as or better than residents.9 This prompted discussion about introducing the skills laboratory to medical students, especially because they seemed receptive to embracing technology in their learning.
This same approach could be applied to computer-based learning to measure how well a surgeon or resident performs tasks and captures errors but in a virtual environment. For example, in one study of first- and second-year residents, half were trained with a video trainer and half with computer-aided virtual reality training.10 Students trained in the virtual reality system improved in both systems but were slightly better with the virtual reality system. If they learned on the video trainer, they improved in both systems, but there was more improvement with the video trainer. In short, experience counted, but knowledge was also transferable. Another study on simulation training showed the effect sleep deprivation had on performance.11 Whereas residents’ time to complete tasks was about the same, they made more errors when they were fatigued. Of note, this information came out about the same time residents’ hours were restricted.
As interest grew, the Accredited Education Institutes were formed by a joint effort of American and Canadian members of the American College of Surgeons. This involved establishing a basic skills laboratory and a comprehensive space that would include hands-on training, teleconferencing and administrative offices.12 One of the greatest benefits has been a validated certification test, the Fundamentals of Laparoscopic Surgery. Another exciting venture in the United States is the American College of Surgeons/Association of Program Directors in Surgery Surgical Skills Curriculum Modules.12 These encompass a series of models suited to teaching skills that include basic and advanced laparoscopy, bone fixation, suturing and endoscopy.
Learning operations such as a hernia repair is complex, but if residents can practise initial steps using a model, this can optimize their opportunities to learn additional complex steps in the clinical arena.13 For example, third- and fourth-year residents performing a total extraperitoneal hernia repair improved after using a simulator. However, minimizing medical errors is far more complex than simply learning the individual steps of an operation. Researchers have hypothesized that it typically takes a series of mistakes before you reach an adverse event. For example, it is not enough just to worry about the surgeon who ties the knot. It is also vital to ensure the correct suture and needle are selected, that the lighting is good and that everybody is engaged in identifying and minimizing error. It has been theorized that it takes an average of 4.5 errors in the system for a medical accident to result.14 Therefore, it is necessary to examine not just what the surgeon is doing, but what the whole team is doing.
In Boston, Mass., surgeons developed a mock operating room that allowed residents to operate on a realistic model (including organs that could be made to bleed).15 This artificial theatre allowed for the evaluation of both technical and nontechnical surgical skills. These skill laboratories also made nurses integral to the simulations (e.g., nurses would let the surgeon know something might be wrong or ensure that the sponge count was correct). This has expanded to include all types of simulator models, including lap-band training simulators, haptic feedback systems and simple models such as the NOTES (natural orifice translumenal endoscopic surgery) training system, making their way into many institutions.8 There is a role for tele-proctoring and giving feedback.16 Many hospitals, particularly those in Boston, require surgeons to ensure they are certified in the Fundamentals of Laparoscopic Surgery.17 This has meant that some surgeons no longer do laparoscopy, some have had to practise more regularly and some have had to remediate. Regardless, credentialing has advanced the field.
No matter what the technology, whether an ultrasound machine or a robot, we need to practise it, and our simulation environments offer a great venue in which to do so. The goal is not just to teach surgeons to tie a knot, but instead to decrease medical error and increase patient safety. Models, simulators and simulation can train students, residents and faculty, and they can do so without monopolizing busy operating rooms. In the near future, it is quite reasonable to expect all surgeons to demonstrate skills on simulators and exercise judgment within simulated and safe environments. Perfect practice will make perfect practice.
Comprehensive training and assessment in minimally invasive surgery
Dr. Teodor Grantcharov
Training outside the operating room is the optimal form for basic skills training in minimally invasive surgery. There have been many local, nonvalidated initiatives throughout the world yet few formal training programs incorporating simulation training and structured assessment into competency-based curricula. It’s been difficult to implement growing research knowledge into clinical practice.
There are a variety of reasons for this slow implementation. Most educators agree that training and assessment tools must be extensively validated before they are implemented into practice. Recent restrictions on residents’ hours together with increasing service requirements have made it difficult for residents to find protected time for training within their weekly limit. There are also a number of financial issues: Who will pay for the expensive training, which might not proffer benefits immediately? At the same time, we have observed a tremendous focus on medical errors and patient safety. All of these factors highlight the importance of the implementation of valid, efficient and safe curricula in surgical education.
When designing a successful curriculum, several principles need to be followed. First of all, the curriculum needs to address all aspects of surgical competency. Traditionally, technical skill has been the main focus, but this is just one of the important components of surgical practice. Other significant factors include knowledge, decision-making, communication and teamwork. The training tools used in any curriculum must be evidence-based. At the same time, it is important to incorporate continuous assessment and ensure that progression to the next steps in the training curriculum happens after proficiency criteria have been met.
In the past decade, there has been an evolution of evidence-based training tools. Simulated models, including virtual reality simulators, box trainers, live animals and full-body simulators, are well validated. These models can be used to teach different skills. For example, virtual reality simulators can assess performance and provide instant feedback to trainees, whereas box trainers allow trainees to get acquainted with various instruments and have a more realistic haptic feedback. A number of assessment tools have been designed and validated. These include virtual reality simulators, global rating scales, procedure-specific checklists and motion analysis. Objective assessment can be used to provide structured and constructive feedback to the trainee and thus serve as an important prerequisite for deliberate practice.
Virtual reality simulators are a reliable and valid tool to teach and assess technical skills in minimally invasive surgery.18 Skills acquired using virtual reality simulation are transferable to the operating room; there is evidence of this in laparoscopy and gastrointestinal endoscopy.19,20 There are a few validated curricula for using advanced and basic laparoscopic skills.21,22
Box trainers represent another group of evidence-based tools. They have been used for many years, and there is evidence that skills developed using box trainers can be transferred to the operating room.23,24 One of the major disadvantages of box trainers is that assessment of skills is difficult.25 It requires the presence of an experienced supervisor who can provide assessment and feedback to the trainee. An example of a validated curriculum using box trainers is the Fundamentals of Laparoscopic Surgery program.26
Continuous assessment is a significant component in a comprehensive training curriculum. Several tools have been designed and validated to provide reliable and objective skill assessment. Live and video-based observation with criteria can be done in a simulated environment or in the operating room using procedure-specific checklists and global rating scales. Motion analysis systems and virtual reality simulation can also provide reliable and objective assessment of technical skills. It is impossible to design a good training program without continuous assessment and feedback, which can improve performance and shorten the learning curve in the operating room.
The past few years have seen important achievements in educational research. Several tools have been designed and validated, and several curricula have been designed. The next step is to implement these in practice through a collaborative, multicentre initiative.
Future work should focus on designing curricula for selection of surgical residents and fellows. Previous studies have demonstrated that about 8% of trainees are not able to develop skills necessary to perform minimally invasive procedures, and it would be important to identify these individuals before they enter a residency program.27
Another project is looking at designing and implementing a competency-based training program versus the time-based training program that is the current model. The profession also needs to address the development of a certification and recertification program, especially with the new legislation that removed the mandatory age of retirement, to ensure that surgeons who practise have the necessary skills and knowledge to act as safe surgeons.
Simulation in surgery: not just more simulation, better simulation
Dr. Peter G. Brindley
Simulation in surgery should be widely promoted and prioritized. However, to fully realize its potential, we need ongoing resources, vigorous research and a system-wide commitment to continuing improvement. We need to mature the discipline of simulation such that it becomes a central part of the science of improving clinical performance. Without such efforts, simulation may be little more than a “faith-based initiative,” with both zealous supporters and reactionary opponents, but little evidence to sway the ambivalent majority.
Simulation has been appropriately identified by numerous professional societies as a key to patient safety and also as a social justice imperative.28 As such, it is not a luxury, and we should not wait. However, as well as “doing it right now” we must “do it right.” For example, simulation is not yet realistic enough, sufficiently proven or sufficiently resourced to justify a wholesale rejection of bedside learning. It is, therefore, best understood as a supplement to, not a replacement for, traditional training and maintenance of competence.
Simulation can shorten the learning curve, decrease knowledge decay, permit development of manual skills before any exposure to patients, improve performance under stress, finesse teamwork and even optimize communication, all without endangering patient safety.29–31 However, it is not enough to simply spend time on a simulator, just as it is not enough to simply spend time on a clinical rotation. Educational time, no matter what form it takes, must be based on sound principles of adult learning.
Good education means clear expectations of educators but also of learners. Simulation is therefore best understood as a technique, not just a technology.29 It is not just about purchasing expensive manikins and expecting results. A deliberate curriculum is required. This should be accompanied by a realistic, immersive and emotionally engaging experience. This should be followed by reflection, which typically takes the form of a structured debrief by experts in feedback and formative evaluation.30 Therefore, the question is not whether all teaching should take place in the simulation laboratory versus the bedside, but how to harness the best learning from both. In other words, however novel or exciting the educational strategy, it is a means to an end (i.e., patient safety and skill development), rather than an end in itself.
Simulation is far from perfect and far from panacea. Brief simulation exposures are unlikely to model the myriad behaviours required of a surgeon. These include “patient ownership,” following a patient throughout his or her illness and even soldiering on when fatigued. Occasional simulation exposures also fail to duplicate the multiple ways in which the same disease can present whether because of anatomy, physiology, genetics, age or culture.31 As such, it is not outdated to promote the apprentice model — the traditional gradual assumption of responsibility — or to demand volume-based expertise. The optimal balance of simulation and bedside experience will also only be achieved when we understand the impact of finite instructional time, limited funds, increasing clinical loads, the huge reduction in trainee work hours, decreased tolerance for trial and error, and even the attitudes of teachers compared with learners. In addition, to truly transform our approach to medical error, simulation needs to be ingrained throughout medical education.31 Regardless, the first step is to understand why we need to get this right.
Medical errors are the eighth leading cause of death, and as many as 100 000 people die annually from preventable medical errors in the United States alone.29,32,33 Surgeon Atul Gawande has pointed out that most errors are not due to inadequate knowledge (i.e., “ignorance”), but rather problems in transforming that knowledge into meaningful clinical actions, especially under real-world conditions (i.e., “implementation”).33 Other high-risk professions, notably aviation, faced similar challenges but reconfigured how they train and maintain competence. This has been associated with a log reduction in flight fatalities. In other words, practical strategies and modifiable curricula already exist.31–33
The birth of aviation safety coincided with the statement that the modern jet “was too much airplane for one man to fly.”33 In a similar vein, the complexity of patient care means that surgeons cannot be trained as if they work in isolation. As a result, team training is needed, and simulation offers a risk-free strategy. Error mitigation means teaching more than just individual aptitudes such as factual knowledge or procedural dexterity. Instead, simulation can be used to practise strategies that identify, avoid, capture and mitigate error, instead of assuming that they result from mere stupidity, laziness or arrogance.32 In short, simulation becomes an agent of “culture change” as we move from a culture of individual competence to one of collective safety.29,31,33
A modern curriculum should also help surgeons get beyond the discussion of whether we are teaching surgery as an “art” or a “science.” In fact, simulation helps convey that, like aviation, surgery probably better resembles engineering.29–32 Applying engineering principles explains the benefits of teaching strategies, such as checklists, standard operating procedures, redundancies and fail-safes, and why we need to perfect these through simulation.29 It also helps explain how we could better respond to error and how we could modernize curriculum development. For example, when dissecting a typical commercial airline crash, there might be a technical problem or a lapse in judgment, but this alone rarely causes a crash. The crew might also be tired, such that decision-making skills or vigilance erodes. The plane might be behind schedule, adding stress and reluctance to invest extra time for safety. Many crews have not flown together, so are unfamiliar with each other’s style. The sum total of these minor stresses is a team that is “maxed out,” with nothing left if adversity strikes.32–34 This is why a single problem would rarely cause a disaster, but when combined they can, as shown by the “Swiss cheese model” of error.24,32 Any simulation curriculum should mirror this complexity.
Whereas personal responsibility is a vital trait for a surgeon, comprehensive error reduction means taking a system approach. This explains why the “name, blame, shame” approach is frankly outdated. It also explains why our educational traditions are greatly in need of change. Unfortunately, however, medical curricula are often “accidental.”35 For example, we often rely on random clinical presentation. We also rely on teachers covering their favourite topics regardless of relevance and despite evidence that the didactic format rarely aids long-term behavioural change.36 Instead, curricula should be deliberately matched to safety. Just like in engineering, most clinical errors are predictable. In other words, we know which problems warrant our finite attention. Regardless, routine audits could establish major problem areas (i.e., common shortfalls or steps that require particular precision or the coordination of many people). These results should then be shared, rather than being the purview of a select few. A relevant curriculum can then be drafted (using all relevant experts and a modified Delphi approach) and α tested. Next, wide-scale dissemination occurs using the optimized material (i.e., β testing).29 The process then begins again. In this way, simulation educators are not merely passing facts along, but are running the patient safety laboratory for the modern hospital.2 Accordingly, educators become important agents of change and as highly valued as good researchers or clinicians.30–32 Again, it becomes less important whether a simulator is used, but more important what is being taught and how deliberately we do so.
Unfortunately, current curricula typically focus on factual knowledge. However, human performance, not knowledge, is the most common reason for errors.31–35 Therefore, our curricula should address human factors such as leadership, teamwork, situational awareness and communication. These skills, known collectively as “crisis resource management” (CRM),31 can be taught through structured clinical exposure, but are particularly suited to team simulation.29–31 The current trade-off is that simulation is safer, but clinical practice is more realistic. As David Gaba has pointed out, CRM has been widely implemented in almost all high-risk professions except for medicine or surgery.31 Perhaps this was because CRM did not readily lend itself to traditional didactic education. With modern simulators, we are running out of excuses.
Aviation offers a prediction for simulation in surgery. It was about 30 years from the first flight simulators until flight simulation became mandatory. Using the same time-line, surgical simulation may become mandatory for trainees and surgeons alike within this decade. If so, then programs have few options other than to lead, follow or begrudgingly comply. The question is not “can we afford to simulate” but rather “can we afford not to?”32
The goal is to have good simulation, not merely more simulation. The challenge is therefore bigger than one of resources. The simulation challenge is whether we have the humility to learn from others, the insight to expunge the worst of our entrenched traditions and the pride to retain what is best. As Arthur Schopenhauer stated, “All truth passes through three stages. First it is ridiculed. Second it is violently opposed. Third it is accepted as self-evident.”32 That it is time for change should be “self-evident.” The future of surgical simulation, and whether it is “ridiculed” or “violently opposed” will illustrate just how seriously we really take the safety of our patients.
Surgical simulation: A surrogate for too few trainers?
Dr. Christopher de Gara
In Canadian residency programs, residents are intensely supervised. As such, the common claim that trainees merely “see one, do one, teach one” is simply not accurate. However, this is not to say that there are no issues with one-on-one supervision. For example, surgeons often express that there is inadequate operating room time, that a resident slows progress or that there are too many people merely standing around and hoping to learn something. Surgeons also argue that there are not enough “ideal” teaching patients. In addition, a general surgery residency is still restricted to 60 months, whereas the curriculum has grown substantially and has become more complex. Add to this that resident work hours have been reduced, thereby lessening direct clinical exposure. All of these factors likely contribute to why surgical simulation is on the rise, namely as a potential solution to these problems.
The medical field often compares itself to the airline industry, which has had great success using simulation as a teaching tool. In 2008, there were 876 deaths from airline accidents. In 1972, there were 3214 deaths. Pilot error was the cause just over half the time (53%).37 Moreover, research has shown it was safer when the copilot was flying, because he or she had a direct supervisor to ensure that the flight was proceeding safely.38 However, differences between aviation and surgery mean that it is impossible to directly relate aviation data to the operating room, especially as accurate figures have not been collected for the deaths directly attributable to surgeon error.
As important as simulators are to a curriculum, success in the simulator laboratory alone should not justify advancement through training. For example, competitions that deem who can complete a task the fastest in the simulation laboratory may be sending the wrong message to trainees. On the other hand, should an otherwise competent resident be held back because he or she cannot suture in a box? In short, competency-based programs do not necessarily ensure competency. This is because we need to question whether the skills learned in a surgical simulator are directly transferable to the operating room. For example, the simulator is not yet up to the task of mimicking a common procedure like hernia surgery in the operating room, yet it is being used to teach it.39 In a similar vein, aviation literature shows that attempts to transfer manual flying skills from the simulation to the flight deck demonstrated no difference, even after the simulation group had practised for hours.40
In an article on simulation and transformational change, Roger Kneebone argued that learning surgical and procedural skills should begin with mastering the basics.41 In contrast, if we learn technical skills in isolation and in a way that is not part of clinical practice, then skills are “decontextualized.” As a result, simulation can actually be counterproductive because learners do not understand what to do next in a surgical setting.
Simplifying complex problems is attractive. However, this method tends to have an “enchanting” effect on the teacher and the learner: creating a false sense of security after doing so well in simulation. Furthermore, simulation has evolved in an ad hoc fashion, driven by simulator availability and by local enthusiasts. Despite great enthusiasm, we still do not know if practice can produce the experienced nonexpert. For example, someone who has become skilled at dealing with predictable and recurring patterns (as is typical with simulator-based cases) does not equate to an adaptive expert who is adept at dealing with new situations.41
Skills centres may be attractive for the very reason that they increase simplicity. However, the downside is that they do not mirror the inherent complexity and uncertainty of surgical practice. Threshold concepts are transformative, irreversible, impossible to unlearn and are a conceptual gateway that leads to the previously inaccessible understanding. For example, there is a period of discomfort when residents are on the cusp of learning a new task. This is followed by uncertainty, which finally progresses to a new state of understanding. Troublesome knowledge, on the other hand, is somewhat counterintuitive but has the capacity to integrate disparate elements into a new conceptual framework.42
The challenge for training programs is that it is imperative to arm residents with the proper surgical skills in a safe and competent fashion. Simulators can be complex and costly and are effective in many ways, specifically in team training and where the focus is on communication. Unfortunately, surgical simulators currently tend to oversimplify tasks. Surgical simulation needs to increase its sophistication and realism before it can live up to its promise of teaching all of the skills necessary to perform an operation safely and competently.
Conclusion
At its 2009 annual symposium, the Canadian Association of University Surgeons invited several surgeons with considerable expertise in surgical training and surgical simulation to present on and discuss the topic of surgical simulation, its role in training and surgical safety. Dr. Daniel Jones, chief of minimally invasive surgery at Harvard University and the 2009 Charles Tator Lecturer, started off the symposium by highlighting the challenges of advanced laparoscopic surgery, the complexity of learning novel techniques and the value of acquiring skills in a simulated environment. Dr. Jones underscored the direction that the American College of Surgeons is moving in with regard to Fundamentals of Laparoscopic Surgery, a certification required for surgeons to perform laparoscopic surgery on patients. Dr. Teodor Grantcharov from the University of Toronto highlighted the role of virtual reality simulators in laparoscopic surgery as well as box trainers, which allow development of formal curricula and, through validated testing, can be used to ensure development of skill sets. Dr. Peter Brindley from the Division of Critical Care Medicine, University of Alberta, cautioned against an overzealous adoption of simulation without recognition of its current limitations. He underscored its value in team-based learning, using the experience of the airline industry as illustrative of how crisis resource management techniques can achieve significant safety benefits. Dr. Chris de Gara, Division of General Surgery, University of Alberta, questioned whether simulators should be used as a tool for determining competency in training. He highlighted the topic of transformational change and the risk of technical skills learned in isolation being decontextualized, which may in fact be counterproductive. Further oversimplification of complex problems tends to have an “enchanting” effect on teacher and learner, creating a false sense of security after doing well in a simulation setting. The need to use simulation appropriately along the entire education continuum, and assessing its role in surgical safety will be an ongoing challenge.
At the lively panel discussion, the audience participated in further exploring the complex but exciting future that surgical simulation represents.
Footnotes
Competing interests: None declared.
Contributors: All authors were involved in the conception and design of this manuscript and each wrote and referenced their individual section. P.G. Brindley and C. de Gara revised the sections to create a cogent document. All authors approved the final version submitted for publication and are responsible for their individual sections.
- Accepted September 22, 2011.