Abstract
Background: This study’s aim was to evaluate the performance of senior orthopedic residents during simulated arthroscopic anterior stabilization (Bankart repair) before and after a national shoulder review course.
Methods: Participants were assessed before and after the Annual Shoulder Review Course over a 3-day period, using a multiple-choice examination and surgery performance assessment. The surgical evaluation was completed by fellowship-trained surgeons using a standardized procedure checklist and a global rating scale. All Canadian senior orthopedic residents were invited to participate in the course.
Results: The 57 participants showed improvement following the course. The written knowledge evaluation mean score increased, and all 3 surgical performance measurements improved: surgical task time improved from 4:40 min to 2:53 min (p < 0.001), surgical technique evaluation increased from 56% to 67% after the procedure checklist (p < 0.001), and anchor placement improved for all 3 aspects. Anchor entry point was the sole measure not to improve enough to reach statistical significance (p = 0.37).
Conclusion: Our data support the inclusion of dry model surgical simulation as part of a surgical skills course for both training and assessment of orthopedic surgery residents.
Learning surgical skills can be understood with Fitts and Posner’s 3-phase framework for acquiring motor skills.1,2 The initial cognitive phase is characterized by learners understanding the skills and watching demonstrations. In the associative phase, learners perform the tasks and associate the psychomotor steps with the knowledge acquired during the cognitive phase. The final autonomous phase focuses on repetition and the automatism of psychomotor movements.1,2
Effective and efficient surgical skills training strategies are needed to counter the impact of trainee work hour restrictions, increased pressures to enhance operating room efficiency and concerns regarding patient safety.3,4 Simulation training with feedback and assessment of surgical skills is increasingly recognized as an education strategy that can assist in the integration of the phases of motor skill learning.5–7 Simulation training has been implemented and evaluated in a number of disciplines (e.g., aviation, military, medicine, surgery, business).8,9 The acquisition of minimally invasive techniques such as shoulder arthroscopic techniques, which are known to be challenging to learn and associated with a steep learning curve, requires many hours of practice.9 Previous studies have shown that shoulder simulation training improves both surgical performance in cadaveric and simulated models.8 Other studies have shown that surgical skills learned on physical or virtual-reality simulators are transferrable to the operating room.10
Arthroscopy simulation is becoming more common in universities and surgical societies.11,12 One example is the Fundamentals of Arthroscopic Surgery Training (FAST) workstation that is currently used by the Arthroscopy Association of North America, the American Academy of Orthopaedic Surgeons and the American Board of Orthopaedic Surgery during courses offered to residents and practising surgeons.11,12
The Canadian Shoulder and Elbow Society (CSES; formerly Joints Canada) Annual Shoulder Review Course committee chose to include a dedicated training session with a surgical simulator to assess the performance of senior orthopedic residents during simulated arthroscopic anterior stabilization (Bankart repair) before and after the CSES Annual Shoulder Review Course. The main purpose of this study was to evaluate the performance of senior orthopedic residents during simulated Bankart repair before and after a national shoulder review course. The primary objective of the study was to assess improvement in knowledge and surgical skills for arthroscopic Bankart procedures. The chosen outcomes were score on a 10-question multiple choice test, surgical performance based on a standardized procedure checklist, and precision of anchor placement. We hypothesized that course participants would show significant improvement in anchor placement precision and on Bankart repair knowledge after the course.
Methods
Course
The CSES Annual Shoulder Review Course is a 3-day course that covers all aspects of shoulder surgery, including sports medicine, open and arthroscopic techniques, trauma surgery and arthroplasty. The course received the ethics committee approval. Teaching related to the management of shoulder instability, including surgical techniques, consists of a 1-hour case-based small-group session, a 1-hour formal lecture and a 2-hour laboratory session using both Sawbones and cadaveric models.
Participants
All Canadian senior orthopedic residents (postgraduate year [PGY] 4 and 5) and fellows were invited to the CSES Annual Shoulder Review Course in Montreal, Québec. Experienced fellowship-trained shoulder surgeons provided the teaching throughout the review course. There were no exclusion criteria.
Written examination
All participants took a written examination before and after the review course. All participants were unaware of the research topic at the precourse test. The written examination included 10 multiple choice questions chosen according to relevant shoulder instability features and generated by an expert committee in accordance with the Delphi method.13 There was no time limit for this examination, and no reference material or communication was allowed.
Surgical performance assessment
All participants underwent a surgical performance assessment, using a rating instrument, before and after the review course. The instrument was created following expert consultation and consensus.13
Several stations were available to allow the evaluation of multiple residents at the same time. Each station provided written instructions, and the required instruments were available on the station’s worktable. Participants were instructed to insert the most inferior of the anteroinferior glenoid suture anchor, using 3 mm Bio-SutureTak anchors (Arthrex), into a synthetic right shoulder physical simulator model, which included synthetic skin and capsule (Sawbones), under arthroscopic guidance with a 30°, 3.5 mm arthroscope. The shoulder was in the beach chair position, but could be modified to the lateral decubitus position if requested. Anterosuperior and anteroinferior portal cannulae were inserted posteriorly before testing. Every resident had to choose the most appropriate cannulae for the procedure. A drill, drill guide, and anchor were available for each trainee.
Experienced, fellowship-trained shoulder surgeons assessed performance using an evaluation form created by a team of experts. The form included a task-specific checklist and an overall global rating scale, based on a 5-point Likert scale, to evaluate surgical technique, including anchor placement and insertion, use of appropriate portals, and camera manipulation. If a visual evaluation of the anchor placement was difficult intraoperatively, the expert assessors could manipulate the camera (once the procedure was completed) to determine the final anchor placement.
Evaluation of anchor placement
The exact position of the anchor was analyzed using a 3D camera (Optotrack 302, NDI) and 3D reconstruction software (Catia V5R20, Dassault system). After each pre- and post-course test, all the glenoids were detached from the synthetic ligaments and tendons. The labrum was slightly elevated. In cases where the anchor was partially inserted through the labrum, the labrum was carefully elevated to expose the anchor.
Every glenoid was then placed against a Plexiglas sheet (Fig. 1). The position was controlled by a line drawn on the translucent sheet. The centre of the Plexiglas sheet and the centre of the glenoid were superimposed. The glenoid was maintained with a clamp. To create a 3D model, a recording of 6 points was made for each specimen with an optoelectronic pointer and a 3D camera (Optotrack 302, NDI). Points A to D were the extremes of a 6 cm × 8 cm rectangle drawn on the Plexiglas sheet (Fig. 1, Fig. 2). Point E was the projection of the entry point of the anchor on the Plexiglas sheet. We measured the distance between anchor and joint surface. Reconstruction of all 5 points was done using computer-aided design software (Catia V5R20, Dassault system). Measurement and calculation of these 5 points using the software extracted the anchor position, including the insertion angle (the angle between the long axis of the anchor and the glenoid joint surface), the glenoid clock face measurement (position of the entry point on the glenoid surface; depends on point E), and depth of insertion of the anchor from the glenoid surface.
Based on literature review, the course committee determined the ideal position: 5 o’clock (on a right shoulder), a 45° angle of insertion, and a final anchor position 2 mm below the articular surface of the glenoid.13–18 To determine the specific error of the anchor placement measures, 3 repeated measurements were taken per specimen. The error was calculated as the difference between the measure and the ideal position (e.g., if the entry point is at 4:45 on the clock face, the error will be 5:00 – 4:45 = 0:15). Paired t tests were then performed to compare mean values of measurements before and after the review course.
Statistical analysis
Results are presented as means ± standard deviations (SDs) and 95% confidence intervals (CIs) for continuous variables and as percentages for categorical variables. We used paired t tests to compare pre- and post-course continuous variables and χ2 tests to compare categorical variables. We considered results to be significant at p < 0.05, and we used SPSS software version 23 to perform all tests.
To perform the power calculation, we used duration of procedure, as this is a quantitative measure. We determined that a 25% improvement would be significant, and with a power of 80%, an α of 0.05 and an SD of 30% we needed a minimum of 24 participants per group.
Results
Participants
Fifty-eight participants were recruited for the study: 16 PGY4 residents, 36 PGY5 residents and 6 clinical fellows. Thirty-six participants were men. All 58 residents completed the precourse test and 57 completed the post-course test.
Written examination
Questions and results of the written examination are shown in Table 1. The mean score on the written examination increased from 59% ± 15% (95% CI 40%–78%) to 68% ± 12% (95% CI 48%–88%) following the course.
Surgical performance assessment
The average total surgical task time decreased from the precourse test to the post-course test by 40%, from 4:40 min (95% CI 2:46–5:34) to 2:53 min (95% CI 2:30–3:16) (p < 0.001). Anchor placement, insertion angle and arthroscopic portal choices all improved. Thirty-five students (61%) increased their manipulation skills, 9 students (16%) had no change in their results, and 13 students (23%) decreased their manipulation skills. The mean score for manipulation skills decreased from 71% to 68%. The overall score for the surgical technique assessment improved from 56% before the course to 67% after the course (p < 0.001) (Table 2).
Evaluation of anchor placement
To evaluate anchor placement, 58 shoulder models were used for the precourse test. Seven were lost or stolen, and of the remaining 51 models, 8 had no anchor left inside (but the anchor tunnel was present), leaving 43 models with anchors present. For the post-course test 57 models were used, 2 were lost or stolen, and of the remaining 55 models, 11 had no anchor left inside (but the anchor tunnel was present), leaving 44 with anchors present. Results of the anchor placement evaluation are shown in Table 3 and Table 4. The depth of anchor placement was measured on a total of 87 synthetic glenoids with anchors present (pre- and post-course tests). Models without anchors were the result of technical errors.
The precision of the anchors’ insertion points improved following course completion (p < 0.001); however, no significant improvement was seen in the angle of insertion or the anchor depth. A clock was applied over the glenoid face to quantify entry points.
Entry point before the review course ranged from 1:00 to 7:20 (mean 4:15 ± 1:10); after the course it ranged from 3:05 to 6:00 (mean 4:25 ± 0:35) (p = 0.37; Fig. 3). The precision improved, as shown by the significant decrease in deviation for the desired entrance point on the clock face, from 1.0 ± 0.8 hours to 0.6 ± 0.4 hours (p < 0.001).
Angle of insertion before the review course ranged from 4° to 79° (mean 43° ± 16°); after the course it ranged from 20° to 76° (mean 44° ± 13°, p = 0.25; Fig. 4).
The depth of anchor insertion ranged from −0.7 mm to 20.0 mm precourse (mean 5.7 mm ± 4.0 mm); after the course it ranged from −1.3 mm to 13.4 mm (mean 6.2 mm ± 3.8 mm, p = 0.43; Fig. 5). Figures 3, 4 and 5 represent individual matched performances.
Discussion
Bankart repairs are commonly performed for recurrent instability of the shoulder, with the objective of restoring normal shoulder function and stability.14 This surgical technique commonly requires the insertion of suture anchors into the glenoid rim and reattachment of the torn labrum using sutures fixed on those anchors. The position and number of anchors have been shown to influence clinical outcomes, and mastering this skill contributes to treatment efficacy.14–19 Senior residents and shoulder clinical fellows improved their knowledge of arthroscopic Bankart lesion repair and their performance of a simulated repair on a dry surgical model. Residents may benefit from courses with curricula that include surgical simulation to both teach and assess surgical skill acquisition, in combination with didactic or small group sessions and cadaveric laboratories. Indeed, the relevance of simulation training as a part of orthopedic residency programs has been gaining in credibility with recent publications.8,20 Moreover, Angelo and colleagues21 established the importance of defining specific benchmarks for evaluation in order to achieve progress. In terms of patient safety, this method has the advantage of furthering essential knowledge that is not appropriate for residents to learn in the operating room. Furthermore, residents should be formally assessed on their ability to perform these tasks to a minimal level of competence before performing them in the operating room. In the present study we saw a 40% improvement in the time to completion for the evaluated task, but we also evaluated the quality of the anchor placement; as pointed out in a recent editorial, “a task may be done quickly, but not necessarily well.”22 One of the strengths of this study is the inclusion of residents from all Canadian residency programs, providing a good national overview. Furthermore, it includes technical and theoretical knowledge assessments with a chosen model that closely resembles a real shoulder with synthetic skin and ligaments. This forced participants to look at the screen when performing the procedure and prevented them from seeing the bone insertion point directly. Entry point measurements were taken by a biomechanical engineering team with cutting-edge technology to ensure the greatest precision possible. Finally, this event was a fantastic opportunity to bring together shoulder surgeons from across the country and promote excellence in shoulder surgery.
Limitations
Limitations of the study include some discrepancies observed when comparing the examiners’ evaluations of surgical performance and the 3D camera evaluations. The 3D evaluations showed a reduction of the mean range and of the error and error range rather than the means themselves (Table 4). The assessments performed by the experts appear to have overestimated the amount of improvement from pre- to post-test in comparison to the 3D anchor evaluations. Possible explanations for this include examiner intra- and interobserver variation, or qualitative improvement of camera management by residents, which gives a more positive impression. Participants may also have improved in the manipulation of the specific anchor instrument through the standard learning curve. Other limitations include the lack of data on participants’ prior experience, the absence of a control group, and the absence of expert results for comparison.23 Future studies could include data from the evaluating surgeons to be used as the gold standard for comparison to establish validated metrics for assessment.21,23,24 Filming the procedure could also have provided additional material for evaluation, as well as information on left- or right-handedness of the residents, which may influence performance.25
Further investigation is required to better understand which methods are suitable for teaching and refining surgical procedures and techniques among senior residents and experienced surgeons, especially in terms of knowledge transfer with performance in the operating room. Many national and international courses offer cadaver training sessions, but these can be costly and difficult to organize compared with traditional didactic teaching methods.26–28 In a context of limited university budgets, partnerships with the industry remain the only avenue. Performance on arthroscopy simulators has been strongly correlated with performance on cadavers,29 and the use of simulators to supplement or replace certain parts of courses is appealing.28 Indeed, it would solve the problem of time lost during the surgical approach before performing the specific step that the course aims to improve. For more experienced surgeons, it is unnecessary to complete all the preparatory stages, especially in arthroscopic surgeries. Using simulators can get the surgeon directly to the specific step that needs to be practised, such as anchor placement in a difficult orientation or learning to use new materials.7 Recent studies have shown that surgical simulation training can improve performance, and longitudinal integration of skills training should continue throughout a training program.8,20,30
Conclusion
Resident knowledge of shoulder instability and simulated performance of an arthroscopy-assisted stabilization improved following a structured intervention. Our data support the inclusion of dry model surgical simulation as part of a surgical skills course both for training and assessment of orthopedic surgery residents.
Footnotes
Members of the Canadian Shoulder and Elbow Society author group: Ghassam Alami (Montreal, Que.), Darren Drosdowech (London, Ont.), Tim Dwyer (Toronto, Ont.), Véronique Godbout (Montreal, Que.), Marie-Ève Lebel (London, Ont.) and Nicholas Mohtadi (Calgary, Alta.).
Participants of the CSES Annual Shoulder Course: Ghassam Alami (Montreal, Que.), George Athwal (London, Ont.), Frédéric Balg (Sherbrooke, Que.), Robert A. Balyk (Edmonton, Alta.), Ryan T. Bicknell (Kingston, Ont.), David Blanchette (Montreal, Que.), Martin Bouliane (Edmonton, Alta.), Patrick Chin (Vancouver, BC), Darren Drosdowech (London, Ont.), Tim Dwyer (Toronto, Ont.), Kenneth J. Faber (London, Ont.), Véronique Godbout (Montreal), Richard Holtby (Toronto, Ont.), Aleem Lalani (Edmonton, Alta.), Peter Lapner (Ottawa, Ont.), Marie-Ève Lebel (London, Ont.), Robert Litchfield (London, Ont.), Michael D. McKee (Toronto, Ont.), Nicholas Mohtadi (Calgary, Alta.), Jaydeep Moro (Hamilton, Ont.), Diane Nam (Toronto, Ont.), Duong Nguyen (Toronto, Ont., Stéphane Pelet (Québec, Que.), Dominique M. Rouleau (Montreal, Que.), David Sheps (Edmonton, Alta.), John Theodoropoulos (Toronto, Ont.) and Christian Veillette (Toronto, Ont.).
Competing interests: D. Rouleau is a consultant for Bioventus and Wright Medical. V. Godbout has received speaker fees from Arthrex.
Funding: The course was fully financed by industry collaboration. The following companies provided funds but did not influence the course content: Arthrex, Conmed, DJO Canada, Ostek orthopaedics Inc, Smith and Nephew, Synthes, Tribe Medical Group, Zimmer and the COA.
Contributors: D. Rouleau, Y. Petit, D. Drosdowech and M.-È. Lebel designed the study. D. Rouleau, R. Bedard, G. Alami, D. Drosdowech, T. Dwyer, V. Godbout, M.-È. Lebel and N. Mohtadi acquired the data, which D. Rouleau, F. Canet and Y. Petit analysed. D. Rouleau and V. Godbout wrote the article, which all authors reviewed and approved for publication.
- Accepted September 7, 2018.