Abstract
Background: Recognition of tissue planes during surgery appears to be a skill acquired with experience. We conducted a pilot study to test this hypothesis using a novel method for evaluating this skill in a simulated environment.
Methods: Twelve surgeons of varying levels of experience were shown 16 captured images from a mesorectal excision. For each image, they were asked to draw the ideal dissection plane with a stylus on a tablet computer. We used a novel metric for comparing agreement between lines to determine the level of precision observed between junior and senior trainees and consultant surgeons and measure the accuracy of junior and senior trainees compared with consultant surgeons.
Results: We observed significant differences in precision for 9 of 16 images; 7 of these followed the predicted stepwise pattern associated with level of experience. Using consultant surgeons as the reference standard, we observed significant differences in accuracy between senior and junior trainees for 11 images, with senior trainees being more accurate in 10 of them. Only 2 images failed to contribute significant findings to our analysis.
Conclusion: The findings of this pilot evaluation of a novel method for measuring a surgeon’s ability to recognize tissue planes in a simulated model show that skill improves with experience. Further evaluation of this method will reveal its utility as an assessment tool and possibly as a training instrument.
It is considered a fundamental principle of good surgical technique to respect tissue planes during surgery. Tissue planes tend to be avascular; therefore, bleeding can be reduced. In addition, there is growing evidence of improved oncologic outcomes associated with adherence to dissection along tissue planes. This has been demonstrated clearly for rectal cancer resections.1 There is compelling evidence to suggest this is also true for colon cancer surgery2 and hepatobiliary surgery.3
One of the challenges encountered when teaching trainees to operate within tissue planes is how to facilitate the trainee’s recognition of the plane. What is intuitively obvious to the expert surgeon is not always obvious to the trainee. There are likely visual clues that allow the expert to see what the novice does not yet appreciate. Currently it is not clear how this visual skill is learned during the course of clinical apprentice-based training.
Our hypothesis is that if the ability to perceive tissue planes is an acquired skill, then it should be possible to develop a visual test of that skill that discriminates between novice and expert surgeons.
Methods
We captured a series of 16 digital images representing progressive stages of a mesorectal excision from digitally recorded video. The chosen procedure was a da Vinci-assisted (Intuitive Surgical) laparoscopic proctectomy, and we obtained consent from the patient for the recording and use of the images for surgical instruction. The use of a da Vinci-assisted procedure, as opposed to a laparoscopic procedure, was of no particular consequence. We chose this particular case simply because the image quality and exposure of the tissue planes captured in the video were of such high quality that they seemed ideally suited for our pilot study. The images themselves were selected from video and chosen on the basis of clear views unobstructed by surgical instruments; they represented multiple viewpoints of the mesorectal envelope, as seen from the posterior, anterior, left and right dissection planes. Images were labelled minimally for orientation only.
Images were transferred to an iPad 2 (Apple Inc.) and presented in Sketchbook Pro software (Autodesk Inc.). Study participants were able to draw on the presented image using a stylus for capacitive touch screens (Slim Stylus, Targus). Each surgeon’s line was saved in a separate layer for later analysis.
We recruited 12 surgeons with varying levels of expertise to view the images on the iPad. Since this was a pilot study and we had no preconception as to what kind of results to expect, we did not perform any sample size or power analysis calculations.
The participating surgeons included 4 consultant surgeons experienced in performing a mesorectal excision, 4 senior trainees and 4 junior trainees. For each image, we instructed the surgeons to draw a free line representing the dissection plane on the image as if the stylus were a scalpel or preferred dissecting instrument. The surgeons were instructed to draw the line as precisely as possible and to extend the line for as long as they felt they could comfortably appreciate the dissection plane. They were permitted to erase and redraw the line, if needed.
Statistical analysis
To compare one drawn line with another, we formulated a distance metric (similar to the Hausdorff measure used in computational geometry4) based on the Euclidean distance between evenly spaced points along the arc. Using Matlab (MathWorks), each line was iteratively bisected 3 times (b = 3), resulting in 8 segments specified by pairs of Cartesian coordinates (p = 2b+1; 9 pairs for the present study; Fig. 1). For each study group (G), this distance metric between any 2 lines (a, b) for a given image (i) was calculated as the summed distance (d), in pixels, between each of the coordinate pairs (x, y) as follows:
Within each study group, there were 6 possible pairs of lines that could be compared for each image. Across the set of images, we calculated the means ± standard deviations of these 6 pairs for each of the 3 study groups.
The mean distance metric is inversely related to group precision (i.e., the greater the mean distance metric, the less agreement or consensus there is within that group with respect to the location of the ideal dissection plane). For each image, we compared this measure of precision for all 3 study groups using 1-way analysis of variance (ANOVA) with a significance level of p = 0.05. When significant differences were found, we performed pairwise comparisons using the Student t test. We used Šidàk’s equation to correct the level of significance for multiple comparisons: α[PT] = 1 − (1 − α[PF])1/c, where α[PT] is the corrected probability of a type-1 error for an individual test and α[PF] is the probability of a type-1 error for a family of tests. For 3 comparisons and α[PF] = 0.05, then α[PT] = 0.017.
Since there is no “correct” line representing the tissue plane in each image, and therefore no gold standard with which to compare the accuracy of each line, for the purpose of our analysis we considered the consultant surgeons to be the experts. Accordingly, for each image, we compared each trainee line with each consultant surgeon line in the fashion described above. This produced 16 comparisons (4 × 4) per group per line. The average of these 16 comparisons represents group accuracy, with smaller distances associated with greater accuracy. These were analyzed using the Student t test, with significance set at p = 0.05.
Results
For 9 of the 16 images (1–3, 5, 6, 9–11, 16), we observed a stepwise increase in precision according to level of experience from junior trainees to senior trainees to consultant surgeons, although not all were significantly different (Table 1). We observed significant differences in precision in 9 of the 16 images (2, 4–7, 9, 10, 13, 16). Considering these 9 images only, pairwise comparisons revealed significant differences in precision between 10 pairings. In only 1 image (7), significantly greater precision was found in the senior trainee group than in the consultant surgeon group.
In 13 of 16 images, the accuracy of the senior trainee group was greater than that of the junior trainee group (Table 2). These differences were significant for 10 of the images (3–7, 9–11, 14, 16). For image 12, the junior trainees appeared to be significantly more accurate than the senior trainees. We used the results for image 5 as a representative example of these findings. Lines drawn by consultant surgeons (Fig. 2), senior trainees (Fig. 3) and junior trainees (Fig. 4) differed significantly on 1-way ANOVA but not by pairwise comparison (Table 1). For this image, senior trainees were significantly more accurate than junior trainees (Table 2).
Images 8 and 15 contributed no significant findings to any of our analyses.
Discussion
Visual processing of the spatial relations of image properties is known as visual spatial ability. It has been proposed that visual spatial ability can be classified in 5 categories: edge and surface extraction, edge orientation encoding, whole object recognition, mental visualization involving the spatial relations of object parts in 2 dimensions (2D), and mental visualization involving 2D and 3-dimensional (3D) spatial rotations and translations. These represent a hierarchy of ability ranging from low- to high-level ability.5,6
Nonverbal, visual–spatial problem solving abilities and the ability to distinguish essential from nonessential detail has been shown to correlate with superior surgical skill in general surgery trainees.7 This finding was based on correlation between a battery of tests measuring visual–spatial perception, motor sequencing and fine-motor coordination and stress tolerance with a novel rating scale of surgical performance. This rating scale was composed of 12 items assessing mainly technical performance.
Since publication of that report, a growing body of evidence has been produced that demonstrates correlation between performance on various tests of visual–spatial ability and surgical performance.8 These studies have focused on a range of topics from measures of whole surgical tasks assessed in a laboratory setting to overall assessment of clinical performance. There seems to be stronger correlation between task performance and high-level visual–spatial abilities, such as 3D spatial rotations.
Relevant to the performance of laparoscopic surgery, a unique method for gauging the perceived depth relations of objects presented in a 2D display provided some support for the claim that visual perceptual skills necessary for understanding 3D structure can be improved with practice.9 However, some neurophysiological studies10,11 have suggested that in specific perceptual learning, the benefits of perceptual training would be relatively limited.
In the present study, we focused on the initial perception and spatial reasoning skills needed to initiate performing a particular surgical procedure. Based on the presumption that in order to dissect a tissue plane one must first be able to visualize that plane, we developed a novel technique for assessing the ability of surgeons to identify dissection planes on static images acquired from actual surgical procedures. It was the goal of this pilot study to evaluate the tool by determining whether there are measurable performance differences between surgeons and trainees.
Despite our small number of participants, we found significant differences consistent with our a priori hypothesis that recognition of surgical planes is a skill acquired through experience. Considering the within-group variability as an inverse measure of concordance or group precision, we found that we were able to demonstrate a stepwise increase in precision from junior trainees to consultant surgeons.
Limitations
One of the limitations of our methodology is the unavailability of a “correct” plane to use as a reference standard. Therefore, in order to assess accuracy we chose to consider the expert (consultant surgeons) group as the reference standard against which we compared trainee groups. We argue that in many instances the perfect plane of dissection may be a matter of conjecture. Using the experts as a control group, we preserved the variability inherent in multiple opinions. We anticipate that a larger number of experts would narrow this variability or improve the precision of the expert group. Interestingly, for this pilot study our small sample size was still sufficient to demonstrate that senior trainees were predominantly more accurate than junior trainees, which was consistent with our hypothesis.
One of the potential confounders of this study is content validity. It is possible that rather than testing differences in ability to perceive tissue planes, we are simply testing differences in trainees’ familiarity with this particular surgical procedure based on their experience and recall. Some of the lines drawn by junior trainees were not only away from the ideal plane, but also clearly demonstrated a lack of recognition of either the anatomy or the next step in the procedure. Future study will need to include an assessment of procedure-specific experience to determine whether this represents an overriding factor.
Another potential weakness is the small number of participants in this pilot study. Despite the ability to appreciate significant differences among the study groups for many of the images, the response of a single individual has the potential to have a large effect on the group. This will be addressed in future studies with larger study groups.
It would be a valid criticism to argue that in practice many factors contribute to the dynamic recognition of surgical planes. The purpose of this pilot was simply to determine if our methodology would be sensitive enough to discriminate between novices and experts based on static images alone. Given that we demonstrated the ability of this methodology to detect expected differences among study groups, we now have the possibility in future endeavours to study the effect of additional clues. This is an area ripe for further investigation, including the use of lead-in video to static images, the contribution of innate psycho-motor ability, the difference in visual clues obtained from open compared with laparoscopic surgery or even the role of narrow band imaging. Some of these areas are already being pursued at our institution.
Not all of the images used in this study contributed to significant findings, which may be a result of type-II error owing to the small sample size. It is also possible that some images were less discriminating than others. Future study will require evaluating a larger library of images so that a better understanding of what represents a discriminating image can be developed.
Conclusion
Our pilot study demonstrated a novel method of evaluating visual perception and spatial reasoning of surgical tissue planes. This method shows promise as a tool that possesses the sensitivity to discriminate between levels of experience of surgical trainees. Many questions remain to be addressed through future study and work already in progress. With further development, this methodology has the potential to be developed into both an assessment tool and a training tool to enhance surgical instruction.
Acknowledgements
This research was supported in part by a grant from the Ontario Research Fund, Research Excellence program.
Footnotes
This paper was presented at the 2013 Canadian Surgery Forum by S. Ali and was recipient of the CAGS Basic Science Award.
Competing interests: None declared.
Contributors: C. Schlachta and R. Eagleson designed the study. C. Schlachta acquired the data, which all authors analyzed. C. Schlachta and R. Eagleson wrote the article, which all authors reviewed and approved for publication.
- Accepted July 30, 2014.