Evaluating resident education

James P. Waddell

Resident education in surgery continues to evolve. Driven by a number of factors, including regulated hours of work, trainees’ expectations regarding lifestyle during training, the introduction of rotation-specific objectives and finally the incorporation of the CANMEDS 2000 objectives into postgraduate training across Canada, fundamental changes in how surgical education is delivered are occurring in all programs. As a consequence of these changes, the Royal College has become more receptive to meeting the different demands and needs of the various specialties in terms of content and delivery of educational materials that provide the basic objectives of training (which in turn are usually written by the appropriate specialty committee).

In the face of these numerous changes in education, it is time to rethink our current methods of evaluation. In most surgical specialties, the method of evaluation has not changed. Over the course of the resident’s training, evaluation includes:

college-mandated in-training evaluation reports
a supporting letter from the program director at the conclusion of training
a final in-training evaluation report
documentation of satisfactory completion of training (required for the candidate to sit the qualifying examination)
the qualifying examination (written and sometimes oral)

The Qualifying Examination looms large throughout the resident’s career, assuming ever more importance as the final years of training begin. With the qualifying examination now occurring in the final months of residency, many trainees find clinical experience in their final year of training severely compromised by the need to study. To complicate matters, in many specialties the final examination consists of both written and oral segments, necessitating hours of preparation by means of mock oral examinations to teach the candidate “examanship” — perhaps a more important attribute for success on the final oral examination than a knowledge of surgery.

It’s time that we and the College rethought this form of evaluation. There is scant evidence to support the oral examination as a satisfactory surrogate for close personal observation by skilled individuals over the several years of residency training. Who would know better the surgical qualities of trainees: someone who subjects them to a gruelling 90-minute oral examination in an artificial environment or someone who has had an opportunity to work with them over several months or even years in a variety of circumstances? The answer seems intuitively obvious, and one has to wonder why we persist in this “all-or-nothing” oral examination at the end of 5, 6 or 7 years of training.

It would be far more sensible for the College to mandate yearly examinations for all surgical trainees. Let’s imagine this scenario: at the end of first year, the Qualifying Examination Part II; at the end of second year, the Principles of Surgery examination; and then each subsequent year, a specialty-specific examination for the appropriate year of surgical training. This would have a number of advantages, not the least of which would be the ability of the training program to spot the struggling resident who lacks adequate knowledge to be advanced to the next level of training. We do this routinely in every other educational environment, and it seems paradoxical that we don’t apply the same type of assessment annually to surgical trainees that we do to high school students. The annual examination would also promote good study habits among trainees and have them working throughout their residency on knowledge acquisition rather “cramming” in the last 18 months of their surgical training.

I think this is an idea whose time has come: regular assessment by written examination throughout the course of the residency, with a matriculation examination at the end of the final year of training and a supporting letter from the program director attesting to the trainee’s clinical skills and ability to function as a consultant surgeon.

The surgical community should encourage the College to continue its progress not only in the delivery of surgical education but also in the assessment of the value of such education.