The editorial by Tandan and Langer in this issue (pages 351 and 352), in which they discuss our paper on variation in surgical rates (pages 361 to 367), raises a number of questions that require clarification and in some instances rebuttal. We agree that studies of variation have focussed on surgical procedures because they are discrete and quantifiable, and we suspect that comparable variation could be demonstrated for medical procedures and diagnostic categories. We also agree that outcome research could begin to settle the question of too much or too little surgery. Our work makes it possible to focus such research on those procedures for which the variation in rates is most marked and on geographic locations with outlying rates (both high and low), two situations where the potential payoff of follow-up studies is greatest. Indeed, our paper suggests a number of steps and outcome studies to clarify the reasons for the variation and to develop strategies to deal with it. The editorial points out the importance of clinician and patient preferences in the decision to operate, and it is our hope that these preferences could be most effectively quantified and assessed using the procedures with the greatest variation and the locations with outlying rates.
We do not agree with the editorial that our selection and classification of procedures was biased and arbitrary. We do not see how the selection of procedures, which was done for an earlier study1 in which we analysed the same 39 procedures, could have biased the results of this study, and the reasons for choosing those procedures are dealt with in both papers. In our opinion there was no bias or arbitrariness in the classification of procedures, which in fact was into three categories: primarily discretionary, intermediate and primarily non-discretionary. Tandan and Langer occasionally forgot to include the important qualifier “primarily” in naming the categories, and they never mentioned the intermediate category. The classification was done by an internist with considerable clinical experience, albeit a “nonsurgeon.” The classification could possibly have been improved if a surgeon or a group of surgeons had done it. However, we believe that the editorial would have been more helpful if it had assessed the classification itself instead of dealing only with its possible shortcomings. In other words, we think the authors should have considered how well the classification used in this analysis actually worked and whether the procedures ended up in appropriate categories. Readers of the paper can certainly try to make that determination themselves.
As stated in our paper, the classification was done independently of and prior to the analysis of variation. Our previous paper identified outliers, and this paper deals with underlying variability. We, of course, knew from our previous study which rates were outliers. But in our subsequent assessment of variability, we devised methods that are relatively unaffected by the presence of outliers. We deliberately designed our index of variation to exclude the most extreme rates so that it would not be unduly influenced by outliers, which distort the assessment of variation.
The use of the primarily discretionary, intermediate and primarily non-discretionary classifications did not imply that the the primary factor in decision making regarding the choice of operation is the surgeon’s opinion. The extensive review of the literature cited in the first paragraph of our paper (21 references) identifies many of the other factors associated with variation in rates. In-depth studies of procedures with the greatest variation and locations with outlying rates could most profitably examine many of the attitudinal and resource-based variables as well as the clinical indications.
We did not ignore the possibility that rates could be too low. On the contrary, our earlier study1 was entitled “High and low surgical procedure rates . . .,” and our index of variation excludes both extremely high and extremely low rates. However, in our earlier study, we found that three-quarters of the outliers were high rates, and most of the low rates were in areas with medical schools and teaching hospitals. These results suggested that the high surgical procedure rates were too high.
We agree with Tandan and Langer that, whenever possible, the variation in rates should be related to the indications for a particular operation. The editorial cites the work of Hall and Cohen2 on hysterectomy in Ontario (reference 41 in our paper), but the authors are apparently unaware that we replicated that study for all of Canada and reported its results in an article entitled “Hysterectomy rates by diagnostic variation among Canadian census divisions”3 (reference 42 in our paper). Our findings and those of Hall and Cohen were essentially the same. Hysterectomy for cancer is primarily non-discretionary and it had the least variaion, whereas hysterectomy for menstrual hemorrhage and sequelae, which is primarily discretionary, showed the most variation in its rates. These findings further support our conclusion that greater variation is associated with procedures (and indications) that are primarily discretionary. Rates by indications could be determined for other primarily discretionary procedures such as prostatectomy, cholecystectomy and, if outpatient data were available, hemorrhoidectomy and varicose-vein surgery.
The editorial points out the problem of the unavailability of outpatient surgery data, a concern that we share and that we discussed in our paper. We explained that “to test the new index” we selected “operations expected to show increased variation because of the absence of outpatient surgery from our data file.” Our conclusions are the same with or without the 10 procedures that are sometimes performed on an outpatient basis, and these procedures could easily be ignored in the analysis, because each of the 39 procedures was analysed independently. If those 10 procedures were eliminated from the analysis, hysterectomy and cesarean section would rank first and second in variation instead of fourth and sixth. We suggested further in-depth studies for three primarily discretionary procedures (prostatectomy, hysterectomy and cholecystectomy) that were done on an inpatient basis (where the data were complete), not for procedures for which the rates were confounded by unknown numbers of outpatient operations.
The strength of our paper is not in how many discretionary procedures are in the top half of the rankings, as the editorial implies, but in the fact that the primarily discretionary procedures rank higher in variability than the intermediate procedures, which rank higher in variability than the primarily non-discretionary procedures.
We agree with the editorial that, “studies using important outcomes as end points . . .” are necessary to develop appropriate practice guidelines and that “it is the health care providers who need to become involved in developing the evidence-based standards of practice . . . .” We are pleased that the editorial acknowledges that our paper has “identified several procedures for which the indications need to be examined and alternative treatments need to be compared in future clinical studies.” Like Tandan and Langer we also look forward to the day when outpatient surgical data are available, so that all-inclusive counts and rates can be calculated for all operations. Meanwhile, we are pleased that our analysis may light the way for subsequent studies of the causes of variation and the development of appropriate guidelines.