In the February 2016 issue of the CJS, Malik and colleagues1 present data comparing recurrence rates for inguinal hernia repairs done in Ontario general hospitals to those done at the Shouldice Hospital. The Ontario Association of General Surgeons has several concerns about this paper, especially about the issue of selection bias and fair up-to-date comparison.
The Shouldice Hospital is a controversial entity among Ontario general surgeons, who generally consider them to be “cherry picking” the easiest hernias. Malik and colleagues acknowledged the potential for selection bias and that their databases lacked detailed clinical information on smoking; obesity; and hernia characteristics, such as size, which would allow for fair comparison. The authors attempted to measure selection bias indirectly by comparing outcomes of a small subgroup of patients who had a consultation at the Shouldice Hospital between 2004 and 2006 and then had their surgeries elsewhere.
The recurrence rate in this group was 3%, which the authors felt was insufficient to explain the large difference in recurrence rate. We, however, beg to differ and would argue that the presented data suggest a large selection bias. Since this study ended in 2007, the subgroup had an average of only 2 years of follow-up, and their 3% early recurrence rate is approximately double the 2-year recurrence rate when data from all years is considered (see Fig. 1 of their study). If one adjusted for secular trends, where the recurrence risk for hernias done from 2003–2007 was half that of the reference range of 1993–1997 (see Table 3 of their study), it would make the recurrence rate in those rejected several times above the concurrent provincial average and in the range where selection bias could account for a significant portion of the results.
The authors’ estimate of a 10% rejection rate based on consultation rates is also highly contentious.
Shouldice promotes itself as not requiring a referral (only one-third of their patients in 2004–2006 had a referral and consultation), so a huge proportion of their selection and rejection process would be completely invisible to the databases used by these researchers. The selection process at Shouldice has a significant emphasis on ideal body weight, and many patients are rejected owing to failure to achieve weight loss goals that are not imposed by most other surgeons. When one considers that 72% of middle-aged men in Ontario are obese or overweight,2 a large proportion of these patients would be ineligible for surgery at the Shouldice Hospital despite its publicly funded status. This is a level of discrimination not seen at other Ontario hospitals.
We would also point out that there have been at least 16 randomized controlled trials of the Shouldice repair wherein selection bias is implicitly eliminated by randomization. A meta-analysis3 and Cochrane review4 of these trials clearly shows a strong advantage for mesh repairs, essentially showing the exact opposite of what Malik and colleagues reported, with mesh-based repairs being 4 times less likely to recur. In short, Malik and colleagues report an effect that is 16 times better than what the randomized literature shows. We also criticize the study for focusing on hospital volume rather than surgeon volume or technique, both of which were tracked in the databases used. The impact of hospital volume would be expected to influence only cases requiring complex hospital care and, indeed, this study found no trend between high- and low-volume general hospitals. Surgical technique, particularly the use of mesh, has been shown in multiple studies to have a significant impact,3,4 yet it was not adjusted for.
We also criticize the study for presenting aggregate data that clearly span a transitional period in hernia repair, where most surgeons made an appropriate evidence-based shift away from tension-based tissue repairs to tension-free mesh repairs.3 The overall 50% reduction in recurrence rate over the course of this study is hidden in the fine print and is probably due to technique. A comparison with an old technique that is largely abandoned only magnifies the difference in recurrence rates and doesn’t inform about current practice.
The study does raise the interesting question as to whether extremely high surgeon volume can impact results. A population-based study from Sweden5 found very little impact of surgeon volume, with recurrence rates generally plateauing at an annual rate of only 10 per year and, though volumes did not approach those of the Shouldice hospital, there was no trend toward better outcomes with higher volumes.
Finally, we point out that the fee code for recurrence is essentially a self-reported variable that is unvalidated in the setting of a private company with a marketing strategy that is based on low recurrence rates.
We feel that this paper gives a misleading picture of the current status of inguinal hernia surgery in Ontario and would caution health care planners that selection bias rather than process issues are by far the most likely explanation of the results presented. We do acknowledge that its publication will hopefully stimulate an important debate about the quality of hernia surgery in Ontario and the importance of measuring adjusted outcomes, of which recurrence is but one.