CAGS Evidence-Based Reviews in Surgery
The term “evidence-based medicine” was first coined by Sackett and colleagues as “the conscientious, explicit and judicious use of currentbest evidence in making decisions about the care of individual patients.”1 The key to practising evidence-based medicine is applying the best current knowledge to decisions in individual patients. Medical knowledge is continually and rapidly expanding, and it is impossible for an individual clinician to read all the medical literature. For clinicians to practise evidence-based medicine, they must have the skills to read and interpret the medical literature so that they can determine the validity, reliability, credibility and utility of individual articles. These skills are known as critical appraisal skills. Generally, critical appraisal requires that the clinician have some knowledge of biostatistics, clinical epidemiology, decision analysis and economics, as well as clinical knowledge.
The Canadian Association of General Surgeons and the American College of Surgeons jointly sponsor a program entitled “Evidence Based Reviews in Surgery (EBRS),” which is supported by an educational grant from ETHICON and ETHICON ENDO SURGERY, both units of Johnson & Johnson Medical Products, a division of Johnson & Johnson, and ETHICON INC. and ETHICON ENDO-SURGERY, INC. divisions of Johnson & Johnson Inc. The primary objective of this initiative is to help practising surgeons improve their critical appraisal skills. During the academic year, 8 clinical articles are chosen for review and discussion. They are selected not only for their clinical relevance to general surgeons but also because they cover a spectrum of issues important to surgeons; for example, causation or risk factors for disease, natural history or prognosis of disease, how to quantify disease (measurement issues), diagnostic tests and the early diagnosis of disease, and the effectiveness of treatment. A methodological article is supplied that guides the reader in critical appraisal of the clinical article. Both methodological and clinical reviews of the article are performed by experts in the relevant areas and posted on the EBRS website. As well, a listserv discussion is held where participants can discuss the monthly article. Members of the Canadian Association of General Surgeons and the American College of Surgeons can access Evidence Based Reviews in Surgery through the Canadian Association of General Surgeons website (www.cags-accg.ca) or the American College of Surgeons website (www.facs.org). All journal articles and reviews are available electronically through the EBRS website. We also have a library of past articles and reviews that can be accessed at any time. Surgeons who participate in the monthly packages can obtain Royal College of Physicians and Surgeons of Canada Maintenance of Certification credits and/or continuing medical education credits for the current article only by reading the monthly articles, participating in the listserv discussion, completing the monthly online evaluation and answering the online multiple choice questionnaire. For further information about EBRS, the reader is directed to the CAGS or ACS website or should email the administrator, Marg McKenzie, at mmckenzie{at}mtsinai.on.ca.
In addition to making the reviews available through the CAGS and ACS websites, 4 of the reviews are published in condensed versions in the Canadian Journal of Surgery and 4 in the Journal of the American College of Surgeons each year. We hope readers will find EBRS useful in improving their critical appraisal skills and also in keeping abreast of new developments in general surgery. Comments regarding EBRS may also be directed to mmckenzie{at}mtsinai.on.ca.
Selected Article
Wind J, Polle SW, Fung Kon Jin PHP, et al. Systematic review of enhanced recovery programmes in colonic surgery. Br J Surg 2006; 93:800–9.
Abstract
Objective: To compare fast-track (FT) perioperative programs for elective colonic resections with traditional care. Data source: Studies were identified by searching the Cochrane Library, MEDLINE and EMBASE; as well, relevant journals and conference proceedings and electronic links to related articles were hand-searched, and investigators were contacted. Study selection: Studies were included if they were randomized or clinical controlled trials comparing a multimodal FT perioperative care program with traditional care in patients undergoing elective colon resections for malignant and benign diseases. Outcome measures: Outcome measures were the number of applied FT elements, hospital stay, readmission rate, morbidity and mortality. Results: Six trials containing 512 patients were included. FT programs contained a mean of 9 (range 4–12) of the 17 FT elements. For the outcome primary hospital stay, the results of 3 randomized controlled trials favoured the FT approach (weighted mean difference −1.56 d, 95% confidence interval [CI] −2.61 to −0.50). For the outcome readmission rate, 2 randomized controlled trials had a pooled nonsignificant relative risk (RR) of 0.42 (95% CI 0.14–1.27). Data were pooled from the 6 studies and showed significantly less morbidity in the FT group (RR 0.54, 95% CI 0.42–0.69). In both cases, the small sample size might have influenced the results. For morbidity and mortality, pooled data are not available, but the reviewers suggest that there is a trend toward lower mortality and morbidity in the FT group. In the nonrandomized trials, only 2 of 3 studies reported prim ary hospital stay, and the lack of difference in 1 study makes the weighted mean difference reported in the review reflective of only 1 estimate. Readmission rates were not consistent between studies, which is evident in the wighted mean difference of 1.55 days (95% CI 0.91–2.66). Results for morbidity and mortality are not reported in detail, but the reviewers suggest that there is a trend favouring FT perioperative care. Conclusions: The authors conclude the following:
FT appears to be safe and shortens hospital stay after elective colorectal surgery. However, as the evidence is limited, a multicentre randomized trial seems justified.
Commentary
Wind and colleagues performed a systematic review and meta-analysis comparing FT programs with traditional care in patients undergoing elective colon surgery. This is an important clinical question because of the current interest in FT programs, which have been lauded for their potential reduction in length of hospital stay, faster return to normal activity and reduction in morbidity.
Most surgeons would agree that the fundamental principles within all FT programs are potentially beneficial to patients: early mobilization, faster discharge and faster return to work are certainly universal goals. The challenge of such programs, however, is the organization and multidisciplinary effort required to change practice from traditional care. Both the cost and the institutional buy-in required demand a high level evidence that these programs are effective. However, the variable definition of “fast track” and the paucity of objective outcomes that can be directly attributed to these programs make this field very difficult to study.
Traditional care, which must be considered the current gold standard, has never been assessed in a randomized controlled trial, nor for that matter, has it been defined. Although the switch to FT programs seems intuitive, the desire to rigorously test a change in the standard of care is not without justification: FT programs will not be considered advantageous if found to increase the risk of anastomotic disruption or perioperative cardiac events missed because of overzealous discharge planning. Thus this meta-analysis was performed to combine the available, albeit poor, evidence to support that FT programs are more than just an intuitive improvement on traditional care. The specific objectives were to determine whether there were differences in the duration of the hospital stay, readmission rate, morbidity and mortality in patients who are enrolled in FT programs, compared with those who receive traditional care
Methodologically, this review has some limitations. The details of the search strategy are incompletely reported, making it challenging to reproduce. The authors searched the Cochrane Library, MEDLINE and EMBASE; they also hand-searched other articles identified by electronic links, contacted investigators and hand-searched relevant journals and conference proceedings. The search included English, Dutch and German articles. The Cochrane Non-Randomized Studies Methods Group recommends that a search strategy for meta-analysis of nonrandomized trials be broad to ensure inclusion of all potential studies1; accordingly, it appears that no “publication type” filters were used by the authors of this review. However, neither the explicit MeSH headings used nor the differences in search strategies between different databases were described. An ideal search strategy would have provided enough description to be reproducible so that future readers could repeat the search with the same result.
Having identified relevant studies, the exclusion process used to determine the final included studies was not well described. In fact, there were 44 studies identified, but 35 were excluded because they provided inadequate detail or had a retrospective or uncontrolled study design. Exclusion criteria may lead to some bias, and a flow chart that details the excluded studies would provide more information and is recommended by both the QUORUM guidelines for meta-analysis of randomized controlled trials and the MOOSE guidelines for meta-analysis of observational studies.1,2 Although the authors stated that they used a checklist to assess validity as proposed by the Dutch Cochrane Collaborators, the results were not shown.
Three RCTs and 3 controlled clinical trials were included. The identified studies are heterogenous. As well, 3 of the studies are nonrandomized studies (presumably, prospective cohort studies). Overall, there is a paucity of high-quality data that limits the conclusions that can be made from pooling the results. The authors provide important details about all the studies in tables, which is helpful because it allows readers to assess the studies independently of the pooled results. Further, the authors provide a long list of the FT elements that can be included in an FT program, which is a useful method of operationalizing the reported programs. However, other outcomes, such as morbidity, are very poorly defined, making the pooled estimates of effect challenging to interpret. One could also argue that they should not be pooled because of the strong likelihood of heterogeneity.
The authors report a pooled weighted mean difference for primary hospital stay that combines randomized controlled and nonrandomized controlled data, which may not be appropriate given the methodologic differences between the 2 types of studies. The 95% CI is wide (−2.61 to −0.50) and, given that the units used are days in hospital, the result may not be clinically significant because the lower margin of error is < 1 day (i.e., 0.50 d).
For the pooled RR estimate comparing readmission rates in the 2 groups, the pooled CI includes 1 (0.73–1.86), which means that this result is not statistically significant. This particular pooled estimate provides a good example of the divergent findings that may occur between randomized controlled trials without sufficient power and non-randomized controlled studies. The pooled estimates for each of these groups show a trend toward a different treatment, although neither is statistically significant.
The RRs of morbidity between the 2 treatment arms were pooled. Both randomized and nonrandomized controlled trials favour FT programs (RR 0.67, 95% CI 0.44–1.02) and (RR 0.49, 95% CI 0.3–0.67), respectively; however, the estimates generated by the randomized controlled trial data are not statistically significant. Additionally, these data are difficult to interpret because the variable morbidity is not defined in the included studies.
Other outcomes are reported but are not pooled because of missing data or because they are undefined or have variable definitions. Certainly, this does not suggest that FT programs are not effective, just that the evidence collected to date is insufficient to provide concrete conclusions. Thus, as the authors conclude, further large randomized controlled trials are warranted to assess the value of FT programs.
Footnotes
* The CAGS Evidence Based Reviews in Surgery Group comprises Drs. J.S.T. Barkun, K. Brasel, T.H. Cogbill, C. Suzanne Cutter, G.W.N. Fitzgerald, H.J.A. Henteleff, A.W. Kirkpatrick, S. Latosinsky, A. Maclean, T.M. Mastracci, R.S. McLeod, L.A. Neumayer, S. Smith, L.R. Temple and Ms. M.E. McKenzie.
Competing interests: None declared.