Abstract
Objective: The aim of this survey was to determine Canadian vascular surgeons’ experience with elective endovascular aortic repair (EVAR) and traditional open repair and their interest in participating in an expertise-based randomized controlled trial (RCT) as opposed to a conventional RCT comparing these 2 procedures.
Methods: A single-page questionnaire was developed and sent by fax, email or post to all vascular surgeons in Canada. Nonresponders were recontacted on 2 additional occasions to improve the response rate. The questionnaire had 2 sections. The first inquired about current and past practice patterns, including experience in both open and endovascular techniques. The second investigated the surgeons’ belief in the value of open as opposed to endovascular repair and the value of expertise-based RCT methodology; it also canvassed their interest in participating in a future trial. Definitions of expertise in open and endovascular repair were drawn from the published literature. Criteria to determine the feasibility of conducting an expertise-based RCT were established a priori.
Results: The questionnaire was sent to 259 surgeons who appeared in multiple vascular surgery databases, and the overall response rate was 56% (95% confidence interval [CI] 50%–62%). The mean career experience was 406 cases (standard deviation [SD] 359) for conventional open abdominal aortic aneurysm (AAA) repair and 24 cases (SD 48) for endovascular repair. Of the responding surgeons, 51% (95% CI 41%–60%) ranked conventional open repair as “probably superior.” Respondents were equally interested in participating in an RCT using either expertise-based methodology (54%, 95% CI 44%–63%) or conventional design (51%, 95% CI 41%–60%).
Conclusion: Uncertainty exists among vascular surgeons in Canada as to the role of endovascular surgery in the repair of AAA. A national RCT comparing open with endovascular repair in the elective setting is potentially feasible with either expertise-based or conventional design. Increases in the number of surgeons who are willing to participate and have expertise in EVAR, in addition to high recruitment rates among eligible patients, will be necessary to make such a trial feasible in Canada.
In randomized controlled trials (RCTs) that screen for abdominal aortic aneurysm (AAA) in Europe and Australia, the prevalence of AAA is 6% (95% confidence interval [CI] 5%–6%) in men over age 65 years1–4 and 1% (95% CI 1%–2%) in women over age 65 years.5 The most important predictor of the annual risk of rupture is the diameter of the aneurysm. In a population-based study, the annual risk of rupture was 1% (95% CI 0%–5%) for aneurysms with a diameter of 4.0–4.9 cm and increased to 11% (95% CI 1%–21%) for aneurysms of 5.0–5.9 cm.6 Prevention of spontaneous rupture and death is the rationale for prophylactic operative intervention in people with AAA.
Two treatment modalities are available: conventional open repair and endovascular aortic repair (EVAR). The first requires major surgery with a laparotomy, resection of the diseased aorta, and admission to a monitored bed with an average stay in hospital of 12 or 13 days.7,8 A tertiary referral centre cohort study of elective open repair of AAAs reported a 30-day perioperative mortality rate of 1.2% (95% CI 0.7%–2.1%); however, most studies have reported mortality rates of 5%–8%.9–18 The 5-year postprocedural survival rate for open repair is 60%.19
Endovascular repair of AAAs involves percutaneous femoral artery access to place a graft that excludes the aneurysm from circulatory flow. This is a minimally invasive procedure, and patients rarely require admission to a monitored setting. Two prospective nonrandomized registries show that EVAR is clinically feasible. 20,21 Two randomized trials, the DREAM trial8 conducted in The Netherlands and the EVAR trial conducted in Great Britain, reported improved perioperative mortality with EVAR in comparison with open repair (pooled relative risk [RR] 0.34, 95% CI 0.17–0.67, p = 0.002). The EVAR 1 trial22 showed a significant difference in aneurysm-related mortality (hazard ratio 0.55, 95% CI 0.31–0.96, p = 0.04), but neither study has shown a difference in long-term mortality between the 2 groups (pooled RR 0.94, 95% CI 0.75–1.18, p = 0.59).7,8,22,23 Therefore, the available data have failed to demonstrate the long-term superiority of one technique over the other in the treatment of AAA.
We (P.J.D., C.S.C.) have previously reported how conventional RCT methodology can bias the results of surgical trials, citing differential expertise, differential intervention crossover rates and differential cointerventions as possible disadvantages. 24 An alternative design, the expertise-based RCT, has the potential to reduce these biases. We have recommended that investigators consider using this design when planning clinical trials to evaluate surgical techniques.
The aim of this survey was to determine Canadian vascular surgeons’ experience with both elective EVAR and traditional open repair as well as their interest in participating in an expertise-based RCT relative to a conventional RCT comparing these 2 procedures. Such a trial would add to the literature in 2 domains: it would establish whether endovascular therapy for AAA is both an effective and a cost-effective treatment, and in addition, such a trial would provide evidence of the utility of expertise-based RCT methodology in surgical trials.
Methods
A list of Canadian vascular surgeons was compiled from the Canadian Society of Vascular Surgeons’ website, the Royal College of Physicians and Surgeons Directory of Fellows and personal records of a senior vascular surgeon (C.S.C.). All candidates on the list were contacted to determine their practice status (active or nonactive), and a survey was administered by fax or email. The preferred method of contact for each participant was obtained and used for all correspondence.
To maximize the response rate, the questionnaire was short (7 questions on 1 page) and was sent with a personalized cover letter describing the study’s purpose. We estimated that completion of the survey would require about 5 minutes. The questionnaire was sent at 4-week intervals from June to August 2005, up to 3 times to each surgeon, unless a response was obtained.
We used the theoretical constructs and opinions of content experts in both measurement scales and vascular surgery to develop the items on the questionnaire, which has 2 sections. The first inquires about current and past practice patterns, including experience in both open and endovascular techniques. This section requires respondents to fill in fields that enumerate the number of procedures done in the past year and during their career (hereafter called annual volume and career volume, respectively) as well as the number of years since the completion of their vascular training. The second section investigates the surgeon’s belief in the value of open versus endovascular repair and his or her willingness to participate in an expertise-based as opposed to a conventional RCT. This section uses Likert scales and yes/no answers (Appendix 1). Scales were designed with 7 levels of discrimination to optimize the questionnaire’s psychometric properties.25
The survey was designed to determine the national interest in conducting an expertise-based RCT, as opposed to a conventional RCT, comparing open with endovascular repair of AAA in patients who are not at high risk for morbidity and mortality. Three criteria were defined a priori as necessary to establish the feasibility of conducting an expertise-based trial: uncertainty among vascular surgeons regarding the value of open versus endovascular repair of AAA, a majority of surgeons who favour or are neutral to the idea of an expertise-based trial design, and a group of surgeons who meet the definition of having expertise in endovascular surgery as well as a group who meet the definition of having expertise in open surgery, with a geographic distribution representative of Canadian vascular practices. We defined surgeons with expertise in EVAR as those with a career volume of ≥ 60 cases (a threshold derived from a cumulative sum analysis that defined the volume of endovascular cases required to achieve a complication rate of less than 10%26). We defined surgeons with expertise in open repair as those with a career volume of ≥ 100 procedures (a number derived from a cumulative sum analysis of open elective repair of AAA that found 100 AAA surgeries were required to allow a surgeon to have a complication rate of less than 10%27). Data for both career volume and annual volume were collected to assist with trial planning and to document clinical activity.
We further assessed the feasibility of a national expertise-based trial as follows. First, we identified eligible centres. These we defined as centres with 1 or more surgeons who were willing to participate in an expertise-based trial and who had expertise in EVAR, along with 1 or more additional surgeons who were willing to participate in an expertise-based trial and who had expertise in open repair. We assessed the probable annual volume eligible for an RCT at each eligible centre, using the formula: estimated annual eligible patients = annual EVAR volume for willing surgeons + 50% annual open volume for willing surgeons.
It has been estimated that 55% of all patients with AAA are eligible for EVAR28; we used a conservative estimate to take into account those patients with AAA who were considered for EVAR but deemed ineligible and to avoid overestimating the feasibility of a trial.
To assess the feasibility of a conventional RCT, we identified eligible centres. These we defined as centres with 1 or more surgeons who were willing to participate in a conventional trial and who had expertise in both EVAR and open repair. The annual number of patients eligible for a conventional randomized trial was estimated as for an expertise-based trial.
Data were tabulated electronically (Microsoft Excel 2003, Microsoft Corporation, 2003) and statistics were calculated with SPSS 12.0 (SPSS Inc., Chicago, Ill., 2003). We acquired descriptive statistics for all respondents and for the subgroups of respondents who met the threshold criteria for expertise in endovascular and open techniques. We used a Spearman correlation when calculating the correlation between career volumes and preference for open or endovascular AAA repair. When there was a missing response to an item, we excluded the respondent from the analysis of that particular item.
Results
The questionnaire was sent to 259 surgeons identified as practising vascular surgery in Canada. The first mailing yielded 111 (43%) responses, the second 22 (8%) and the final 11 (4%). The overall response rate was 56%. Respondents included 43 inactive or nonpractising surgeons, who were excluded from further analyses, and 101 practising surgeons. The average time in practice was 14.3 years (standard deviation [SD] 9.0, minimum 0.5, maximum 35 y), and 57% (95% CI 48%–67%) used EVAR in their practice. The mean number of open AAA repairs performed in the previous year was 34 (SD 34, median 30, minimum 0, maximum 300), and the mean number of endovascular AAA repairs was 7 (SD 12, median 1, minimum 0, maximum 65). The mean career volume of AAA repairs was 406 (SD 359, median 300, minimum 0, maximum 1600), and the mean career volume of endovascular procedures was 24 (SD 48, median 3, minimum 0, maximum 310). According to the predetermined definitions of expertise, 80 respondents met criteria for expertise in open repair, and 13 met criteria for expertise in endovascular repair.
Among all surgeons, 51% (95% CI 41%–60%) ranked open repair as “probably superior,” 16% (95% CI 9%–23%) ranked EVAR as “probably superior,” and 34% (95% CI 24%–43%) were uncertain (Fig. 1); 51% (95% CI 41%–60%) were willing to participate in a conventional RCT, 44% (95% CI 34%–53%) were not, and 8% (95% CI 3%–13%) were unsure; 54% (95% CI 44%–63%) were willing to participate in an expertise-based trial, 41% (95% CI 32%–51%) were not, and 7% (95% CI 2%–12%) were unsure (Fig. 2). The preferred study design according to a 7-point Likert scale is represented in Figure 3.
There were 13 respondents who met the criteria for expertise in EVAR, of whom 5 ranked EVAR as “definitely” or “probably superior,” 3 were uncertain, and 5 ranked open repair as “definitely” or “probably superior.” Eleven of the 13 respondents with expertise in EVAR were willing to participate in a conventional RCT, and 7 of the 13 indicated interest in participating in an expertise-based RCT.
There were 80 respondents who met the criteria for expertise in open AAA repair, of whom 11 ranked EVAR as “definitely” or “probably superior,” 25 were uncertain, and 44 ranked open repair as “probably,” “definitely” or “definitely markedly superior.” Forty of those with expertise in open AAA repair were willing to participate in a conventional RCT, and 44 were willing to participate in an expertise-based RCT.
Among surgeons with high-volume endovascular experience, there was a nonsignificant trend favouring EVAR (Spearman correlation, p = 0.140). Surgeons with high-volume open repair experience showed no trend in favouring either open or endovascular repair (Spearman correlation, p = 0.457).
Table 1 shows the number of centres and patients eligible for enrolment into either a conventional or expertise-based RCT, according to the reported volumes.
Discussion
Our survey demonstrates that more than one-half (57%) of practising Canadian surgeons use EVAR in their practice. As expected with an emerging technology, this represents an increase when compared with the 40% identified by McAuley and colleagues in a similar survey conducted in 2002.29 In their work, 52% of respondents (95% CI 42%–62%) were uncertain about the benefit of EVAR. Similarly, among the respondents to our survey, 51% ranked open repair as “probably superior.” In Canada, however, our results show uncertainty regarding the best technique for AAA repair along with interest in a future RCT. Therefore, Canadian surgeons are ideally positioned to conduct an RCT in this field.
Two RCTs have compared open with endovascular repair of AAAs. The DREAM trial8,23 reported an initial perioperative advantage for endovascular repair, compared with open repair (1.2% v. 4.6%), which was no longer evident at 2 years. The limitations of this trial include early trial termination before randomization of the predetermined sample size; potential compromise of the randomization concealment owing to the use of fixed blocks in an unblinded trial; and a low number of outcome events. The EVAR-1 trial7,22 had a moderate sample size and demonstrated an early, significant perioperative mortality advantage for EVAR (1.7% v. 4.7%). However, all-cause mortality did not differ between the 2 treatment groups at 4 years.22 The expertise of the surgeons involved is a source of bias in the 2 published RCTs. To participate in the DREAM trial, surgeons were only required to have performed 5 EVAR procedures and to be supervised by a proctor for a total of 20 endovascular procedures. Similarly, in the EVAR-1 trial, a centre could participate if the surgical team had performed only 20 procedures. The expertise of any given surgeon is unclear.
Endovascular expertise has been quantified in the literature by using procedural volumes. A cumulative sum analysis of surgical experience in endovascular repair has found that, at minimum, 60 cases are required to achieve a 10% complication rate.26 In another publication, Lobato and colleagues30 used a Cox regression analysis to determine factors predictive of technical success and found that a procedural volume of 55 cases and a frequency of less than 10 days between procedures is necessary to maintain competence and reduce complications.30 Although these threshold volumes provide a bench-mark, they represent only a baseline definition of competence because it is possible that a surgeon with more expertise may achieve a major complication rate of less than 10%. Defining surgical expertise is likely more complex than the number of cases done per year or their frequency would suggest,31,32 but practical and functional measures are necessary to operationalize expertise in the trial setting.
Expertise-based RCTs control for differential expertise between surgeons performing different procedures and likely reduce crossovers and biases related to sugeons’ preferences for a particular procedure.24 Differential expertise bias is avoided because surgeons in both arms are chosen for their expertise and belief in the given procedure. Crossovers may be reduced because each surgeon will perform only one or the other procedure after randomization and may not have the choice of doing either in their armamentarium. In an expertise-based RCT of EVAR versus traditional open repair, investigators would randomize patients to receive surgery from a surgeon with expertise in EVAR who is committed to performing only endovascular repair or to a surgeon with expertise in open repair who is committed to performing only open repair. Surgeons with expertise in the relevant technique would treat the patients in each arm of the trial, and therefore, the results of an expertise-based RCT are more likely than a conventional RCT to provide an unbiased assessment of each treatment. Although the concept of an expertise-based trial was introduced in 1980,33 this methodology has rarely been used in surgical trials. In our study, 45% of respondents preferred a conventional RCT, and 55% of respondents were ambivalent or preferred an expertise-based trial. Various factors might have influenced this response. The materials sent to the respondents provided only a brief explanation of the difference between the 2 methodologies, without extensive justification. Unfamiliarity with the expertise-based design and with the arguments for its use might have negatively affected the views of some respondents. Alternatively, awareness of our group’s interest in expertise-based trials (which was disclosed in the covering letter) might have biased results toward this methodology.
We have identified 13 Canadian vascular surgeons, distributed through-out the country, who have expertise in EVAR according to our definition. In addition, it is likely that this number will increase in the next few years as the use of endovascular techniques becomes more widespread. Our results suggest that there is sufficient expertise in Canada to conduct an expertise-based RCT.
Using the responses from this survey of Canadian vascular surgeons, we have estimated the volume of annual eligible patients available to participate in an RCT (Table 1 and Table 2). We have found that 3 centres meet criteria for involvement in an expertise-based RCT, with a total annual eligible patient volume of 211 patients. For a conventional RCT, 7 centres meet criteria for involvement, with an annual eligible patient volume of 489 patients. These estimates must be interpreted with caution because they are based on responses from only 46% of practising vascular surgeons and reflect practice patterns in 2005 in an evolving field. All surgeons from eligible centres did not respond, and therefore, patient volumes may be under-estimated. Finally, the concrete request for a commitment to participate might have yielded fewer positive responses than a theoretical question.
To assess the feasibility of a future trial, it is necessary to estimate the sample size. If perioperative mortality is used as a primary outcome, then a trial that has a perioperative mortality of 5% in open repair and 2% in endovascular repair would need a sample size of 1176 patients to yield the 80% power needed to show a significance of 0.05%. At 100% eligibility and 100% patient agreement, these data suggest that this trial would require 6 years of recruitment if conducted with an expertise-based design and 3 years of recruitment with a conventional design. If long-term aneurysm-related mortality is used as a primary outcome, and a 7% event rate is anticipated in the open repair group, compared with a 4% event rate in the endovascular repair group, a significant difference would be found with 80% power if 1816 patients were randomized; this would require 8 years of recruitment with an expertise-based design and 4 years of recruitment with a conventional design. According to our data, the duration of the trial would be longer with the expertise-based design. These data do not take into account the evolution of endovascular techniques over the course of the trial and the likelihood that more surgeons with expertise will exist in the future. Regardless of this, measures should be taken during the planning of the trial to promote the need for a trial and to increase information about the validity advantages of expertise-based design.
A limitation of this study is the use of a self-reported survey with several potential biases, including recall and social desirability bias.34 The use of quantitative data without a mechanism for verification (e.g., formal chart review of each surgeon’s practice) limits the inferences that we may draw from the results. However, we have found other examples of this type of survey that form the basis of feasibility assessment for RCTs.35 The response rate of 56% is in keeping with studies of this type. For example, Asch and colleagues36 found that the average response rate to mail surveys sent to physicians was 54% (SD 17%).
Conclusion
Uncertainty exists among vascular surgeons in Canada as to the role of endovascular surgery in the repair of AAA. A national RCT comparing open with endovascular repair in the elective setting is potentially feasible with either expertise-based or conventional design. Increases in the number of surgeons with expertise in EVAR and willingness to participate, in addition to high recruitment rates among eligible patients, will be necessary to make such a trial feasible within Canada.
Acknowledgements
Many thanks to Chris Coroneos for his work faxing and collating questionnaires.
Appendix 1
Footnotes
Competing interests: Dr. Mastracci has received travel assistance to attend meetings from Gore Medical and Cook Inc. Dr. Cinà is a consultant for, and has received fees and travel assistance from, Cook Inc., Vascutek and Sigvaris.
Contributors: Drs. Mastracci, Deveraux and Cinà designed the study. Dr. Mastracci acquired the data, which all authors analyzed. Dr. Mastracci wrote the article, and all authors revised it. All authors gave final approval for the article to be published.
- Accepted February 2, 2007.