Developing priority criteria for general surgery: results from the Western Canada Waiting List Project
======================================================================================================

* Mark C. Taylor
* David C. Hadorn
* the Steering Committee of the Western Canada Waiting List Project

## Abstract

**Objective:** The Western Canada Waiting List Project (WCWL) is a federally funded partnership of 19 organizations, including medical associations, health authorities, ministries of health and research organizations that was created to develop tools for improving the management of waiting lists. The WCWL general surgery panel was 1 of 5 panels constituted under this project.

**Method:** The panel developed and tested a set of standardized clinical criteria for setting priorities among patients awaiting elective general surgery of all kinds. The criteria were applied to 561 patients in 3 western provinces. Regression analysis was used to determine the set of criteria weights that collectively best predicted clinicians’ overall urgency ratings.

**Results:** The priority criteria accounted for almost two-thirds of the observed variance in clinicians’ urgency ratings (adjusted *R**2* = 64.1%) for a mixed group of patients. The panel modified the criteria and weights based on empiric findings and clinical judgment. Interrater and test-retest reliability of criteria items appeared to be good, based on clinicians’ ratings of 6 videotaped, standardized patient interviews.

**Conclusions:** The panel considered the criteria easy to use and reasonably reflective of expert surgical judgement regarding clinical urgency. Further development and testing of the tool appears warranted.

Waiting lists for health care services are common in all publicly funded national health services. They are a source of public distress and political consternation, in large part because they entail extended suffering, disability and, occasionally, death for patients on those lists. The prevalent impression that waiting lists may not be fair1,2 exacerbates this situation. A growing body of evidence indicates that patients’ chances of receiving needed services in a timely manner, based solely on clinical urgency, are uncertain. A recent report concluded that the management of waiting lists across Canada is generally “chaotic,” as well as “non-standardized, capriciously organized, poorly monitored, and…in grave need of retooling.” The authors concluded that it is “impossible to…rationally manage the patients on those lists”;3 therefore it is impossible to guarantee fairness.

Similar concerns underlie doctors’ participation in New Zealand in developing point-count priority criteria for assessing urgency for a variety of clinical conditions. Clinicians “almost universally acknowledged that decisions about urgency and priority were made inconsistently. Often, the ‘squeaky wheel would get the grease,’ and more deserving but uncomplaining patients would be disadvantaged.”4

Most systems currently used to categorize patients by urgency for general surgery are based on the time-honoured method of “emergency, urgent, semiurgent, or routine.” Such broad classification systems are highly subjective and inadequate to assess and compare urgency and case-mix of patients on waiting lists.

## Priority criteria

In response to the need for better management of waiting lists, an increasing number of clinicians and health authorities are adopting point-count measures for assessing patients’ relative clinical urgency or priority.5 Similar point-count measures are used in many settings throughout medicine to assess risk of adverse events and severity of illness, including assessment of neonatal stability (Apgar score) and estimating the probability of dying in intensive care units (e.g., APACHE score). Such measures function as additive or linear models from a statistical viewpoint.6

Similarly, priority criteria estimate severity of illness as an indicator of urgency, although additional considerations are often included as criteria, including such social and role factors as whether patients’ illnesses are interfering with their ability to work, to care for dependants or to live independently. The principal functions of priority criteria are: (1) to guide decisions about the relative urgency and order of surgery among patients on waiting lists and (2) to develop case-mix descriptions of patients on waiting lists. These descriptions can be used to assess and compare waiting lists across regions and over time.

Initial experience with priority criteria in New Zealand, Sweden, England, Northern Ireland and Wales was reviewed in a recent report prepared by the Health Policy and Economic Research Unit of the British Medical Association.7 The report endorsed this approach:

> The widespread introduction, in this country [England] of some form of priority scoring system for surgical waiting lists would have benefits, e.g. greater transparency for all (patients, managers, politicians and doctors) on the decision taken to offer surgery to a patient; in the long term a system that is equitable across the whole country; provision of service led by clinical need and in the control of clinicians.

Little information has been published concerning the clinical validity of priority criteria. In New Zealand, regression analysis was used to generate weights for sets of criteria, based on a comparison with overall clinical judgment.8 However, the number of patients included in most of these analyses was relatively small (e.g., 97 cataract patients). Only coronary artery bypass graft surgery, with 260 patients, assessed a sufficient number to permit calculation of stable criteria weights.

Most previous studies of general surgery waiting lists have focused either on numbers of patients on waiting lists or on waiting times.9–13 In New Zealand, a panel of surgeons and general practitioners developed criteria for assessing the urgency of patients in need of cholecystectomy. These criteria were later expanded in certain regions (including Auckland, New Zealand’s largest city) into sets of generic criteria covering all general surgical procedures. In assessing these criteria, Dennett and colleagues at Auckland Hospital found poor correlation between the clinical judgment ratings obtained on a visual analogue scale (VAS) of overall urgency and the scores generated by priority criteria for cholecystectomy14 and for general surgical procedures.15

Little or no effort has been made to assess the intra- or interobserver reliability of priority criteria: the extent to which raters arrive at the same (or similar) ratings using the criteria when evaluating the same (or similar) patients. A study conducted in the mid-Hampshire region of England found good agreement between the priority scores assigned by general practitioners and specialists to patients with hip or knee arthritis or cataracts (unpublished data). Such assessments, which are essential to the interpretation of studies of this kind, are extremely rare.

## The Western Canada Waiting List Project

The Western Canada Waiting List Project (WCWL) was established in 1998 to address some of the problems in waiting list management identified in the report cited above.3 In particular, the project focused on developing, testing, and refining clinical measures capable of assessing and comparing the relative urgency of patients on waiting lists.

WCWL is a collaborative undertaking by 19 partner organizations: 7 regional health authorities, 4 medical associations, 4 provincial ministries of health, and 4 health research centres. Clinical panels consisting of specialists, family physicians and other relevant health care providers were constituted to address each of 5 areas: cataract surgery; general surgery; hip and knee replacement; magnetic resonance imaging; and children’s mental health services. In this paper we describe the experience of the general surgery panel.

## Methods

The general surgery panel comprised 7 academic and community general surgeons, 2 family physicians, and representatives from health research and administration. Panelists were drawn from the provinces of Manitoba, Saskatchewan and Alberta. The members of the panel decided to incorporate under its mandate all adult general surgery, defined as those procedures done in the operating room. The rationale for this decision was that surgeons currently prioritize across the entire range of conditions and procedures, and if the criteria were to be useful they should facilitate such cross-comparison.

The initial step in the process was to identify the clinical and social factors considered (informally) by surgeons when prioritizing patients on waiting lists. Summaries of the literature were prepared addressing inguinal hernia, cholecystectomy, breast cancer and colorectal cancer, to assist the panel in selecting priority criteria. The panel elected to adopt the New Zealand generic general surgical criteria as a starting point (Table 1, column 1). These criteria were incorporated into a questionnaire that was used to score a series of patients by 7 surgeon-panelists and by 6 general surgeons not associated with the project. The pilot testing data collection took place in Winnipeg, Regina and Edmonton, between November 1999 and May 2000.

View this table:
[Table 1](http://canjsurg.ca/content/45/5/351/T1)

Table 1 
Summary of the Criteria and Score Development for the Western Canada Waiting List Project’s General Surgery Priority Form

### Determining priority coefficients

Participating clinicians assigned each patient to the appropriate level on each criterion (e.g., mild pain, moderate limits in role function) and rated the overall urgency of each patient on a 10-cm VAS. The latter served as the dependent variable in the regression analyses, which were used to determine the statistically optimal set of weights on each criterion, to best predict (or to correlate with) overall urgency. The optimal scaling method of the SPSS statistical software (SPSS Version 10 for Windows, SPSS Inc., 1999) was used for initial data transformation in this process.

Standard linear regression analysis was then conducted, constrained to retain all predictor variables (criteria) regardless of cross-correlations among criteria.* The calculated item weights were based on t-values from the linear regression, with weights distributed across criteria as a proportion of the total model t-value. Weights for levels within each criterion item were determined by multiplying rescaled item values (obtained from optimal scaling transformations) by the weight for that item. The extent of correlations among criteria was measured using Pearson r statistics.

An interim analysis was presented to the panel at its second meeting in January 2000. At this time, panelists agreed that the priority criteria form was easy to use and eemed reasonably reflective of how surgeons view their patients. Some concern was expressed that the 2 items on role and social functioning might be assessing the same domain. However, no changes were made in the criteria at that time.

### Determining reliability of priority criteria

Preliminary work to evaluate the reliability of the priority criteria was undertaken in January 2000. One of the authors (M.C.T.) developed 6 hypothetical patient scenarios, which were reviewed by other members of the research team. These simulated patient encounters were videotaped using actors experienced as standardized patients, with M.C.T. serving as interviewer. The cases were reviewed and scored independently by 13 raters, including 9 surgeons, from 3 western Canadian provinces. Inter-rater reliability was assessed for each of the criteria items and the VAS urgency rating, with the intraclass correlation coefficient (ICC) and the κ statistic as the primary measures of agreement. In addition, at its final meeting, the panel re-scored 2 of the scenarios and discussed differences in ratings to identify sources of disagreement, including ambiguity or lack of clarity in the instrument.

A final panel meeting was held in June 2000, at which the results of the initial pilot testing and reliability work were presented. After extensive discussions, panelists modified the original criteria and regression weights to improve clinical utility and face validity. Weights were apportioned among items so that the total maximum achievable score, summing across the most severe response category for each criterion, was 100 points.

Further empirical work was conducted to evaluate the interrater and test–retest (intrarater) reliability of the revised priority criteria. The methods were similar to those described above for the preliminary reliability assessment; the scenarios were identical, but the presentation order was changed for each wave of data collection to control for order effects. The first wave of reliability data was collected in November 2000 and the second wave in January 2001, with test–retest data obtained from 12 raters, including 10 surgeons.

## Results

For the pilot testing, 561 patients were scored using the initial criteria (Table 1). A regression analysis was performed, based on data from a subset of 237 patients who had complete information on all 8 criteria items. The adjusted coefficient of determination for multivariate analysis (*R**2*) for this analysis, using the VAS urgency score as the dependent variable, was 64.1%. The 8 criteria thus explained almost two-thirds of the observed variation in clinicians’ overall urgency ratings for a mixed group of patients.

Univariate correlations among criteria items are presented in Table 2. High correlations were observed between items 4 and 5 (usual degree of impairment in role function and in social activities, respectively); between items 1 and 3 (usual intensity and frequency of pain); and between items 3 and 4 (frequency of pain and impairment in role function).

View this table:
[Table 2](http://canjsurg.ca/content/45/5/351/T2)

Table 2 
Correlation Matrix for the General Surgery Criteria Using Pilot Testing Data From 238 Patients

The criteria weights (scores) resulting from regression analysis are presented in Table 1. At its final meeting, the panel made a few organizational and wording changes in the criteria, including collapsing items 4 and 5 together and modifying response choices (Table 1, last column). Modest changes were made to the regression-based weights, taking into account clinical face validity and modifications to the content of the instrument. The greatest shift was for “history of major complications” (increase in maximum number of points from 4 to 8). “Usual intensity of other forms of suffering” was reduced from a maximum weight of 15 to 12. Rebalancing for the remaining items required increasing or decreasing 1 or 2 points for the most severe response categories. In addition, the panel decided that the 2 life expectancy items (7 and 8) should be applied to all patients and not be restricted to cancer patients as was the case in the initial criteria development. Finally, it was agreed that a set of operational definitions and instructions would be prepared to accompany the criteria.

Results of the reliability assessment of the revised priority criteria (Table 3) found excellent interrater agreement for the VAS urgency ratings for the 6 standardized patients (ICC = 0.83). Three criteria had excellent reliability (ICC > 0.75); 3 items had fair to good reliability and 1 item had poor reliability (ICC < 0.40). The lowest reliability was observed for item 5 (recent history of major complications or significant examination or test results). test–retest reliability over a 2-month interval was also assessed, based on input from 10 surgeons who participated in both waves of data collection. Relatively good intrarater consistency in scoring was found for the majority of criteria items. The visual analogue rating of urgency had an excellent test–retest ICC value of 0.92.

View this table:
[Table 3](http://canjsurg.ca/content/45/5/351/T3)

Table 3 
Interrater and Test–Retest Reliability of the Revised General Surgery Priority Criteria by 10 Surgeons

## Discussion

General surgeons from the western provinces in Canada accepted and endorsed the ability of clinical priority criteria to reflect global expert judgments of urgency. Based on discussions at panel meetings, participants considered the criteria to have reasonable face validity and to be easy to use. The high correlations observed between related criteria, as described above, make good clinical sense, further supporting the clinical validity of the criteria.

The panel decided against developing separate sets of criteria for cancer and noncancer patients, but initially included 2 additional life-expectancy items for cancer patients. This issue arose because, at times, it seemed difficult to balance the impact of life-expectancy considerations in cancer diagnoses against the suffering and disability that are often more prominent in noncancer conditions. However, the panel recognized that, in practice, surgeons must assess and compare all patients against each other, in large part because the same operating room resources are at stake. Ultimately, the panelists decided to use a single set of criteria with 8 items for all patients, which seemed to produce reasonable orderings of patients, based on panelists’ clinical judgements. Because the *R*2 value is quite respectable, it seems reasonable to conclude that when used in a mixed population the criteria should be able to distinguish among patients with different levels of urgency.

The reliability results suggest that clinicians using the general surgery priority criteria can achieve reasonably good interrater agreement and good intrarater stability in scoring over time. The creation and use of 6 videotaped simulated patient interviews provided an excellent source of standard material for this purpose. No special effort was made to standardize the rating process, e.g., by providing examples of patients conforming to “mild suffering,” etc. or to provide specific definitions of the various levels on each criterion. As such, the observed results represent a “worst case” scenario, which can almost certainly be improved upon with practice and clarification of terms. Further refinement of item 5 (recent history of major complications) would potentially enhance the overall reliability of the instrument.

A number of outstanding questions remain. It has not yet been demonstrated in any definitive way that the weighted scores will actually rank patients in the appropriate order based on urgency. Ideally, such demonstration would involve following patients over time and comparing health outcomes and length of time waiting. When such studies are performed, measures similar to the priority criteria reported here may be suitable for capturing health outcomes, such as the impact of treatment on pain and function.

A number of operational challenges can be foreseen with the use of priority criteria for scheduling of surgery. For example, patients with minimally symptomatic benign conditions, such as hernias, will always score lower than patients with symptomatic, serious conditions. As new, high-scoring patients are seen, the low scoring patients will never reach the top of the list. It is possible to address this problem by adding points to the scores of patients simply for time spent waiting. However, this could lead to a different problem: patients with less severe conditions regularly “bumping” patients with more severe conditions. It was for this reason that all WCWL panels decided against incorporating time for waiting into the criteria.

Another concern, raised regularly during the project was the possibility that patients and clinicians would “game the system” by virtue of knowing how the point system works. These concerns need to be addressed through careful monitoring, use of standard raters or other techniques. However, the current chaotic and unregulated system can in most areas be easily gamed and is not subject to audit.

It is hoped that further development will lead to an instrument that can be widely used for prioritization and case-mix description of patients on waiting lists for general surgery. It is imperative that such an instrument be developed in order to permit assessment and accountability — and, ultimately, fairness — in the context of general surgery waiting lists.

## Acknowledgements

Members of the Steering Committee of the Western Canadian Waiting List Project are as follows: Dr. Tom Noseworthy, Department of Public Health Sciences, University of Alberta, Edmonton (Chair); Dr. Morris L. Barer, Centre for Health Services and Policy Research, and Department of Health Care and Epidemiology, University of British Columbia, Vancouver; Dr. Charlyn Black, Manitoba Centre for Health Policy and Evaluation, and Department of Community Health Sciences, University of Manitoba, Winnipeg; Ms. Lauren Donnelly, Acute and Emergency Services Branch, Saskatchewan Health, Regina; Dr. David Hadorn, Western Canada Waiting List Project; Dr. Isra Levy, Health Programs, Canadian Medical Association, Ottawa; Mr. Steven Lewis, Access Consulting, Saskatoon; Mr. John McGurran, Department of Public Health Sciences, University of Toronto, Toronto; Dr. Sam Sheps, Department of Health Care and Epidemiology, University of British Columbia, Vancouver; Dr. Mark C. Taylor, Department of Surgery, University of Manitoba, Winnipeg; Mr. Laurence Thompson, Health Services Utilization and Research Commission, Saskatoon; Mr. Darrell Thomson, Economics and Policy Analysis, British Columbia Medical Association, Vancouver; Ms. Barbara Young, Clinical Evaluation Services, Calgary Regional Health Authority, Calgary.

The Western Canada Waiting List Project was supported by a financial contribution from the Health Transition Fund (Health Canada) as Project NA489. The views expressed herein do not necessarily represent the official policy of federal, provincial or territorial governments.

We are indebted to the following 19 partner organizations for their ongoing support throughout the project: British Columbia Medical Association; Capital Health Region (Victoria); Vancouver/Richmond Health Board; British Columbia Ministry of Health; University of British Columbia, Centre for Health Services and Policy Research; Alberta Medical Association; Capital Health Authority (Edmonton); Calgary Regional Health Authority; Alberta Health and Wellness; University of Alberta, Department of Public Health Sciences; Saskatchewan Medical Association; Regina Health District; Saskatoon District Health; Saskatchewan Health; Health Services Utilization and Research Commission; Winnipeg Regional Health Authority; Manitoba Health; Manitoba Centre for Health Policy and Evaluation; Canadian Medical Association.

We acknowledge the members of the general surgery panel who contributed to the development of the general surgery priority criteria tool: Dr. Andrew Cave, Ms. Carolyn DeCoster, Dr. C.J. de Gara, Dr. Neil Hagerman, Dr. Tim Hilderman, Ms. Patricia Hosang, Dr. Elissa Hunter, Dr. Jeremy Lipschitz, Dr. Duncan Scurrah, and Dr. Garth Warnock, and we thank colleagues of the panel members who participated in the pilot testing and reliability work in Winnipeg, Regina, and Edmonton. We also thank Dr. N.G.N. Prasad, Dr. Barbara Conner-Spady, Ms. Elaine Dunn, Ms. Helen Roman-Smith, Ms. Anne-Marie Pedersen, and Ms. Katerina Carastathis for their contributions to data collection and analysis, and document preparation.

## Footnotes

*   * In standard regression, predictor variables are often dropped if they are significantly correlated with other, more highly predictive variables. However, panelists wished to retain all criteria in order to ensure adequate face validity, even where significant correlations did exist among criteria. Thus, for example, it would probably be unacceptable from a clinical point of view to remove explicit consideration of the extent of pain from the questionnaire, even if this factor tended to correlate with (or to be “captured by”) scores on other items.

*   Accepted February 8, 2002.

## References

1.  Lewis S, Barer ML, Sanmartin C, Sheps S, Shortt SE, McDonald PW. Ending waiting-list mismanagement: principles and practice. CMAJ 2000;162:1297–300.
    
    [FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTYyLzkvMTI5NyI7czo0OiJhdG9tIjtzOjE4OiIvY2pzLzQ1LzUvMzUxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Sanmartin C, Shortt SE, Barer ML, Sheps S, Lewis S, McDonald PW. Waiting for medical services in Canada: lots of heat, but little light. CMAJ 2000;162:1305–10.
    
    [FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTYyLzkvMTMwNSI7czo0OiJhdG9tIjtzOjE4OiIvY2pzLzQ1LzUvMzUxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  McDonald P, Shortt S, Sanmartin C, Barer M, Lewis S, Sheps S. Waiting lists and waiting times for health care in Canada: more management!! more money?? Ottawa: Health Canada; 1998.
    
    

4.  Hadorn DC, Holmes AC. The New Zealand Priority Criteria Project: I: overview BMJ 1997;314(7074):131–4.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTQvNzA3NC8xMzEiO3M6NDoiYXRvbSI7czoxODoiL2Nqcy80NS81LzM1MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.  Hadorn DC, the Steering Committee of the Western Canada Waiting List Project. Setting priorities for waiting lists: defining our terms. CMAJ 2000;163: 857–60.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjMvNy84NTciO3M6NDoiYXRvbSI7czoxODoiL2Nqcy80NS81LzM1MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

6.  Hadorn DC, and the Steering Committee of the Western Canada Waiting List Project. Setting prioritieson waiting lists: pountcount systems as linear models (accepted for publication in J Health Serv Res Policy).
    
    

7.  Health Policy and Economic Research Unit, British Medical Association. Waiting list prioritisation scoring systems: a discussion paper. London: British Medical Association, 1998. (Available free of charge from BMA House, Tavistock Square, London WC1H 9JR.) Available [www.bmj.com/cgi/content/full/318/7176/78](http://www.bmj.com/cgi/content/full/318/7176/78) (accessed July 16, 2002).
    
    

8.  Hadorn DC, Holmes A, The New Zealand Priority Criteria Project. Criteria pilot tests. Available [www.bmj.com/cgi/content/full/314/7074/131/DC5](http://www.bmj.com/cgi/content/full/314/7074/131/DC5) (accessed June 14, 2002).
    
    

9.  West R. Waiting for general surgery: a question of priorities. J Manage Med 1991;5:19–26.
    
    

10. Parmar JR. A waiting list initiative in general surgery — experience in a large district general hospital. Ann R Coll Surg Engl 1993;75:4–6.
    
    

11. Mitchell IM, Quinn DW. Urgency and priority models. Model has limited practical application. BMJ 1998;316(7147):1828–9.
    
    [FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTYvNzE0Ny8xODI4IjtzOjQ6ImF0b20iO3M6MTg6Ii9janMvNDUvNS8zNTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12. DeCoster C, Carriere KC, Peterson S, Walld R, MacWilliam L. Waiting times for surgical procedures. Med Care 1999;37 (6 Suppl):JS187–205.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00005650-199906001-00016&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10409009&link_type=MED&atom=%2Fcjs%2F45%2F5%2F351.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000080995100016&link_type=ISI) 

13. Olson DW, de Gara CJ. How long do patients wait for elective general surgery [see comment]Can J Surg 2002;45:31–3. Comment in: *Can J Surg* 2002;45:6–7.
    
    

14. Dennett ER, Kipping RR, Parry BR, Windsor J. Priority access criteria for elective cholecystectomy: a comparison of three scoring methods. N Z Med J 1998; 111:231–3.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9695752&link_type=MED&atom=%2Fcjs%2F45%2F5%2F351.atom) 

15. Dennett ER, Parry BR. Generic surgical priority criteria scoring system: the clinical reality. N Z Med J 1998;111:163–6.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9612483&link_type=MED&atom=%2Fcjs%2F45%2F5%2F351.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000073548300004&link_type=ISI)