Intended for healthcare professionals

CCBYNC Open access
Research

Impact of reduction in working hours for doctors in training on postgraduate medical education and patients’ outcomes: systematic review

BMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d1580 (Published 22 March 2011) Cite this as: BMJ 2011;342:d1580
  1. S R Moonesinghe, consultant and honorary senior lecturer in anaesthesia1, director2,
  2. J Lowery, specialty trainee, anaesthetics3,
  3. N Shahi, specialty trainee, surgery4,
  4. A Millen, specialty trainee, surgery5,
  5. J D Beard, honorary professor of surgical education and consultant in vascular surgery6
  1. 1University College Hospital, London NW1 6BU, UK
  2. 2Surgical Outcomes Research Centre, Joint UCL/UCLH Comprehensive Biomedical Research Centre, London WC1E 6BT
  3. 3Centre for Anaesthesia, Central London School of Anaesthesia, London NW1 2BU
  4. 4Department of Surgery, Diana Princess of Wales Hospital, Grimsby DN33 2BA
  5. 5Department of Vascular Surgery, Doncaster Royal Infirmary, Doncaster DN2 5LT
  6. 6Department of Surgery, Northern General Hospital, Sheffield S5 7AU
  1. Correspondence to: S R Moonesinghe ramani.moonesinghe{at}uclh.nhs.uk
  • Accepted 24 January 2011

Abstract

Objectives To determine whether a reduction in working hours of doctors in postgraduate medical training has had an effect on objective measures of medical education and clinical outcome.

Design Systematic review.

Data sources Medline, Embase, ISI Web of Science, Google Scholar, ERIC, and SIGLE were searched without language restriction for articles published between 1990 and December 2010. Reference lists and citations of selected articles.

Study selection Studies that assessed the impact of a change in duty hours using any objective measure of outcome related to postgraduate medical training, patient safety, or clinical outcome. Any study design was eligible for inclusion.

Results 72 studies were eligible for inclusion: 38 reporting training outcomes, 31 reporting outcomes in patients, and three reporting both. A reduction in working hours from greater than 80 hours a week (in accordance with US recommendations) does not seem to have adversely affected patient safety and has had limited effect on postgraduate training. Reports on the impact of European legislation limiting working hours to less than 56 or 48 a week are of poor quality and have conflicting results, meaning that firm conclusions cannot be made.

Conclusions Reducing working hours to less than 80 a week has not adversely affected outcomes in patient or postgraduate training in the US. The impact of reducing hours to less than 56 or 48 a week in the UK has not yet been sufficiently evaluated in high quality studies. Further work is required, particularly in the European Union, using large multicentre evaluations of the impact of duty hours’ legislation on objective educational and clinical outcomes.

Introduction

There has been a progressive reduction in the working hours of doctors in training in Europe and North America over the past 20 years. Legislation reducing junior doctors’ working hours (known as Code 405) was implemented by the New York State Department of Health in 19891 and limited doctors in that state to working an average of 80 hours a week. This was followed by national guidelines recommended by the US Accreditation Council for Graduate Medical Education (ACGME) in 2003.2 Further refinements of these limits were recommended by the US Institute of Medicine in December 2008.3 In the United Kingdom, a progressive reduction in junior doctors’ working hours has resulted from the New Deal negotiated by the British Medical Association4 and the European Working Time Directive (93/104/EC).5

While the aim of these changes is to improve working conditions and safety, the medical profession has raised concern about the potentially adverse effects on postgraduate training for junior doctors and the provision of high quality care for patients. These concerns are particularly relevant in the European Union but might also be important in the US as the working hours of interns and residents continue to be reviewed.6

When changes to doctors’ working hours are considered and implemented there is a need for an evidence based approach to evaluating their impact on both educational and clinical outcomes. We carried out a systematic review to determine the impact of a reduction in working hours of doctors in postgraduate medical training on objective measures of educational and clinical outcome.

Methods

We adhered to MOOSE (Meta-Analysis of Observational Studies in Epidemiology) guidelines7 and previously published recommendations for systematic reviews of observational studies8 in the conduct of our review.

Data sources

We searched Medline, Embase, Google Scholar, the Educational Resources Information Centre (ERIC), and the System of Information on Grey Literature in Europe (SIGLE). The search was limited to articles published between 1 January 1990 and 20 December 2010, with no language restriction. We identified 49 084 articles, which we then screened for inclusion.

Search strategy

The Medline search was conducted by exploding the following MeSH terms: “Medical Staff, Hospital” or “Personnel Staffing and Scheduling” or “Workload” or “Time Factors” and combining with the following terms, which we also exploded: “Education, Medical, Continuing” or “Education, Medical, Graduate” or “Specialization” or “Education, Medical” or “Internship and Residency”. We searched the following keywords separately: “medical training” and “European Working Time Directive”. The search found 16 132 articles. After snowballing, we included 39 in the final analysis.

In Embase we searched the following terms: MeSH terms “working time” or “personnel management” or “work schedule” or the keyword “European working time directive” and combined the following terms: MeSH terms: “medical education” or “patient care” or “medical education” or “patient care” or postgraduate education” or “surgical training” or “training” or “staff training” or “residency education” or the keyword “postgraduate education”. This search found 8556 articles. After removing duplicates from the Medline search and snowballing, we included seven in the final analysis.

We searched Google Scholar using the following exact phrases on advanced searches limited only to medicine, pharmacology, and veterinary sciences: “working hours” (11 070 articles), “working time” (5020); “duty hours” (1530) and “duty hour” (535). After removing duplicates from the Medline and Embase searches and snowballing, we included 27 studies in the final analysis.

We searched the System for Information on Grey Literature in Europe (SIGLE) combining the keywords: “European Working Time Directive” and “medical education” (4831); “European Working Time Directive” and “patient” (479); “European Working Time Directive” and “medical” (807); “European Working Time Directive” and “surgery” (93). No eligible studies were identified.

We searched the Educational Resources Information Centre (ERIC) using the following keywords in combination: “medical education” and “working hours” or “duty hours”. We found 31 articles; none was eligible for inclusion.

Study selection

We selected studies that assessed the impact of a change in duty hours, with details reported of what change had been implemented, and used an objective measure of outcome related to postgraduate medical training, patient safety, or clinical outcome.

We excluded studies reporting subjective measures, such as surveys or questionnaires, unless the results included an objective externally validated measure, such as case numbers or results of assessments. We also excluded studies that assessed the effect of changes in duty hours on medical staff (for example, measures of fatigue, physical or psychological wellbeing) as opposed to patients. Any study design was eligible for inclusion.

Of the 49 084 citations screened, we identified 157 articles to review in detail. We “snowballed” these articles by examining reference lists and searching for citations on Medline, Embase, and ISI Web of Science; this process identified a further 68 articles. Two authors (from SRM, JL, NS, and AM) independently reviewed these 225 papers, and 72 met inclusion criteria: 38 of postgraduate medical training, 31 of outcomes related to patients, and three that reported both (figure).

Figure1

Review process and identification of studies for inclusion

Data extraction

Two reviewers (from SRM, JL, NS, and AM) independently extracted information from each article using standardised data extraction forms, and one author (SRM) reviewed all studies. Using two separate forms (one for studies of educational outcomes, one for studies of patients’ outcomes), we extracted data on study authors, geographical location, year of publication, study cohort characteristics, working hours and pattern before and after the intervention, outcome measures, and main results from all studies.

We also assessed the quality of the studies, again by extracting data onto separate data extraction forms. For studies of training outcomes, we extracted number of participants, study design, institutional setting, source, and method of data collection; whether the study was single or multicentre; whether a description of overall institutional activity was included in studies that used case volume as an outcome measure; and whether statistical analysis was reported. The quality of reporting of cohort characteristics in studies of postgraduate training outcomes was assessed as being “good” if the study reported both the number of hours a week worked and described shift patterns, “moderate” if the study reported only one of these, and “poor” if the study reported neither.

For studies of patients’ outcomes, we extracted data on number of participants, study design, institutional setting, source and method of data collection, whether the study was single or multicentre, whether the study included a control group, whether outcomes were risk adjusted or whether the study presented a comparison of patients’ characteristics at baseline, and whether statistical analysis was reported.

Results

Tables 1-3 summarise the key findings from studies evaluating medical training, for improvement in outcomes, no change or variable outcomes, and deterioration in outcomes, respectively . Tables 4-6 summarise the key findings from studies assessing outcomes related to patients, for improvement in outcomes, no change or variable outcomes, and deterioration in outcomes, respectively . Though we did not restrict our search to English language publications, all the studies we identified originated from either the UK or US. Table 7 documents the changes to duty hours recommended in the UK and US. All studies from the US reported the impact of a change from more than 80 duty hours a week to fewer than 80, in accordance with Code 405 legislation or ACGME recommendations. The UK based studies reported the impact of various changes in total duty hours and shift patterns, in accordance with UK legislation and the European Working Time Directive. The standardised data extraction tables described in the methods section are shown in the appendix on bmj.com.

Table 1

 Studies on effect of reduced working hours that predominantly found improvement in postgraduate education

View this table:
Table 2

 Studies on effect of reduced working hours that found no change or variable outcomes in postgraduate education

View this table:
Table 3

  Studies on effect of reduced working hours that predominantly found deterioration in postgraduate education

View this table:
Table 4

 Studies on effect of reduced working hours that predominantly found improvement in patients’ outcomes

View this table:
Table 5

  Studies on effect of reduced working hours that found no change or variable changes in different patients’ outcomes

View this table:
Table 6

  Studies on effect of reduced working hours that predominantly found deterioration in patients’ outcomes

View this table:
Table 7

 Summary of changes and recommendations in duty hours

View this table:

Studies of postgraduate medical training

We analysed 41 studies of postgraduate medical training. All were “before and after” cohort studies. Twenty seven originated from the US, and 14 were conducted in the UK. Twenty eight studied training in surgery or surgical subspecialties (22 in US, six in UK), five studied training in obstetrics or gynaecology, or both (four in US, one in UK), six studied training in anaesthesia (all from the UK), one US study was in paediatrics, and one UK study was of medical trainees. Of these 41 studies, two showed an improvement in training outcomes after a reduction in working hours, 12 reported a deterioration, and the 27 remaining showed no change or a combination of positive and negative results. Most studies used multiple outcome measures to evaluate the impact of a reduction in working hours on postgraduate education and training.

Postgraduate training results according to effect of reduced hours

Improved training outcomes

Two papers reported an improvement in training outcomes after a reduction in working hours: one UK study of medical trainees9 and one US study of surgical residents.10 Both were of low methodological quality as they did not report statistical analyses of the results. Roedling et al reported the change in shift pattern,9 and Schneider et al reported percentage compliance with duty hours changes,10 but neither reported the actual number of hours worked by trainees.

Deterioration in training outcomes

Twelve studies found a detrimental association between reduced working hours and measures of postgraduate training outcome: six from the UK (three in surgery,11 12 13 two in anaesthesia,14 15 and one in gynaecology16) and six from the US (all in surgery).17 18 19 20 21 22 Eleven of these reported operative caseload as an outcome measure11 13 14 15 16 17 18 19 20 21 22; one UK study used continuity of care by neurosurgical trainees as an outcome.12 Of the 11 studies that used operative caseload, three single centre studies did not report institutional operative volumes,17 18 20 and therefore we are unable to determine if the reduction in trainees’ caseload could have occurred as a result of a change in institutional activity.

No change in training outcomes

Of the 27 studies that found no change or mixed results of training outcomes associated with reducing working hours, 20 originated from the US (four in obstetrics and gynaecology,23 24 25 26 16 in surgery,27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42) and seven from the UK (four in anaesthesia,43 44 45 46 three in surgery47 48 49). Of the 20 studies from the US, 12 used operative caseload as an outcome measure, two analysed postgraduate examination results, and six used both. Four US papers studying surgical training reported results from large multicentre or multiprogramme cohorts.29 30 39 40 The 16 remaining papers from the US were studies in single institutions or residency programmes. Only four reported both the actual number of hours worked by trainees before and after rota changes and the shift patterns worked.18 27 33 36

Postgraduate training results according to outcome measures analysed

Training opportunities

Six studies used measures of training opportunities (such as supervised operating lists or teaching sessions). Of these, one UK study of internal medicine physicians found an improvement in attendance at training sessions,9 and one UK study of anaesthetists found a deterioration in the number of training opportunities in obstetric anaesthesia.15 The remaining four studies were all conducted in UK anaesthetics departments and found no change in supervision levels overall after various changes in duty schedules.43 44 45 46

Examination scores

Nine studies, all originating from the US, measured changes in examination scores for cohorts of trainees before and after duty hour reforms. Two found an improvement in scores,10 27 and seven found no difference.24 31 32 34 35 38 40

Caseload

Thirty seven studies included caseload as an outcome measure. Only one study, from the US, showed an increase in caseload after a reduction in working hours.10 Eleven studies showed a reduction in operative caseload after reform of duty hours: six US and two UK studies of surgical trainees,12 13 48 one UK study of gynaecology trainees,16 and two UK studies in anaesthesia.14 15 Twenty five studies found no change in operative caseload, whether in surgery (17 studies: 14 in US, three in UK), obstetrics and gynaecology (four studies, all in US), or anaesthesia (four studies, all in UK) after reductions in working hours.

Schneider et al’s 2007 study in US surgical residents was the only paper to report an increase in caseload after a reduction in working hours, and it also found an improvement in postgraduate examination scores.10 This was a single centre study of low methodological quality, however, as there was no statistical analysis conducted on the results.

Of the 11 papers that reported a reduction in caseload, eight were in surgery, two in anaesthetics, and one in gynaecology. All except one21 were single centre, and two did not conduct statistical analysis of their results.11 15 Six of the surgical papers were conducted in the US,17 18 19 20 21 22 including the only large multicentre study to show a reduction in caseload.21 Of the two UK studies of surgical trainees, one showed a reduction in operative caseload with a reduction in hours from 72 to 48 a week,13 and the other with a reduction from 58 to 54 hours a week.11 Both these studies were of low methodological quality: Kara et al did not conduct a statistical analysis of their results11 and Stephens et al reported a single centre cohort study of only the most junior surgical trainees.13 Both studies of trainees in anaesthetics that reported a reduction in caseload were conducted in the UK. One paper examined the impact of reducing hours from 60 to 56 a week in a specialist paediatric centre14; the other reported the effect of reducing hours from 72 to 58 a week on trainees’ exposure to obstetric general anaesthesia.15 The authors of the latter paper did not statistically analyse their results and commented that the total operative caseload of the department had also fallen, providing an alternative explanation for the reduced number of cases with trainee involvement after reducing working hours.15 The only study of gynaecology trainees that reported a reduction in caseload was conducted in a single UK centre and found a reduction in both the overall number of cases involving trainees and in subgroup analyses of specific procedures.16

Most papers analysed found no difference in caseload associated with a reduction in working hours. Eighteen originated from the US: 14 in surgery22 27 28 30 31 32 33 34 35 36 37 41 42 and four in obstetrics and gynaecology.23 24 25 26 Four of the US papers examining surgical training analysed large multicentre cohorts,29 30 39 40 though the actual working hours of the residents studied in these four papers was not reported and so compliance with duty hour recommendations could not be assessed. Only six of the 14 US papers that used operative caseload as an outcome measure reported institutional case numbers before and after duty hours reform.19 22 31 33 36 37 Four UK studies of anaesthesia training reported no change in either caseload or supervision rates43 44 45 46; three of these documented concurrent institutional activity,44 45 46 but two did not conduct statistical analyses on their results.44 46 Four studies in obstetrics and gynaecology conducted in the US reported no change in caseload with reform of duty hours; one was multicentre,26 three reported institutional volumes.23 25 26 Only two of these four papers detailed both the actual hours worked and the shift patterns.23 24

Studies of patients’ outcomes

Thirty four papers documented the impact of reducing the working hours of doctors in training on patients’ outcomes (tables 4-6) . Only three studies originated from the UK,50 51 52 the rest were conducted in the US. Most were “before and after” cohort studies, but one was a randomised controlled trial.53 Most studies reported clinical outcomes such as morbidity and mortality or measures of resource use such as length of stay. Seven, however, reported on patient safety indicators such as rates of adverse events or medical errors,50 53 54 55 56 57 58 and one used continuity of care in paediatrics as an outcome measure.59 Sixteen studies were in surgery or surgical subspecialties,31 35 38 56 57 58 60 61 62 63 64 65 66 67 68 69 seven were in internal medicine,50 52 70 71 72 73 74 two were in critical care,53 75 two were in paediatrics,59 76 and one was in obstetrics.54 Six studies evaluated multispecialty cohorts.51 55 77 78 79 80 As none of these studies adjusted their statistical analyses appropriately for multiple comparisons, it is possible that some of the significant results in studies that reported multiple outcomes, particularly those where there was a combination of positive, negative, and neutral results, might have occurred by chance. For studies reporting multiple outcome measures, we have categorised them according to whether most outcomes were better, worse, or showed no difference with a reduction in working hours.

Studies showing predominantly improved outcomes in patients

Four studies showed improvement in patients’ outcomes with a reduction in working hours; one was conducted in the UK50 and three in the US.53 61 63 One of these was a randomised controlled trial of high methodological quality that randomised residents in critical and coronary care units to two different working patterns. Patient safety indicators including medical and diagnostic error rates were compared and found to be improved after a change in doctors’ working hours from a traditional shift system, with up to 37 continuous duty hours and 77-81 hours worked a week, to a rota that eliminated extended shifts and reduced the working week to 60-63 hours.53 Only one of the remaining three cohort studies was a large multicentre study, which found a difference in mortality rates.61 This paper studied trauma admissions over a four year period and compared outcomes in teaching hospitals before and after duty hour reforms with concurrent outcomes in non-teaching centres (and therefore unaffected by trainee working hours). The authors found improvements in mortality and length of stay in intensive care in teaching departments that were not seen in the control hospitals.61 One small single centre study from the UK found that rates of adverse events were reduced with a shortening of working hours, though this study could be criticised as different medical specialties were examined in the two cohorts.50 Finally, a large single centre study of patients undergoing laparoscopic cholecystectomy showed an improvement in complication rates after a reduction in working hours of surgical trainees.63

Studies showing predominantly worse outcomes in patients

Two large studies in trauma and orthopaedics, both from the US, found that rates of complications, but not mortality, worsened with a reduction in working hours. Browne et al’s multicentre analysis of patients undergoing surgery for hip fracture found that the occurrence of nine postoperative complications was worse in teaching hospitals (but not in control non-teaching hospitals); the incidence of eight other complications was similar and only one was improved.60 The length of stay in hospital and routine discharge rates were also worse in teaching hospitals after reforms in duty hours. As the statistical analyses were not adjusted for multiple comparisons, the significant findings could still have occurred by chance. Salim et al’s study of trauma patients showed an increase in complication rates in a large single centre cohort over four years (two before and two after duty hour reform).62 Baseline characteristics of the two cohorts were presented and were similar, though the outcomes were not adjusted for patients’ risk; it is possible therefore that the difference in complications could be attributable to unknown differences in risk factors in patients. Furthermore, though the differences in complication rates were significant, they were small (total complication rate 5.64% v 7.28%; preventable complications 0.89% v 1.28%; non-preventable complication rate 4.75% v 5.81%), and therefore the clinical relevance of these changes could be questioned. Nevertheless, the authors highlighted that these results were worrying and could at least in part be attributable to the change in duty hours.

Studies showing predominantly no difference in outcomes in patients

Most studies showed that duty hour reform did not affect standards of care of patients. Twenty eight studies reported no significant difference in patients’ outcomes after reductions in the working hours of doctors in training or reported a combination of positive, negative, and similar outcomes. Only two of these originated from the UK.51 52 The study by Collum et al examined changes in hospital standardised mortality ratios (HSMRs) in one region of the UK over a three year period and found that they improved over time in line with improvement in such ratios in the rest of the UK.51 The authors reported no specific details about changes in duty hours, either locally or nationally; it is therefore possible that similar changes in working patterns were implemented outside the geographical area of interest. This raises the question of whether all these improvements were associated with reducing working hours or coincidental with it. McIntyre et al examined outcomes of emergency medical admissions over a two year period and found similar mortality, length of stay in hospital, and 30 day readmission rates.52 This was a small single centre study, however, and might have been underpowered to detect differences in outcome.

Twenty six studies from the US showed similar or mixed patients’ outcomes after a reduction in duty hours. Several of these analysed relatively small or single centre cohorts of patients and might have been underpowered to detect changes in some of the outcomes reported. These included studies in critical care,75 obstetrics,54 neonatology,76 paediatrics,59 trauma,31 57 69 surgery,35 38 66 67 68 and internal medicine.70 73 74 Five US papers included concurrent analyses of “control” hospitals or wards that did not employ doctors in training and were therefore unaffected by changes in duty hour regulations.56 64 65 71 72 Four of these five papers compared mortality rates: three found similar improvements over time in both teaching and non-teaching hospitals in cardiothoracic,64 trauma,65 and medical patients64 65; 72 one study of medical inpatients was underpowered to detect a difference.71

Three other US studies compared outcomes in hospitals of different “teaching intensity” (as defined by the ratio of residents to beds).58 79 80 Rosen and co-workers studied patient safety indicators in surgical patients and found a combination of positive, negative, and neutral results for different outcomes.58 The two other studies were both conducted by Volpp et al, with identical methods and over the same time period in two different US healthcare systems (Veterans Affairs and Medicare) and looked at mortality rates in medical and surgical patients separately.79 80 In Veterans Affairs hospitals, outcomes for medical patients improved with duty hour reforms in hospitals of higher “teaching intensity”; for surgical patients, however, there was no significant change in outcome associated with the reduction in working time.79 These data are similar to those from one other study that found an improvement in mortality in medical patients but not surgical patients.77 In the study of Medicare hospitals, however, there was no apparent change in outcome in either medical or surgical patients after a reduction in duty hours.80 Similarly, Silber et al found no difference in length of stay in hospital either in medical or surgical patients admitted to either Medicare or Veterans Affairs hospitals.78

Discussion

In this systematic review of the impact of a reduction in working hours for junior doctors, using objective measures of postgraduate medical training, patient safety, and clinical outcome, we found that most studies found either a beneficial or neutral impact on patient safety and clinical outcome and limited or no effect on postgraduate training as measured by procedural volume per trainee or examination results. When duty hours are reduced to below 56 or 48 hours a week, in accordance with European legislation, we could not draw conclusions on the impact on patients’ outcomes or medical training because of conflicting results from different institutions and specialties and the poor quality of some of the studies evaluated.

Study limitations

Our study is limited by several factors. Firstly, the heterogeneity of the included studies does not permit meta-analysis of the results, and their predominantly retrospective observational methods means that the individual studies, and therefore our review, could be subject to bias. Secondly, the included studies varied considerably in quality, though we did include an evaluation of their quality in our results (see tables B and D in the appendix on bmj.com). Some of the studies that evaluated patient safety did not include risk adjustment or present baseline characteristics of patients, which would allow true comparison of clinical outcomes. Several studies that evaluated training included no statistical analysis or had poorly defined cohort characteristics. As several postgraduate education studies did not report denominator data on overall hospital activity, a change in hospital workload could have accounted for differences in trainee caseload numbers after reducing working hours. Thirdly, the outcome measures used in the individual studies varied considerably, and, particularly in the studies evaluating postgraduate medical training, the reported measures might not have been appropriate for the intended purpose. Finally, publication bias could have affected our results, though we attempted to avoid this by conducting an extensive review of the “grey literature.”

Scope and limitations of published literature

We can make several observations regarding the nature and origin of the published studies in this topic. Firstly, the concern over the implications of duty hour regulations has led to a much larger and higher quality body of work from the US than Europe. This could in part be because the most significant changes in the US occurred in 1989 and 2003, as opposed to European changes, which are more recent; there might be European studies yet to be completed or published. Secondly, it is notable that despite the lack of language restrictions and an extensive review of the “grey literature,” we were unable to identify any European studies that fulfilled our inclusion criteria and were conducted outside the UK. This could reflect that the concerns of the US and UK medical communities are not reciprocated elsewhere or perhaps that compliance with working time legislation is not as rigorous in mainland Europe as in the UK. Thirdly, most studies of postgraduate education analysed cohorts of doctors in training in “craft specialties” that have an emphasis on technical competence, such as surgery, anaesthesia, and obstetrics and gynaecology. It is therefore possible that internal medicine physicians do not believe that restrictions on working hours will have negative implications for training. Alternatively, it could be that there are fewer easily interpretable measures of training outcome for physicians than there are in more technically focused specialties. Finally, there is a much larger body of literature examining the implications of reducing working hours on patients’ outcomes originating from the US compared with the UK. Possible reasons for this include a belief by the UK healthcare community that legislative changes are unlikely to affect quality of care of patients, or perhaps, more probably, the lack of robust systems for collecting and analysing large datasets of risk adjusted outcomes in patients in the UK and Europe.

Impact on patients’ outcomes

The literature examining the association between reducing working hours and objective measures of outcomes in patients shows that there is no clear signal to indicate either benefit or harm. While it might seem intuitive that doctors working fewer hours will be less tired, make fewer errors, and that therefore patients’ outcomes should improve, both clinical experience and the results of studies conducted with identical methods but in different healthcare systems indicate that other influences might be at least as important.79 80 These could include the number and quality of clinical handovers,81 the level of supervision of doctors in training,82 and factors unrelated to doctors’ duty hours and schedules such as the continuity of care provided by the entire multidisciplinary team,83 the standard of nursing care,84 and many other differences within and between institutions in delivery of healthcare. The heterogeneity of results in our review might be due to such differences related to structure and process, and explaining these variations is the focus of a growing body of work.85 86 87 A greater understanding of such issues would be valuable in ascertaining if there are standards of care that might help to improve outcomes in patients when doctors’ working patterns are changed so that such standards could be more widely implemented.

Impact on educational outcomes

The studies focusing on training outcomes indicate that the Code 405 or ACGME regulations in the US have had an inconsistent effect and that there are insufficient data from studies of high methodological quality to be able to draw firm conclusions on the impact of the European Working Time Directive or New Deal in the UK. When interpreting these findings, one should consider that, alongside the reduction in working hours, other changes in postgraduate medical education and provision of healthcare might influence the quality of training. In the UK, training has moved away from an apprenticeship model to a time limited programme, with greater emphasis on clinical supervision. Concurrently, and consequently, there has been a change in the role of the junior doctor in healthcare service delivery. Use of ancillary staff to provide services previously provided by junior doctors, such as phlebotomy, cannulation, and basic administrative duties, allows more time for useful training activity in a working week limited by duty hour regulations. It is possible that cohorts of doctors whose training outcomes did not change with a reduction in working hours have also seen an alteration in their clinical and administrative responsibilities. The requirement for this sort of change was a key recommendation of the recent inquiry into the implications of the European Working Time Directive on medical training in England: “make every moment count.”88

One of the difficulties in evaluating the effects of changes in working hours is the lack of validated measures for assessing the outcome of training. We chose to focus on studies that reported objective measures, such as operative case numbers or clinic attendances, as these are less likely to be biased than subjective surveys of the profession’s opinion. Even these objective measures, however, do not fully evaluate the quality of training. For example, case numbers might be a helpful guide to experience but not of training quality: performing 100 simple cases might not be as useful as undertaking 10 complex procedures. Studies in “craft specialties” focused predominantly on the number of operative cases undertaken. In all these specialties, however, time spent outside the operating theatre, such as in the outpatient clinic, emergency room, and on inpatient wards, also has value and is important for the development of a rounded independent practitioner. Furthermore, quantitative measures such as procedural volume must be differentiated from measures of knowledge, skill, attitude, and behaviour, all of which are essential attributes for the independent medical practitioner. In the UK, workplace based assessments, such as direct observation of procedural skills, case based discussion, and multi-source feedback, are now being used to evaluate competence and aid progression of junior doctors.89 Use of other assessments of procedural skill, such as cumulative sum (CUSUM) analysis, and evaluating the association of the outcomes of these assessments with patients’ outcomes also requires further evaluation.90

Ultimately, the goal of postgraduate medical training is to produce doctors who are safe competent independent practitioners. Both in the UK and US, super-specialisation, particularly in “craft specialties,” means that a consultant or attending physician today might not be required to carry out the same range of independent duties as their predecessors. Additionally, the medical profession is seeking alternate methods of training doctors outside their working hours, including the use of medical simulation and clinical skills laboratories for training in human factors and technical procedures. It might therefore be possible for a time limited training programme to produce doctors who are clinically safe and able to work independently in the modern healthcare system but who might have previously been considered to have insufficient experience to fulfil a wider range of clinical responsibilities.

Conclusions and policy implications

The most important test of success of postgraduate training is the professional performance of doctors who reach the end of it. In this review, studies reporting patient safety and clinical outcome measures examined only the immediate effect of reductions in duty hours on care of patients; however, the potential impact of such changes can be fully evaluated only some years after duty hours reforms, when cohorts of doctors trained in a limited number of hours a week are finally practising independently. To that end, we have the following suggestions. Firstly, a consensus should be reached by the medical profession on appropriate measures to assess the quality of postgraduate medical training. These should be both quantitative and qualitative and could include both process measures, such as procedural volume, supervision levels, and time spent in “alternative” training environments such as simulators, and outcome measures such as the results of formative and summative assessments. Secondly, once such measures have been agreed, we recommend that they are confidentially reported to the organisations responsible for the quality assurance and regulation of training, such as the ACGME in the US, and the Medical Royal Colleges and General Medical Council in the UK. The reporting process should be conducted both on an individual doctor basis for trainees, interns, and residents, and on an institutional level for teaching departments and programmes. This would enable comparison between different regions and specialties and identification of practices that lead to high quality training. Finally, we recommend the conduct of longitudinal studies, evaluating the relation between the agreed process and outcome measures for postgraduate training and objective measures of outcomes related to patients for clinicians in the first few years of independent practice, such as rates of clinical errors or complications or medical negligence or malpractice claims. Such studies would enable the evaluation of the validity and reliability of measures of training quality for the prediction of the future standard of care of patients provided by independent clinicians and would permit the refinement and re-evaluation of these measures.

It has been stated that “training is patient safety for the next 30 years.”88 We wholeheartedly support this view. Our study, which summarises a diverse and sometimes methodologically flawed body of literature, has been unable to reach firm conclusions, but we consider that this in itself is an important observation. We have highlighted the need for a more systematic approach to evaluating the impact of legislative changes of duty hours and the challenges of conducting high quality audit and research in this area. In the future, it will only be through the conduct of large, collaborative, multicentre evaluations of training and outcome that both the public and the profession can be reassured that the standard of medical training, and therefore of future care of patients, is of the highest possible quality and will be maintained or improved over time.

What is already known on this topic

  • There has been a progressive reduction in the working hours of doctors in postgraduate training both in the US and Europe, in line with recommendations and legislation aimed at improving patient safety and doctors’ working conditions

  • Concerns have been raised over the potential for unintended adverse consequences for patients’ outcomes and clinical training

What this study adds

  • Reducing doctors’ working hours from over 80 hours a week has had limited impact on postgraduate training and minimal effect on patients’ outcomes in the US

  • In the UK, there is insufficient evidence from high quality studies to draw a conclusion on the effect of reducing working hours to less than 56 or 48 hours a week on objective measures of postgraduate training or clinical outcomes

Notes

Cite this as: BMJ 2011;342:d1580

Footnotes

  • Contributors: SRM and JDB were responsible for the conception and design of the study. SRM had principal responsibility for analysing and interpreting the data and for drafting the article, revisions, and final approval. JL, NS, and AM contributed to analysis and interpretation of the data. JDB contributed to drafting, revision, and final approval of the article. SRM is the guarantor.

  • Funding: SRM works within the UCL/UCLH Joint Comprehensive Biomedical Research Centre, which received a proportion of funding from the National Institute for Health Research’s funding scheme. Part of this work was conducted while SRM was a National Institute for Academic Anaesthesia (NIAA) Research Fellow, supported by a grant awarded to the NIAA’s Health Services Research Centre by the Frances and Augustus Newman Foundation. This study was not commissioned and no project specific funding was received. The funders had no role in the study design, data collection, analysis, and interpretation; writing the manuscript; or the decision to submit the research for publication.

  • Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Data sharing: No additional data available.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

References

View Abstract