Abstract
Background: Quality in health care can be evaluated using quality indicators (QIs). Elements contained in the surgical operative report are potential sources for QI data, but little is known about the completeness of the narrative operative report (NR). We evaluated the completeness of the NR for patients undergoing a pancreaticoduodenectomy.
Methods: We reviewed NRs for patients undergoing a pancreaticoduodenectomy over a 1-year period. We extracted 79 variables related to patient and narrator characteristics, process of care measures, surgical technique and oncology-related outcomes by document analysis. Data were coded and evaluated for completeness.
Results: We analyzed 74 NRs. The median number of variables reported was 43.5 (range 13–54). Variables related to surgical technique were most complete. Process of care and oncology-related variables were often omitted. Completeness of the NR was associated with longer operative duration.
Conclusion: The NRs were often incomplete and of poor quality. Important elements, including process of care and oncology-related data, were frequently missing. Thus, the NR is an inadequate data source for QI. Development and use of alternative reporting methods, including standardized synoptic operative reports, should be encouraged to improve documentation of care and serve as a measure of quality of surgical care.
Quality improvement is an important component of health care systems. Quality in health care can be evaluated in terms of the structures, processes and outcomes of care.1 Process and/or outcome data are used as quality indicators (QIs) for performance management. Outcome data, such as survival time, complication rates or quality of life data, are often difficult to obtain or take a long time to mature. As a result, process of care data are frequently used as a surrogate for outcome data when measuring the quality of a health care system because process of care data are often available from administrative data sources.
At present, few QIs are available that measure the processes of care that occur during a surgical procedure. There is interest is measuring components of the operative report as a potential source of data for QIs. Completeness and accuracy of an operative report may be a means to assess the quality of care delivery and to identify opportunities for new quality initiatives. Thus, elements of the operative report have the potential to be used as QIs, but to our knowledge, this has not yet been established in the literature.
A narrative operative report (NR) is currently the standard documentation method used for the vast majority of surgical procedures in North America. It is an open format description of the operative steps performed during a surgical procedure dictated by a surgeon in narrative form. The content of the NR is not standardized or regulated.2 The primary function of the NR is to document procedural events for clinical and medicolegal reasons. An NR may potentially be used to measure the quality of the surgical procedure if intraoperative process of care measures can be extracted in a robust manner. But, at the present time little is known about the quality or the completeness of the NR. A study by Govindarajan and colleagues3 found that NRs can be used to extract data about nontechnical competencies of a surgical procedure, but the authors did not address issues related to the quality of health care. Others have reported that NRs are of variable quality owing to incomplete and/or inadequate reporting.2,4–8 For patients with cancer, Edhemovic and colleagues4 reported that NRs failed to adequately document the oncologically relevant elements that occur in rectal cancer procedures. Incomplete and inconsistent documentation in the NR may compromise the ability of physicians to make optimal decisions regarding further treatment.9
Newer documentation methods have been developed that allow for standardized reporting of operative procedures. Proponents of standardized operative reports, also known as synoptic operative reports (SRs), point to more complete documentation with fewer omissions in SRs than NRs, resulting in higher quality data, as reasons to adopt the SR. Data from SRs may be used to improve communication between different health care providers to optimize clinical care, resulting in higher quality of care. For example, SRs that include details regarding the margin status of a procedure (i.e., R0, R1 or R2) may help inform the planning of postoperative adjuvant therapy, such as external beam radio-therapy. Also, data from an SR can be used for performance evaluation, quality improvement and research purposes.
Pancreaticoduodenectomy is performed for pancreatic cancer. Institutions that perform a high volume of pancreaticoduodenectomies have better outcomes than centres that perform only a few procedures per year.10–14 However, surgeon-specific processes account for a substantial component of the observed volume–outcome associations for pancreaticoduodenectomy.15–20 It is postulated that processes related to the technical proficiency and adherence to oncologic principles during the pancreaticoduodenectomy contribute to the improved outcomes observed. Thus, operative notes may be a useful source of intraoperative process of care data for this procedure.
The objective of our study was to evaluate the potential of the NR as a possible source of quality improvement data. Using document analysis, we assessed the completeness of reporting of data in NRs from pancreaticoduodenectomy procedures to evaluate the quality of data available in NRs.
Methods
We identified the NRs of patients who underwent a pancreaticoduodenectomy between Jan. 1, 2008, and Dec. 31, 2008, from a prospective maintained database at the University Health Network, Toronto, Ont. This academic institution has a high volume of hepato-pancreato-biliary (HPB) procedures yearly, performed by 10 subspecialty-trained surgeons. We obtained ethics approval from the institutional research ethics board before the study commencement.
We analyzed the contents of dictated NRs. A draft framework of data elements considered important for an NR for pancreaticoduodenectomy was developed based on a literature review of outcomes following pancreaticoduodenectomy, operative variables that were collected in an existing provincial clinical database and input from general surgeons with content expertise. Potential data elements were pilot-tested for face validity by 5 surgical oncologists, including HPB surgical oncologists, and modified based on expert input to create a final set of variables.
We evaluated 79 variables covering 3 domains of interest: process of care, surgical manoeuvres and oncology-related variables. Of the 79 variables, 60 were considered mandatory and 19 were deemed optional.
The standard NR consists of a verbatim transcribed account of the procedure narrated by a physician member of the surgical team. This document is created free-form and is unstructured in format and content. We analyzed dictated NRs from the patients’ electronic medical records; handwritten notes in the paper chart were excluded. Data were extracted from the NRs by an independent data extractor. Demographic and clinical characteristics of the patients and the characteristics of the physician narrator were also recorded.
Statistical analysis
We calculated summary statistics for patient demographic information and the level of training of the individual who narrated the report. The variables were grouped into data elements, and we calculated the median number of variables reported for each data element. Data pertaining to concomitant procedures performed at the same time as the pancreaticoduodenectomy were excluded from analysis, as these elements were unique to each situation and the content was nonstandard.
We performed univariate analysis using the Mann–Whitney U test, χ2 test or Fisher exact test, as appropriate. The data were analyzed using SPSS 15.0. In addition, we performed comparative analysis of the 5 most and least complete dictated NRs for variables of interest as a form of sensitivity analysis.
Results
A total of 78 pancreaticoduodenectomies were performed, and 74 NRs were available for data extraction. In 4 cases, an NR was not dictated and was absent from the electronic medical record. These cases were excluded from our analysis.
Patient characteristics
There were 74 patients analyzed. In 61 patients (82%), a standard Whipple type pancreaticoduodenectomy was performed. Thirteen patients also underwent concomitant vascular procedures including portal vein resections (13 of 74) and/or arterial resections (2 of 74). Twenty-one patients had additional non-HPB procedures.
The majority of patients were men (43 of 74, 58%) and older than 60 years (45 of 74, 61%; Table 1).
Patient and narrator characteristics, n = 74
Narrative report characteristics
The average time to dictation of the NRs was 1.5 days. The physician team member who dictated the NR was the attending surgeon (14 of 74, 19%), clinical fellow (43 of 74, 58%) or senior surgical resident (16 of 74, 22%; Table 1). There were no instances of duplicate NRs. None of the NRs was reviewed or verified by the staff surgeon via the electronic records system.
The median number of variables reported was 43.5 (range 13–54; Tables 2–4). No NR was complete for all 60 mandatory variables. The processes of care and oncologic variables were least complete, with several omissions (Tables 2 and 4). A median of 3 of 9 (range 0–7) processes of care variables were reported. No procedure was complete for all process of care variables. The most commonly omitted process of care variables were urgency of surgery (13 of 74, 18%), time out performed (11 of 74, 15%) and American Society of Anesthesiologists (ASA) status (0%; Table 2).
Note and procedures process characteristics, n = 74
Surgical manoeuvres, n = 74
Operative variables, n = 74
Oncology-specific findings were reported for a median of 5 of 9 (range 0–9) variables. Tumour size (38 of 74, 51%), lymphadenectomy performed (29 of 68, 43%), clinical resection and/or margin status reported (18 of 74, 24%) and lymphadenopathy (22 of 74, 30%) were the least frequently reported oncologic variables (Table 4).
Administrative variables and surgical technique variables were most commonly complete and were reported a for median of 4 of 4 (range 3–4) and 20 of 28 (range 1–26) variables, respectively. All identifying patient information was reported. The procedure performed was reported in 66 (89%) of the cases. Variables associated with surgical technique most commonly reported were those related to pancreatic mobilization and resection with a median of 5 of 6 variables (range 1–6; Table 3).
Narrative report completeness and physician dictator characteristics
The χ2 tests revealed no significant results when comparing narrator type (attending surgeon, clinical fellow, senior surgical resident) for each of the reported variables, except for time out performed, bile duct anastomosis type and specimen disposition (all p < 0.05). Fellows reported time out performed more often, surgical house staff (clinical fellows or residents) reported the bile duct anastomosis type more often, and senior residents reported specimen disposition more often than the other narrator types.
Sensitivity analysis was performed for comparative analysis. The 5 most and least complete NRs were identified and compared. The 5 most complete NRs included 54 (87%) variables. These NRs were all dictated by the same clinical fellow within a week of the procedure date. The 5 least complete NRs included 36 (< 58%) variables. These NRs were dictated by surgical house staff within a week of the procedure date. None of the most or least complete reports was dictated by attending surgeons.
Narrative report completeness and perioperative outcomes
We evaluated the association between perioperative outcomes and completeness of NRs. Completeness was divided into quartiles. Perioperative outcomes of the least and most complete NRs were compared (Table 5) by univariate analysis. Completeness of the NR was positively associated with operating room (OR) times (p = 0.048). In the most complete NR quartile, a median of 48 variables were present and the median OR time was 396 (unknown–800) minutes versus the least complete quartile, which had a median of 39.5 variables present and a median OR time of 347 (251–495) minutes. There was no association between the completeness of the NR and length of stay in hospital (p = 0.96) or major perioperative complications (p = 0.42).
Narrative operative report completeness and perioperative outcomes
Discussion
The NRs are the usual form of documentation used to record the details of a surgical procedure. The purpose of our study was to examine the completeness of NRs in order to evaluate their potential as a source of quality assurance data.
Our results demonstrate that NRs are frequently incomplete. Variables related to surgical technique and administrative details were often present, whereas oncology-related and process of care details were commonly omitted in the majority of NRs. Thus, NRs are a poor source for quality assurance data.
We found that narrator characteristics were associated with NR completeness. When we compared the 5 most and least complete NRs, the 5 most complete NRs were all dictated by the same clinical fellow, whereas the least complete NR (13 variables reported, 16%) was dictated by the least experienced narrator, a senior surgical resident. This suggests that narrator training affects the quality of NRs. To date, surgical education places little, if any, emphasis on teaching trainees how to dictate NRs, and as a result the quality of NRs are expectedly variable.21,22
Also, we report on an association between the completeness of NRs and operative duration. The longer the procedure time, the more likely that the NRs were more complete. Longer operative durations may be associated with more complex procedures, suggesting that the completeness of the NRs may be associated with procedure complexity. This finding is consistent with the results of previous work by Stewart and colleagues,8 who reported that procedural quality was directly related to the operative documentation. We did not find a significant association between NR completeness and patient-related outcomes, such as length of stay in hospital or perioperative complications. However, our study was underpowered to explore this question.
There is scant literature on the quality of operative documentation in surgery. Edhemovic and colleagues4 demonstrated that the most complete parts of NRs contained the least important information (patient information, indication for the procedure and closure technique). Our findings were similar. Information on elements without long-term implications (e.g., incision type, anastomotic suture technique) was virtually always complete, whereas information on oncology-related variables (e.g., clinical margin status, extent of lymphadenectomy performed) was often omitted.
To our knowledge, our study is the first evaluation of the NR for the pancreaticoduodenectomy operation. Strengths of this study are that the documents analyzed reflect NRs from a large contemporaneous sample of an experienced group of HPB surgical oncologists with a uniform approach to the pancreaticoduodenectomy. Thus, the variations that we identified reflect variations in the quality of the NRs rather than substantial variations in the procedure.
Limitations
A limitation of our study is its retrospective nature. In particular, we excluded 4 procedures for which no NR was dictated. Thus, our results are skewed in the direction of NRs being more complete than in the real-world setting.
Another limitation of this study is that we did not perform veracity checks of the variables studied, as NR completeness rather than NR correctness was the focus of this study. Thus, the fidelity of the NR as a reflection of the actual conduct of the procedure could not be assessed in our study. However, other authors have examined this association using cognitive task analysis. Stewart and colleagues8 found differences in the reporting of key steps of laparoscopic cholecystectomy in the NRs of patients who underwent laparoscopic cholecystectomy and in whom bile duct injury occurred. In patients who had a bile duct injury — a “bad” outcome — key elements of the surgical procedure were omitted, suggesting that the completeness of NRs reflects differences in the quality of the procedure performed.
Implications
Owing to the inadequacies of NRs that we have reported, we recommend the use of SRs to complement or replace NRs as a quality initiative. An SR may improve the completeness and quality of reports by minimizing inconsistent, inaccurate or missing information transfer between care providers, which can lead to suboptimal patient care. Several studies report that SRs provide more complete information than the NRs.4,23–27 In other specialties, particularly in pathology, synoptic reporting has been widely embraced, which has improved interdisciplinary communication and led to more effective coordination of clinical care for individual patients.28–30 There is wide acceptance of SRs by clinicians who prefer the readability of SRs over NRs.27–30 Further structured synoptic reporting results promote quality by standardizing the reporting processes among patients and institutions. Several jurisdictions mandate the use of synoptic pathology reports as a performance indicator.23,31–34
Little research has been conducted to evaluate the potential benefits and/or limitations of synoptic reporting of operative procedures.2,4–6,8 However, owing to the potential benefits of SRs, many groups are developing and using SRs at the institutional level as quality improvement strategies. But there are also potential problems with SRs.
A major objection is that SRs, with their pro forma structure, may not be flexible enough for some procedures, particularly complex cancer procedures, that do not lend themselves to standardization. For these cases, SRs may not be able to accurately reflect the details of the procedure. However, Park and colleagues35 have recently developed and implemented an electronic SR for pancreatectomy. They established that an SR is feasible and acceptable to surgeons, even for this complex, multistep procedure. In their study, the mean time for SR completion was only 4 ± 1.6 minutes per case. Furthermore, the SR document was more complete and reliable than NRs.35 A possible remedy to the structure of the SR is to include an optional free text field. This would allow nonstandardizable information to be included within an SR.
Other objections to SRs that have been suggested is that they can be difficult to complete, take longer to complete than NRs and add to the surgeon’s workload. However, several studies have reported that SRs take less time to complete than NRs.28,35 This suggests that the surgeon’s workload is actually decreased with SRs. Thus, the perception of SRs being more work is likely related to poor implementation strategies and/or existing knowledge gaps rather than being an intrinsic property of SRs.
In addition, the associations between SRs and patient outcomes, such as complication rate, positive margin rate and/or survival, have not yet been established. A few studies suggest that the quality of documentation is directly related to the quality of surgery,8 but the more likely mechanism for quality improvement is through more efficient communication of information among care providers, which allows for optimal treatment recommendations. However, more research should be done in this area. This topic was beyond the scope of this present study, which focused on the completeness of NRs. However, in future work, we will examine the association between the format (NR v. SR), quality of documentation and patient-related outcomes.
Conclusion
Unstructured NRs for cancer surgery are seldom complete and are of poor quality. Clinically important variables are frequently missing from NRs. As a result, an NR cannot be used as a data source for quality assurance purposes. Similar considerations also limit the use of the NR for research and medicolegal applications. Development and use of an SR should be encouraged to improve documentation of care and serve as a measure of quality of surgical care.
Acknowledgements
We thank the HPB Surgeons at the University Health Network: Bryce Taylor, MD; David Grant, MD; Mark Cattral, MD; Ian McGilvray, MD, PhD; Paul Grieg, MD; Carolanne Moulton, MD, PhD; Sean Cleary, MD, MSc; and Steven Gallinger, MD, MSc, for contributing cases, as well as Ms. Marina Englesakis for her assistance performing literature reviews, Mr. Harden Huang and David Chan for informatics support and Ms. Stacey Stegienko for administrative support.
Footnotes
Competing interests: None declared.
Contributors: M.E. Wiebe, L. Sandhu, E.D. Kennedy, N.N. Baxter, A.R. Gagliardi, D.R. Urbach and A.C. Wei designed the study. M.E. Wiebe, L. Sandhu and A.C. Wei acquired the data. M.E. Wiebe, J.L. Takata and A.C. Wei analyzed the data. M.E. Wiebe and A.C. Wei wrote the article. All authors reviewed the article and approved its publication.
- Accepted November 8, 2012.
References
Article tools
Related Articles
Cited By...
- No citing articles found.