Abstract
Background: For academic hiring committees and surgical trainees, the benefits of a graduate degree are unclear. We sought to identify if graduate degrees or professorship status were associated with increased research productivity among Canadian academic surgeons.
Methods: We included general surgeons from the largest hospitals associated with accredited residency programs. We classified staff surgeons active between 2013 and 2018 by degree (MD only, master’s degree, PhD) and professorship (assistant, associate, professor) status. We identified their publications from January 2013 to December 2018. Variables of interest included publications per year, citations per article, journal of publication, CiteScore, author’s Hirsch (h) index and the revised h-index (r-index). We used Kruskal–Wallis tests and the Dunn multiple comparison test to assess statistical significance.
Results: We identified 3262 publications from 187 surgeons, including 78 (41.7%) with no graduate degree, 84 (44.9%) with master’s degrees and 25 (13.4%) with PhDs. Surgeons with graduate degrees had more publications per year, higher CiteScores, more citations per article, and higher h- and r-indices than those without graduate degrees. Surgeons with doctorates had the highest median values in all domains, but differences were not significant compared with surgeons with master’s degrees. Seventy-seven (41.8%) surgeons were assistant professors, 63 (34.2%) were associate professors and 44 (23.9%) were full professors. Statistically, full professors had a greater number of publications per year and higher h- and r-indices than their counterparts.
Conclusion: Surgeons with graduate degrees or more advanced professorships had the greatest research productivity. Surgeons with doctorates trended toward greater research productivity than those holding master’s degrees.
Academic general surgery in Canada continues to transform, with additional fellowship or research training becoming increasingly common for surgeons holding academic positions.1,2 More than 60% of surgeons hired in the 2010s have graduate degrees; meanwhile, the pressure to conduct research and drive academic funding is growing, with the number of publications from academic surgeons steadily increasing since the 1980s.1 In response to this, more general surgery residents are pursuing advanced research degrees during their surgical training.1,3,4 The pursuit of these advanced degrees holds a substantial opportunity cost to surgical residents and their residency programs. With a cost of about $65 000 per year per surgical resident undergoing research training, it also contributes to health care expenditure.3 However, long-term academic outcomes for surgeons who hold graduate degrees remain uncertain.
To support resident attainment of graduate degrees, the Royal College of Physicians and Surgeons of Canada (RCPSC) created the accredited Clinician Investigator Program (CIP) in 1995.5 Unsurprisingly, surgical residents who obtain a graduate degree during residency publish more frequently, publish in higher-impact journals and are more likely to be first authors during their training.5–7 General surgery residents who obtain a PhD during residency publish more frequently and in higher-impact journals during their training than those with a master’s degree.5 However, a recent publication suggested that, although completion of a CIP-like program leads to increased research productivity during residency, it did not increase the proportion of residents pursuing an academic career.6 In a previous study of a cohort of Canadian surgeons, more than half the general surgeons with doctorate degrees did not publish as last authors after training,5 suggesting a less clear association between graduate degrees obtained during residency and future research productivity.
Considering the diversity of general surgery practices across Canada, which spans both community and academic sites, and the research requirements of many surgical programs, it is important to quantify the academic benefits provided by higher-level graduate degrees. For academic hiring committees and general surgery trainees alike, the long-term benefits of pursuing a graduate degree remain unclear, particularly the benefits of a PhD relative to a master’s degree.8 For general surgery residents, one of the most common resident types to participate in CIPs, understanding the long-term academic benefits is of utmost importance as they consider prolonging their training to pursue these advanced degrees.9 Similarly, this information is crucial for academic hiring committees who are tasked with selecting surgeons from a pool of well-qualified applicants, often for positions with dedicated research time or research goals.
We sought to better inform surgical residents, training programs and hiring committees who are tasked with meeting the continually demanding academic landscape for surgeon scientists across Canada. The primary objective of this study was to compare the academic productivity and publication impact of Canadian general surgeons without advanced degrees to those with master’s or PhD training. We also assessed for associations between professorship status and academic productivity.
Methods
We conducted a retrospective, observational, comparative study to evaluate the associations between graduate degree status or academic professorship status and research impact among academic general surgeons at the largest academic centres in Canada. We adapted our methods and search criteria from a previous study that evaluated the association between graduate degree status and research productivity among Canadian neurosurgeons.10
We included general surgeons from each of the largest hospitals associated with English-speaking, RCPSC-accredited, general surgery residency training programs. We defined the largest hospital associated with each program as the site with the most general surgeons on staff in November 2020. In cases where an equal number of surgeons were employed in 2 affiliated hospitals, we defined the site with more publications as the larger hospital. We obtained lists of general surgeons at these sites from program and provincial medical registrar websites and confirmed them by contacting current general surgery residents within each program. We excluded retired surgeons. Surgeons hired during the study period were included for the duration of their time as staff. For recently hired surgeons, we rounded their number of active years down to the nearest year to allow a transition period (e.g., if a surgeon was hired and worked 3 years and 11 months, we excluded the first 11 months of their employment from data analysis). We obtained dates of employment from provincial medical registrar and university websites. We calculated data affected by number of years active based on their number of years as staff. We used Theses Canada, current publications, university websites and information gathered by contacting surgeon offices directly to first categorize surgeons as MD only, MD–master’s or MD–PhD.11 We also categorized surgeons by professorship status (assistant, associate or full professor) from information obtained through university websites.
Publications for identified surgeons were retrieved from Scopus between Nov. 23 and Nov. 25, 2020.12 An experienced medical librarian performed the search in a reproducible manner. A second search was performed on Feb. 10, 2021, to retrieve information of 6 authors recognized to have incomplete data from the initial search. To capture a sample of research productivity, we limited searches to 2013–2018 to retrieve papers within a 6-year period, taking into consideration fluctuations in the research life cycle and allowing a minimum of 2 years for publications to be cited as of 2020. Furthermore, publication metrics are heavily manipulated by the number of years active, so we calculated these based on surgeons’ work in the previous 6 years. In rare cases, this included research performed in residency. We retrieved publications for all hospital affiliations for each author to capture as much of their research output as possible within the specified date range. We manually screened publications for accuracy with errata, and eliminated duplicates.
For each surgeon, we collected number of publications, authorship position for each paper (e.g., first, middle, last), number of citations per paper, 6-year Hirsch (h) index, number of authors per paper and CiteScore of journals for the included research.13 The h-index is a metric used to quantify a researcher’s academic productivity; it has been cited more than 10 000 times since its inception in 2005.13 An author’s h-index is described as the number of papers published by the author with greater than or equal to “h” citations. For example, if an author has published 5 papers and 4 of those papers have 5 or more citations, whereas the fifth paper only has 2 citations, this author’s h-index will be 4. This index is designed to allow for an indication of both author productivity and perceived impact.14 The CiteScore is a metric that is based on the yearly average number of citations of recent articles in the journal based on a 3-year citation window, with higher scores indicating a greater number of citations. In addition, we calculated a 6-year revised h-index (r-index), as described by Romanovsky,15 where first- or last-authorship publications are weighed 4 times as heavily as middle-authorship publications to account for differences in the levels of responsibility correlated with different authorship order. This r-index has been suggested for use with biomedical research.15
Statistical analysis
Our primary outcome was the association of graduate degree status (MD only, MD–master’s and MD–PhD) with number of publications per year of work during the study period. Secondary outcomes evaluated publication impact and included association of graduate degree status with the number of publications in specific authorship roles (e.g., first, middle, last author), number of citations, h-index, r-index and CiteScore. We conducted a secondary analysis evaluating the association between professorship status and these same primary and secondary outcomes. We analyzed data with and without outliers removed. Outliers were defined as surgeons with values greater than 2 standard deviations above the median annual number of publications in their associated group.
We completed statistical analysis using Prism (version 8, GraphPad). We used the D’Agostino–Pearson normality test to determine the need for nonparametric testing, which was required for all subsequent analyses. Between-group comparisons of data were carried out using the nonparametric Kruskal–Wallis test with the Dunn multiple comparisons test. All data are presented as medians with interquartile ranges (IQR). Statistical significance (α) was set at 0.05. We defined our research methods in a presearch protocol, and we did not deviate from our outcomes of interest.
Results
After assessment of the number of surgeons per academic centre in Canada, we included the following hospitals: Vancouver General Hospital (Vancouver), Foothills Medical Centre (Calgary), University of Alberta Hospital (Edmonton), Royal University Hospital (Saskatoon), Health Sciences Centre (Winnipeg), Victoria Hospital (London, Ontario), St. Joseph’s Hospital (Hamilton, Ont.), Sunnybrook Health Sciences Centre (Toronto), Health Sciences North (Sudbury, Ont.), Kingston General Hospital (Kingston, Ont.), Ottawa General Hospital (Ottawa), Royal Victoria Hospital (Montréal), Queen Elizabeth II Health Sciences Centre (Halifax) and Health Sciences Centre (St. John’s). We excluded the hospitals affiliated with the Université Laval (Québec) and Université de Sherbrooke (Sherbrooke, Que.), as their primary language of education is French. Within included hospitals, 187 (range 8–22 per hospital) general surgeons were identified, with a total of 3262 publications between 2013 and 2018. The number of publications per surgeon during the 6-year study period ranged from 0 to 182 publications.
After categorization of surgeons by their academic degree status, we found that most staff surgeons were MD only (n = 78, 41.7%) or MD–master’s (n = 84, 44.9%), with fewer MD–PhD surgeons (n = 25, 13.4%; Figure 1). Surgeons categorized as MD–master’s (median 2.8, IQR 1.0–5.9) and MD–PhD (median 3.7, IQR 0.9–5.8) surgeons had a statistically similar number of publications per year (p > 0.99), but both had significantly more than MD-only surgeons (median 0.7, IQR 0.2–1.5; p < 0.001; Figure 2). Similarly, MD–master’s and MD–PhD surgeons had a significantly greater number of first authorships, middle authorships, last authorships, 6-year h- and r-indices, and higher publication CiteScores than MD-only surgeons (Figure 2). The average number of years worked was similar by degree status, as was the number of authors per paper (Table 1). Comparing MD–master’s surgeons to MD–PhD surgeons, there was no statistical significance in any outcome measure, although MD–PhD surgeons did have higher median values in all measured domains (Table 1). The MD-only surgeons were either first or last author for 25.0% of their publications, compared with 39.4% for the master’s group and 41.2% for the PhD group (medians calculated after surgeons with 0 publications were excluded). Only 22 (11.8%) surgeons had no publications in the study period (Appendix 1, Table 1S, available at canjsurg.ca/lookup/doi/10.1503/cjs.010121/tab-related-content). Of the 42 surgeons hired between 2013 and 2018, 11 (26.2%) had no graduate degree, 27 (64.3%) had a master’s degree and 4 (9.5%) had a PhD. In the cohort of 145 surgeons who were hired before 2013, 67 (46.2%) had no graduate degree, 57 (39.3%) had a master’s degree and 21 (14.5%) had a PhD.
Flow diagram of degree and professorship status for all included surgeons.
Research productivity of 187 general surgeons in Canada, stratified by graduate degree status. h = Hirsch index, IQR = interquartile range, r = revised Hirsch index. *p < 0.05, †p ≤ 0.0001.
Research productivity from 2013 to 2018 of 187 general surgeons in Canada, stratified by graduate degree status
After exclusion of 12 outliers, (Appendix 1, Table 2S) 175 surgeons remained (74 MD only, 79 MD–master’s, 22 MD-PhD). Results were similar after this exclusion, with MD–master’s (median 2.3, IQR 1.0–4.7) and MD–PhD (median 3.4, IQR 0.5–5.0) surgeons having a statistically similar number of publications per year (p > 0.99), but both groups having significantly more publications than MD-only surgeons (median 0.6, IQR 0.2–1.3; p < 0.0001; Appendix 1, Figure 3S). Similarly, MD–master’s and MD–PhD surgeons had significantly more first-, middle-and last-authorship publications, as well as statistically higher h- and r-indices, than their MD-only colleagues (Appendix 1, Table 3S).
Categorization of surgeons by their professorship status showed that 77 (41.8%) were assistant professors, 63 (34.2%) were associate professors and 44 (23.9%) were full professors (Figure 1). Three surgeons from our initial cohort were excluded, as 2 were clinical lecturers and 1 had no formal professorship status at the time of data collection. Faculty advancement was associated with the number of publications per year, number of last- and middle-authorship publications, h-index and r-index, with associate professors having significantly more publications and higher indices than assistant professors, and professors having significantly more publications and higher indices than both groups (Figure 3). Professors and associate professors had a statistically similar Cite-Score, both of which were statistically higher than than assistant professors (Table 2). After exclusion of 14 outliers based on academic position, 170 surgeons remained (72 assistant professors, 58 associate professors and 40 professors) (Appendix 1, Table 4S). Evaluating this subgroup showed similar results as when outliers remained (Appendix 1, Table 5S).
Research productivity of 184 general surgeons in Canada, stratified by academic professorship status. h = Hirsch index, IQR = interquartile range, r = revised Hirsch index. *p < 0.05, †p ≤ 0.0001.
Research productivity from 2013 to 2018 of 184 general surgeons in Canada, stratified by academic professorship status
Discussion
The landscape for general surgeons continues to transform. There remains a need for clinician scientists, surgeons with an educational focus, clinical specialists, surgeons with administrative roles and community surgeons across Canada. As residents try to direct their training toward specific career paths, and hiring committees hope to select appropriate surgeons for the positions they are tasked with filling, understanding the impact of a graduate degree on future research productivity is important.
Achievement of a graduate degree was associated with increased research productivity as a staff general surgeon. We observed a marked increase in scholarly output between those without a graduate degree and those with a master’s or PhD. Those with a PhD publish a median of 3 more manuscripts per year than MD-only surgeons, and those with a master’s degree publish about 2 more manuscripts on an annual basis. According to our data, surgeons holding PhD degrees seem to publish more than those holding master’s degrees, but this difference was not statistically significant. These are similar findings to a study on Canadian neurosurgeons; using similar metrics, the timing of degree acquisition (before, during or after residency) did not seem to affect findings in this study.10
Increased research productivity was associated with academic advancement. Full professors published the most (median 4.2 manuscripts/yr), followed by associate professors (median 2.0 manuscripts/yr) and assistant professors (median 0.7 manuscripts/yr). These values were statistically different from one another. In a national survey of general surgery program directors in the United States, research productivity was deemed very important to surgeons pursuing academic faculty positions and less so for those pursuing community surgery.3 In our study, surgeons with full professorships exceeded those with associate or assistant professorships in nearly every measured publication domain. This suggests there is no lag in productivity from loss of motivation as surgeons reach full professorship.
Research impact spans far beyond the raw number of authored manuscripts. To quantify the academic output of our included surgeons, we identified a variety of secondary outcomes. Surgeons with MD–PhDs had the highest CiteScores and h-indices, suggesting a higher research impact, followed by surgeons with MD–master’s and, lastly, by MD-only surgeons. The differences between the surgeons with doctorate and master’s degrees did not reach statistical significance. This strengthens the argument that surgeons with graduate degrees not only publish more frequently, but also publish in journals cited more frequently. The median h-index over the 6-year study period among surgeons with PhD degrees was 2.0 points greater than those with master’s and 8.0 points greater than those without graduate degrees; the median CiteScore for those with a PhD was 0.3 and 1.2 points greater than those with a master’s or no graduate degree, respectively. The h-index effectively eliminates bias attributed to highly cited, individual publications or a large volume of low-impact publications, which provides strength to these findings.16 We obtained surgeons’ h- and r-indices from the 6 years included in the study in an attempt to reduce bias favouring higher indices of older surgeons who have had more time in their career to publish and be cited. Surgeons with PhD degrees had the most publications as last author and the highest r-indices, followed by those with master’s degrees and then MD-only surgeons. This finding mirrors results from Keough and colleagues,10 who suggested that neurosurgeons with higher-level graduate degrees are also those driving research in their field. An important drawback of the h-index is that all authorship positions are weighted equally.15 This does not apply to biomedical research, where authorship position often reflects input into the manuscript.15 Among surgeon scientists, it is not uncommon that large research groups work together to provide a breadth of experience, but that the bulk of work is conducted by the first and last authors. The r-index “places more value on conducting and directing original, independent research as compared with contributing to research projects conducted and directed by others.”15 In our study, the difference in r-index values between surgeons with graduate degrees compared with MD-only surgeons was even greater than that of h-index values, suggesting that surgeons with advanced degrees are more likely to be major contributors in their publications, rather than named in the study based on status or involvement in large research or clinical groups. We considered evaluation of surgeons with other research-defining indices, such as the g-index, which gives greater credit to authors who have authored manuscripts with outstanding citation numbers, but determined that this could skew our results if an author published 1 highly cited manuscript during our relatively short study period.17
Despite potentially increasing long-term research productivity, impact and likelihood of achieving academic positions, surgical residents should recognize drawbacks of pursuing a graduate degree during their training. It presents a financial opportunity cost as training is prolonged by 3–4 years for residents pursuing doctorate studies. Previous research also shows that dedicated research time during residency is a period of vulnerability and that these trainees report less satisfaction in various fields of their training.18 In addition, although our study supports an association between increased academic productivity, impact and academic advancement, a major confounder of our findings is the time that general surgeons are afforded toward research and clinical work, which likely differs substantially by academic site. In a survey of CIP alumni, 67% reported that more than half of their work hours are dedicated to research.9 It is difficult to determine whether their research productivity led to academic positions with protected time or whether protected time has led to increased research productivity and impact. Furthermore, the link between protected research time and research productivity is unclear, limiting researchers’ ability to explain why surgeons with graduate degrees publish more than their colleagues without graduate degrees.8,19 Unfortunately, with our current methodology, we were unable to elucidate the weight that each of these variables had on research productivity. It is likely that surgeons who pursued graduate degrees were more likely to have an interest in pursuing research in their careers, which is difficult to quantify. In our analysis, we attempted to reduce the possibility that surgeons with substantial time dedicated to research led to skewed data by reporting results with outliers excluded, which showed similar results to those with the full study sample. Our data were skewed to the right in most fields, as was predicted given that key individuals contribute unevenly to academic publications. Regardless, it appears that, for general surgeons in Canada, completing a graduate degree is associated with greater research productivity. This is in keeping with previous data suggesting that 75% of CIP alumni continue with research as academic surgeons and 74% receive external funding as a staff surgeon.9 Beyond this, we found that 73.8% of the surgeons in our sample population who were hired between 2013 and 2018 had graduate degrees, compared with 53.8% hired before 2013, emphasizing the importance of graduate degrees in the pursuit of academic general surgery positions in Canada.
Limitations
We obtained only publications indexed in Scopus, identified by author affiliations aligned with hospital centres. There is potential error in the data analysis from inconsistencies of how author names are published in journals, which we attempted to overcome by selecting all applicable author titles. Using quantitative metrics to evaluate research productivity, including citation counts and CiteScores, has numerous limitations, such as not knowing the context of why and how publications were cited. Furthermore, the CiteScore is a journal-level metric that can be used only as a proxy for evaluating papers published therein, and does not provide a direct measure of impact. The number of authors per paper ranged from 1 to 784 authors, and determining the surgeons’ contribution to their publications was beyond the scope of the current study. Finally, despite our statistically significant results, we were able to describe only trends and were unable to determine causality between graduate degrees and productivity.
Conclusion
Among general surgeons in Canada, research productivity and impact was greater among surgeons with a graduate degree than among those without a graduate degree. Surgeons with doctorates appeared to have more publications and a greater perceived research impact than those with master’s degrees, but this difference was not statistically significant. In addition, surgeons with full professorships had higher productivity than associate and assistant professors. This information is of interest to surgical residents considering a career as clinician investigators and hiring committees hoping to select the most appropriate applicants for academic positions.
Acknowledgement
The authors thank Dr. Michael B. Keough for sharing his experience with a similar project, which has been cited throughout the manuscript.
Footnotes
This research was presented at the University of Alberta’s General Surgery Resident’s Research Day in Edmonton, Alberta, Canada on Apr. 23, 2021, and at the Canadian Surgery Forum, Sept. 21–24, 2021.
Competing interests: None declared.
Contributors: Kieran Purich, Kevin Verhoeff, Janice Kung, James Shapiro and David Bigam contributed to the conception and design of the work. Alexander Miles acquired the data, which Kieran Purich and Kevin Verhoeff analyzed and interpreted. Kieran Purich, Kevin Verhoeff, Alexander Miles and Janice Kung drafted the manuscript. All authors revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
- Accepted August 8, 2021.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.