Abstract
Doctoral examiners are increasingly assessing theses that include publications by the candidate. Little research has investigated how examiners assess a thesis that includes such publications. Most past research has focused on how examiners assess traditional theses, looking at the criteria that they use, their expectations of the thesis, the types of comments they provide on the thesis, and how they judge the quality of the thesis. Since candidates are now including peer-reviewed publications in their thesis, a question emerges as to how examiners go about assessing this form of thesis. Thus the aim of this study is to determine how examiners assess a publication-based thesis (PBT). A case study approach with mixed methods was used in this study. Data were gathered from examiners at the University of Otago who had experience assessing PBTs over the past ten years. Data were procured through a survey, interviews and examiner reports. The survey data were analysed using a descriptive analysis, while free form comments and interview data were analysed using a general inductive approach. A linguistic analysis was used to identify the nature of commentaries on examiners’ reports.
Prior to the survey, a literature search was conducted to determine the types of PBTs. A PBT refers to a thesis that contains publications of the candidate, with three main types identified: thesis with publications appended, a hybrid thesis where some published work is included as chapters, and PhD by publication that comprises a series of publications.
Collectively the 62 survey respondents had assessed 600 theses over a ten year period, with an estimated 44% being PBTs. Examiners in this study assessed more hybrid theses than theses with publications appended, or PhD by publication. Further, PBTs were more frequently assessed in Health Science and Science than in Commerce and Humanities.
Most examiners faced some issues while assessing PBTs. One issue was the unclear contribution of the candidate in the thesis as the candidate did not necessarily provide any information about his/her contribution. Examiners were also concerned if the candidate was not the first author in the publications. Another issue was a lack of coherence in hybrid PBTs which were sometimes perceived as being fragmented.
Most examiners were positively influenced by high impact publications, but a small number of examiners were not influenced by any form of publications in the thesis. Most examiners in this study also extended their usual set of assessment criteria because they did not receive adequate information regarding the publications and the candidate’s role in these. Their extended criteria included: how much did the candidate contribute in thesis; whether the candidate was the first author in multi-authored publications; and what the other co-authors contributed in the publications.
Most examiners provided a similar amount of commentary on PBTs as they would for a traditional thesis, but their comments on PBTs were mainly minor comments instead of major comments. From the linguistic perspective, the minor comments were mainly focused on feedback than on summative assessment. In terms of feedback, the examiners provided more directive feedback than referential and expressive feedback.
Examiners indicated that doing a PBT gives the candidate a richer learning experience as well a brighter job opportunity in the tight job market. Some of the drawbacks of doing a PBT were the lengthy timeframe of getting a paper accepted on time, the difficulty of publishing in a reputable journal, and having to change writing style between a PBT and the thesis. Also, the limited number of words in a publication may not give the candidate the freedom to explain their research in detail and not hearing the candidate’s voice in the thesis, especially when the thesis includes multi-authored publications.
A key outcome from this study is the presentation of guidelines for doctoral candidates, supervisors, examiners and doctoral administrators who are overseeing the examination process. Additionally, the findings revealed that in order to achieve alignment between the intended learning outcomes and the assessment regime, an oral examination should be part of the examination process.