In reality, high quality oral examinations of medical students with precise pre-definition of relevant patient-oriented tasks and of the respective expectation levels cannot be taken for granted and are difficult to develop and to implement. Therefore, a structured training program for oral examiners in respect to the final medical exam has been implemented at the University of Ulm.
According to the present survey the trained examiners had the impression that passing of incompetent candidates is infrequent, but nevertheless an existing phenomenon during the oral part of the final exam. As the number of failing candidates in the oral exam is rather low, this may speak in favour of the candidates’ competence in general and consequently in favour of the quality of present curriculum. These results correlate well with the results of the written part of the final exam, centrally designed for the whole country, where the number of failing candidates is also rather low. For example, in fall 2011 182 students of the Medical Faculty of the University of Ulm participated in the written part of the final exam, with only three candidates failing. In the oral part none of them failed. A study performed by Seyfarth et al. (2010) compared the grades on the oral and written components of the final medical exam and proposed an improved concordance between the two components since 2002, when the actual German national medical licensing regulations came into force .
The dark side is represented by the fact that more than half of the examiners participating in our study nevertheless already had the experience of seeing incompetent candidates be passed on their oral exam. Passing incompetent candidates might endanger patient care and health. Consequently, the participating colleagues request to examine the candidates in the final exam more critically, by means of concrete clinical examples and clearly defined grading criteria. This is consistent with findings from the 1990′s that it seems to be far more difficult to rate bad or borderline performances during oral examinations than to rate good performances .
Surprisingly, the answers of the study participants belonging to operative disciplines turned out to differ significantly from the answers of the participants from non-operative disciplines. One rather provocative explanation for these differences could be that so far the colleagues from operative fields have not been familiar enough with the didactic theories concerning oral examinations and that subsequently higher learning and training effects could be achieved. On the other hand, for the colleagues from operative disciplines the trainings did not only lead to individual learning and training effects, but also to examination-related improvements in their respective departments. This might indicate that the surgical participants handled the newly acquired competences in a very active way.
A certain limitation of the study is the relatively low number of participants: more than 300 persons were addressed to participate in present survey, but only 63 persons answered the questionnaire – although factors reported to enhance the response rate were specifically addressed in the study, such as survey length (the questionnaire focused on only 28 items), design issues (clear layout), and research affiliation (cover letter by the Dean of Education of the Medical Faculty). This is partly explainable by the known high turnover of staff at University hospitals and by the well-known time restrictions of the target group, as recently outlined in this journal , and partly by the relative high frequency of such electronic surveys, leading to a certain “survey-fatigue” of the potential participants; response rates to online surveys have significantly decreased since 1986 as an effect of the population being “oversurveyed” [13, 14].
The fact that workshop participation was voluntary might also have slightly biased the study results. When the training was initiated in 2007, the Medical Faculty voted for a bonus system instead of an obligatory participation; the bonus system offers a small financial incentive not to the attendees but to the respective departments. The word-of-mouth recommendation and the rather positive feedback of former participants result in consistent high participation numbers (also on the base of high staff turnover at the University Hospital). Meanwhile, almost all examiner novices participate in the training, either by intrinsic motivation or sent by the chairmen of their department.
Another limitation of the study results from the fact that answers to the survey items are based on self-assessment of the participants. The quality of self-assessment with its tendency towards under- or overestimation of competence has been discussed very differently throughout the literature, but trainings with expert feedback (as performed in present examiner trainings) have been reported to have the capacity to generate a good relation of self-assessment and objective reality [15–17].
Furthermore, the use of untrained examiners as an “objective” control group in the high-stakes situation of the final exam could not be considered a reasonable and acceptable alternative.