Background Integrity concerns around written exams have intensified with the rise of Generative AI technology. Many institutions are now introducing or expanding interactive oral assessments (IOAs) in health professions education (HPE), especially in Anglo-Saxon systems where written and practical exams dominate. While IOAs may address integrity issues, questions remain about their scalability and acceptability. Italian universities, with decades of consistent IOA use across all educational programs, are well-suited to investigate these concerns. We examined students’ and faculty's perspectives on IOAs in Dentistry-in-English and Medicine-in-English courses at our institution. Summary of work Within a constructivist paradigm, two anonymous online surveys were developed: one for current medical and dental students and one for faculty. The surveys explored perspectives on IOAs using open-ended questions and Likert-scale ratings. Circulated by the Program’s Dean, the surveys’ qualitative responses underwent thematic analysis, while quantitative data were analysed descriptively. To allow for deeper exploration of findings and member checking, the “Course F for Fair Exam Experience (CoFFEE)” event was organised – a townhall-style event centred on collaboration and inclusivity. Preliminary interpretations were shared, and participants discussed challenges and solutions. Summary of results At the end of data collection, surveys were completed by 188 students (~51% response rate) and 93 faculty members (~66% response rate). Following quantitative analysis, students rated IOAs as reasonably valid but reported reliability concerns due to inter-rater variability, emotional stress, and logistical challenges (delays, scheduling issues). Faculty highlighted the linguistic challenge of examining non-native English speakers in English, and echoed the influence of examinee stress. Thematic analysis identified the need for empathetic and structured assessment practices. Both groups advocated for standardised evaluation, mock exams, and clear communication. Member checking reinforced the need to improve reliability, create shared understandings and foster mutual respect and intercultural sensitivity. Discussion and Conclusion Our findings highlight the strengths and challenges of IOAs. Students and faculty recognised the value of IOAs for assessing critical skills but stressed the need for increased reliability and logistical efficiency. The CoFFEE event underscored the importance of student-faculty dialogue in refining assessment practices. Take home message Standardised procedures, guidelines, and student-faculty collaboration could improve the implementation of IOAs and educational experiences to meet the evolving global needs in HPE.
Oral Exams: Balancing Assessment Integrity with Fairness – Lessons from Italian Medical Education / Melcarne, R.; Yucel, S.; Vitale, S.; De Benedictis, D.; Leopardi, E.; Violani, C.; Familiari, G.; Maranghi, M.; D'Amati, Giulia. - (2025). (Intervento presentato al convegno Convegno Sapienza "Evidence based education e didattica universitaria. Teorie, metodi e strumenti per la Qualità dell'Inclusione e dell'Innovazione." tenutosi a roma).
Oral Exams: Balancing Assessment Integrity with Fairness – Lessons from Italian Medical Education.
Melcarne R.
Primo
;Yucel S.;Vitale S.;De Benedictis D.;Leopardi E.;Violani C.;Familiari G.;Maranghi M.;D’Amati Giulia
2025
Abstract
Background Integrity concerns around written exams have intensified with the rise of Generative AI technology. Many institutions are now introducing or expanding interactive oral assessments (IOAs) in health professions education (HPE), especially in Anglo-Saxon systems where written and practical exams dominate. While IOAs may address integrity issues, questions remain about their scalability and acceptability. Italian universities, with decades of consistent IOA use across all educational programs, are well-suited to investigate these concerns. We examined students’ and faculty's perspectives on IOAs in Dentistry-in-English and Medicine-in-English courses at our institution. Summary of work Within a constructivist paradigm, two anonymous online surveys were developed: one for current medical and dental students and one for faculty. The surveys explored perspectives on IOAs using open-ended questions and Likert-scale ratings. Circulated by the Program’s Dean, the surveys’ qualitative responses underwent thematic analysis, while quantitative data were analysed descriptively. To allow for deeper exploration of findings and member checking, the “Course F for Fair Exam Experience (CoFFEE)” event was organised – a townhall-style event centred on collaboration and inclusivity. Preliminary interpretations were shared, and participants discussed challenges and solutions. Summary of results At the end of data collection, surveys were completed by 188 students (~51% response rate) and 93 faculty members (~66% response rate). Following quantitative analysis, students rated IOAs as reasonably valid but reported reliability concerns due to inter-rater variability, emotional stress, and logistical challenges (delays, scheduling issues). Faculty highlighted the linguistic challenge of examining non-native English speakers in English, and echoed the influence of examinee stress. Thematic analysis identified the need for empathetic and structured assessment practices. Both groups advocated for standardised evaluation, mock exams, and clear communication. Member checking reinforced the need to improve reliability, create shared understandings and foster mutual respect and intercultural sensitivity. Discussion and Conclusion Our findings highlight the strengths and challenges of IOAs. Students and faculty recognised the value of IOAs for assessing critical skills but stressed the need for increased reliability and logistical efficiency. The CoFFEE event underscored the importance of student-faculty dialogue in refining assessment practices. Take home message Standardised procedures, guidelines, and student-faculty collaboration could improve the implementation of IOAs and educational experiences to meet the evolving global needs in HPE.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


