Validity and Reliability of the Multiple-Mini Interview (MMI) in Medical Student Selection: A Comparison of Onsite and Online Platforms

Abstract

Objective: The objective of this study was to compare the administration of Multiple-Mini Interview (MMI) between the online and onsite platforms, in terms of validity evidence. Materials and Methods: We retrospectively reviewed records of past MMI scores, examination materials, and participant questionnaire responses over a six-year period at the Faculty of Medicine Siriraj Hospital, Mahidol University. A total of eight MMI administrations were included, four onsite and four online. Validity evidence was assessed based on three key sources: test content, response processes, and internal structure. Results: Over six years, eight MMIs were conducted, with 237 out of 340 candidates admitted to a medical school. Content analysis of the test specification tables indicated that both onsite and online platforms adequately addressed similar objectives. Participant satisfaction ratings were comparable between onsite and online MMIs. Qualitative analysis revealed minor issues in a few stations regarding clarity of instructions and scoring criteria. Additionally, some online MMI stations showed discrepancies between task time requirements and allotted time, and more technical issues were also reported. Score analysis showed that both highest and average scores from online MMIs were slightly lower than those from onsite MMIs. However, both formats yielded moderately reliable test scores (Cronbach’s Alpha of 0.49 - 0.51). Conclusion: The online MMI is a viable alternative to the traditional onsite MMI. Both platforms effectively covered the same assessment objectives, yielded comparable reliability and participant satisfaction.