Publication: Item analysis to improve reliability for an internal medicine undergraduate OSCE
1
Issued Date
2005-06-01
Resource Type
ISSN
13824996
Other identifier(s)
2-s2.0-23844493891
Rights
Mahidol University
Rights Holder(s)
SCOPUS
Bibliographic Citation
Advances in Health Sciences Education. Vol.10, No.2 (2005), 105-113
Suggested Citation
Chirayu Auewarakul, Steven M. Downing, Rungnirand Praditsuwan, Uapong Jaturatamrong Item analysis to improve reliability for an internal medicine undergraduate OSCE. Advances in Health Sciences Education. Vol.10, No.2 (2005), 105-113. doi:10.1007/s10459-005-2315-3 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/16965
Research Projects
Organizational Units
Authors
Journal Issue
Thesis
Title
Item analysis to improve reliability for an internal medicine undergraduate OSCE
Abstract
Utilization of objective structured clinical examinations (OSCEs) for final assessment of medical students in Internal Medicine requires a representative sample of OSCE stations. The reliability and generalizability of OSCE scores provides validity evidence for OSCE scores and supports its contribution to the final clinical grade of medical students. The objective of this study was to perform item analysis using OSCE stations as the unit of analysis and evaluate the extent to which OSCE score reliability can be improved using item analysis data. OSCE scores from eight cohorts of fourth-year medical students (n = 435) in a 6-year undergraduate program were analyzed. Generalizability (G) coefficients of OSCE scores were computed for each cohort. Item analysis was performed by considering each OSCE station as an item and computing the corrected item-total correlation. OSCE stations which negatively impacted the reliability were deleted and the G-coefficient was recalculated. The G-coefficients of OSCE scores from the eight cohorts ranged from 0.48 to 0.80 (median 0.62). The median number of OSCE stations that negatively impacted the G-coefficient was 3.5 (out of a median of 25 total stations). When the "problem stations" were deleted, the median G-coefficient across eight cohorts increased to 0.62 - 0.72. In conclusion, item analysis of OSCE stations is useful and should be performed to improve the reliability of total OSCE scores. Problem stations can then be identified and improved. © Springer 2005.