Publication: Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents
Issued Date
2009-09-01
Resource Type
ISSN
13824996
Other identifier(s)
2-s2.0-70349497458
Rights
Mahidol University
Rights Holder(s)
SCOPUS
Bibliographic Citation
Advances in Health Sciences Education. Vol.14, No.4 (2009), 575-594
Suggested Citation
Cherdsak Iramaneerat, Carol M. Myford, Rachel Yudkowsky, Tali Lowenstein Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents. Advances in Health Sciences Education. Vol.14, No.4 (2009), 575-594. doi:10.1007/s10459-008-9142-2 Retrieved from: https://repository.li.mahidol.ac.th/handle/20.500.14594/28360
Research Projects
Organizational Units
Authors
Journal Issue
Thesis
Title
Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents
Abstract
The investigators used evidence based on response processes to evaluate and improve the validity of scores on the Patient-Centered Communication and Interpersonal Skills (CIS) Scale for the assessment of residents' communication competence. The investigators retrospectively analyzed the communication skills ratings of 68 residents at the University of Illinois at Chicago (UIC). Each resident encountered six standardized patients (SPs) portraying six cases. SPs rated the performance of each resident using the CIS Scale-an 18-item rating instrument asking for level of agreement on a 5-category scale. A many-faceted Rasch measurement model was used to determine how effectively each item and scale on the rating instrument performed. The analyses revealed that items were too easy for the residents. The SPs underutilized the lowest rating category, making the scale function as a 4-category rating scale. Some SPs were inconsistent when assigning ratings in the middle categories. The investigators modified the rating instrument based on the findings, creating the Revised UIC Communication and Interpersonal Skills (RUCIS) Scale-a 13-item rating instrument that employs a 4-category behaviorally anchored rating scale for each item. The investigators implemented the RUCIS Scale in a subsequent communication skills OSCE for 85 residents. The analyses revealed that the RUCIS Scale functioned more effectively than the CIS Scale in several respects (e.g., a more uniform distribution of ratings across categories, and better fit of the items to the measurement model). However, SPs still rarely assigned ratings in the lowest rating category of each scale. © Springer Science+Business Media B.V. 2008.