The use of global rating scales for OSCEs in veterinary medicine.
Authors: Read Emma K, Bell Catriona, Rhind Susan, Hecker Kent G
Journal: PloS one
Summary
# Editorial Summary: Global Rating Scales in Veterinary Clinical Skills Assessment Objective Structured Clinical Examinations (OSCEs) form a cornerstone of veterinary education, yet debate continues over whether standardised binary checklists or multidimensional global rating scales (GRS) provide more reliable assessment of clinical competence. Read and colleagues compared these two scoring methods across 12 raters (from University of Calgary and Royal Dick School of Veterinary Studies) evaluating 12 students performing four practical tasks—bovine haltering, gowning/gloving, equine bandaging, and skin suturing—using generalizability theory and ANOVA to determine scoring reliability. GRS scores proved notably more reliable than checklists for novice raters (0.70 vs 0.42 respectively), whilst maintaining superior reliability for experienced assessors (0.86 vs 0.76), suggesting that GRS may better capture the nuanced judgement required in clinical skill evaluation. Importantly, the study demonstrated that novice raters trained on checklists could subsequently use GRS with confidence, indicating that institutions need not restrict GRS assessment to expert examiners. For equine professionals involved in teaching or assessment, these findings suggest that multi-dimensional evaluation tools capturing overall clinical reasoning and technique—rather than rigid task completion lists—offer more dependable measurement of competence and may be particularly valuable when training newer assessors.
Read the full abstract on PubMed
Practical Takeaways
- •If your school or training program uses OSCEs to assess clinical skills, GRS tools will provide more reliable and consistent scoring than simple checklists, particularly for less experienced evaluators
- •Initial training on checklist-based assessments can effectively prepare raters to transition to GRS scoring without loss of reliability
- •GRS should be considered the preferred method for evaluating competency in practical skills like bandaging, suturing, and animal handling
Key Findings
- •Global rating scales (GRS) demonstrated superior reliability compared to checklists for both novice (0.70 vs 0.42) and expert raters (0.86 vs 0.76) in OSCE assessments
- •Checklist scores were significantly higher than GRS scores across all rater groups
- •GRS reliability improved with rater training and experience, with expert raters achieving 0.86 reliability compared to novice raters at 0.70