Authenticity in Residency OSCE: Balancing Realism, Fairness, and Defensibility

Arab Board of Health Specializations (ABHS)

General Secretariate/Assessment Department

Sabeeh AL Mashhadani

Abstract

The Objective Structured Clinical Examination (OSCE) is a widely adopted method for assessing clinical competence in residency programs. Concerns about its artificial nature have led to calls for greater authenticity. In this paper, authenticity in OSCE is defined as making the exam look, feel, and assess like real clinical work, while preserving fairness and defensibility. We explore how integrated-task OSCEs, Entrustable Professional Activities (EPAs), programmatic assessment, and global judgments contribute to authenticity in postgraduate medical training. Recent literature highlights the value of entrustment-based approaches in aligning OSCEs with workplace practice and supporting readiness-for-independence decisions.

Introduction

Since the mid-1970s, when Harden and colleagues first formalized the OSCE model, structured clinical examinations have been widely adopted as a cornerstone of medical assessment (1). The innovation offered a systematic way to observe clinical skills across multiple standardized stations, but over time educators have noted limitations when applying this format in postgraduate training (2).

Defining Authenticity in Residency OSCE

In residency programs, authenticity in OSCEs is best understood as designing stations that simulate the kinds of responsibilities residents actually face in clinical environments. Instead of narrowly scripted scenarios, authentic OSCEs ask residents to demonstrate how they integrate clinical reasoning, decision-making, and communication under conditions that resemble real patient care (4,5).

Programmatic Assessment in Residency

In postgraduate training, OSCEs should be situated within a programmatic assessment framework (6). Rather than serving as a standalone hurdle, OSCEs act as standardized benchmarks that complement workplace-based assessments, in-training evaluation reports (ITERs), and portfolios. This integration enhances validity and fairness by ensuring that OSCE performance is considered alongside longitudinal evidence of competence.

Entrustable Professional Activities (EPAs) and OSCE Design

At the residency stage, the central question of assessment becomes one of trust: can the learner take on clinical responsibilities without direct supervision? Entrustable Professional Activities (EPAs) provide a framework for answering this question. When OSCE stations are built around EPAs—such as leading a ward handover or managing an acutely ill patient—they not only assess technical skill but also readiness for independent clinical practice (7–10).

Global Judgments versus Checklists

While checklists provide transparency and standardization, they risk reducing competence to mechanistic task completion. Evidence suggests that examiner global judgments and entrustment scales often correlate more strongly with real-world performance in residency (11,12). Such holistic ratings capture adaptability, judgment, and professional maturity—essential markers of readiness for independent practice. With adequate examiner training and calibration, global judgments can enhance authenticity without compromising fairness.

Fairness and Defensibility in Residency OSCEs

For residency assessments, authenticity must remain balanced with fairness and defensibility. Strategies include standardized prompts, structured rubrics that combine checklists with global ratings, examiner training, and digital scoring platforms that ensure transparency and auditability. These safeguards protect the defensibility of entrustment decisions while allowing authentic assessment of resident performance.

Conclusion

Authenticity in residency-level OSCEs is achieved by making exams reflect clinical reality while safeguarding fairness and defensibility. Embedding OSCEs within programmatic assessment, aligning them with Entrustable Professional Activities, and incorporating examiner global judgments creates a balanced and meaningful assessment system. Such an approach supports defensible decisions about residents’ readiness for independent practice and ensures that OSCEs contribute effectively to postgraduate medical education.

References

1.     1. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. BMJ. 1975;1(5955):447–451.

2.     2. Govaerts MJ, van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47(12):1164–1174.

3.     3. Van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205–214.

4.     4. Hodges B. OSCE! Variations on a theme by Harden. Med Educ. 2003;37(12):1134–1140.

5.     5. Hauer KE, Ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ. 2021;26(1):1–17.

6.     6. Van der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309–317.

7.     7. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176–1177.

8.     8. Mubuuke AG, et al. Mapping OSCE tasks to Entrustable Professional Activities: Enhancing the authenticity of clinical skills assessment. BMC Med Educ. 2022;22:508.

9.     9. Ten Cate O, Peters H. Medical education and training in the era of entrustable professional activities: moving from time-based to competence-based. Acad Med. 2023;98(5):e1–e6.

10.  10. Shallwani S, Gupta P, Goyal R, et al. From OSCE to EPA: Bridging assessment frameworks in postgraduate medical education. Front Med Educ. 2023;2:118.

11.  11. Regehr G, MacRae H, Reznick R, Szalay D. Comparing the psychometric properties of checklists and global rating scales for OSCEs. Acad Med. 1998;73(9):993–997.

12.  12. Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive to level of training. Med Educ. 2003;37(11):1012–1016.