Competency-based medical education assessment tools face challenges in validity, equity, and faculty readiness in anesthesiologyNew Assessment Tools Aim to Improve Anesthesiology Training Quality

Frontiers in Medicine Published September 28, 2026 DOI ↗ Editorial oversight: Dr. Amelia Tan, PhD · Internal Medicine & Chronic Disease

AI-generated summary of the cited source, checked by automated accuracy review. How we work

Key Takeaway

Note that rater variation and limited faculty training remain primary barriers to effective CBME assessment in anesthesiology.

This narrative review explores the current landscape of competency-based medical education (CBME) assessment tools, such as milestones and entrustable professional activities, specifically within the field of anesthesiology. The authors synthesize evidence regarding the validity of workplace-based assessments, the reliability of these tools, and the systemic challenges regarding equity and faculty readiness.

Key findings indicate that workplace-based assessment tools are primarily validated through single-center, small-sample designs with limited longitudinal outcome data. While acceptable reliability can be achieved through rational design, rater variation remains the primary source of measurement error. Regarding equity, national studies identified different rating growth trajectories for female and underrepresented-in-medicine residents, though structured tools may help moderate bias. A significant implementation bottleneck is identified in faculty readiness, as most clinical teachers lack formal CBME training and anesthesiology-specific evidence is limited, often relying on cross-specialty surveys.

Limitations noted include the scarcity of anesthesiology-specific evidence and the reliance on cross-specialty data for assessing faculty readiness. The review suggests that the successful integration of CBME in anesthesiology requires addressing these gaps in training and longitudinal data. These findings are relevant for program directors and educators seeking to standardize and improve the reliability of residency assessments.

This review looked at how medical students and residents are evaluated in the field of anesthesiology. Specifically, it examined competency-based medical education tools, which are designed to ensure that new doctors are ready to provide safe care. These tools include milestones and workplace-based assessments to track a student's progress.

While these tools can be reliable, the review found that the biggest source of error is often the variation between different people grading the students. Additionally, some national studies showed that female residents and those from underrepresented backgrounds sometimes received different rating trends. Using well-structured tools may help reduce these types of biases.

A major challenge identified is that many clinical teachers do not have formal training on how to use these new assessment methods. Because specific evidence for anesthesiology is currently limited, the findings are not yet enough to change daily practice. These results suggest that while the tools are promising, more training for teachers and more data are needed.

What this means for you:

New assessment tools show promise for training, but teacher training and consistent grading remain key hurdles.

Common questions

How reliable are the current tools for training anesthesiology residents?

The review found that acceptable reliability can be achieved with a rational design. However, the most common source of measurement error is still rater variation. This means that different instructors might grade the same student differently, which is a challenge for consistent training.

Are these assessment tools fair for all residents?

National studies found different rating growth paths for female residents and those underrepresented in medicine. The review suggests that using well-structured assessment tools may help to moderate these bias effects and create a more equitable evaluation process for all students.

Are the teachers ready to use these new assessment methods?

The review identified a major bottleneck because most clinical teachers currently lack formal training in competency-based medical education. Because specific evidence for anesthesiology is limited, many findings currently rely on surveys from other medical specialties.

Study Details

Study typeSystematic review

EvidenceLevel 1

PublishedSep 2026

View Original Abstract ↓

Competency-based medical education (CBME) is driving a paradigm shift in anesthesiology graduate medical education, moving from fixed-duration training to assessment-centered models. This narrative review aims to synthesize the sources of validity evidence for existing assessment tools (milestones, entrustable professional activities, and workplace-based assessments), equity threats during evaluation and their intervention strategies, and the impact of faculty readiness on assessment quality. Informed by the Scale for the Assessment of Narrative Review Articles as a quality-appraisal instrument, we searched PubMed, Web of Science, CNKI, and Wanfang databases through June 2026. Two authors screened sources and extracted data through iterative team discussions, using a framework approach for thematic organization. Current evidence indicates that workplace-based assessment tools have been validated primarily through single-center, small-sample designs with limited longitudinal outcome data. Generalizability theory analyses demonstrate that acceptable reliability can be achieved with a rational design; however, rater variation remains the predominant source of measurement error. With respect to equity, national studies have identified differential rating growth trajectories among underrepresented-in-medicine residents and female residents, although tool structuration may moderate bias effects. Faculty readiness appears to be a major implementation bottleneck, as most clinical teachers lack formal CBME training. However, anesthesiology-specific evidence is limited and partly relies on cross-specialty surveys. The Chinese elite teaching hospital consortium has proposed a localized milestone evaluation framework, but it remains in early pilot stages. The success of CBME assessment in anesthesiology relies on dynamic balance among tool validity, assessment equity, and faculty readiness, with the interaction of these three domains producing emergent system-level behaviors. Future research should prioritize tracking longitudinal predictive evidence and conducting specialty-specific randomized trials of equity interventions.

Competency-based medical education assessment tools face challenges in validity, equity, and faculty readiness in anesthesiologyNew Assessment Tools Aim to Improve Anesthesiology Training Quality

Common questions

Study Details

Evaluating the Efficacy of Traditional Chinese Medicine Decoctions for Rheumatoid Arthritis Management

Traditional Chinese medicine decoctions may improve rheumatoid arthritis symptoms

Clinical research that matters. Delivered to your inbox.

Competency-based medical education assessment tools face challenges in validity, equity, and faculty readiness in anesthesiologyNew Assessment Tools Aim to Improve Anesthesiology Training Quality

Common questions

Study Details

Evaluating the Efficacy of Traditional Chinese Medicine Decoctions for Rheumatoid Arthritis Management

Traditional Chinese medicine decoctions may improve rheumatoid arthritis symptoms

Clinical research that matters. Delivered to your inbox.

Related in Rheumatology

From Other Specialties