Mode
Text Size
Log in / Sign up

Scoping review of AI and algorithmic literacy measurement in health professions educationHealth workers need better ways to measure their AI skills today

AI-generated summary of the cited source, checked by automated accuracy review. How we work

Key Takeaway
Note that AI literacy measurement in health education lacks standardized, health-specific tools and explicit theoretical grounding.

This scoping review examines the conceptualization and measurement of AI and algorithmic literacy among health workers, specifically within medical students, nursing students, and health professions education contexts. The review synthesizes data from 12 studies, noting that evidence was concentrated in health professions education, accounting for 10 of the 12 studies included. The scope is limited to education settings, with no studies exploring public health practice, representing 0 of the 12 studies in that domain.

The authors report that explicit, theory-grounded definitions of AI literacy were uncommon across the included literature. Measurement tools frequently relied on self-reported instruments, with the Artificial Intelligence Literacy Scale (AILS) used in 3 studies, the Meta Artificial Intelligence Literacy Scale (MAILS) in 2 studies, and the Scale for the Assessment of Non-Experts’ AI Literacy (SNAIL) alongside self-developed tools. Only one of the 12 studies explicitly defined and measured algorithmic literacy as a distinct construct.

Regarding digital health literacy (DHL), links were only implied rather than explicitly defined. Competencies assessed aligned mainly with functional and critical dimensions, particularly awareness, use, evaluation, and ethics, while communicative literacies were infrequently assessed. The review highlights that AI and algorithmic literacy among health workers remains underdeveloped, weakly integrated with digital health literacy, and inconsistently measured using non-health-specific self-report tools.

These findings point to the need for clearer conceptual alignment, health-specific measurement, and systems-based approaches to workforce readiness as AI-enabled tools expand across healthcare and public health. The authors caution that current research largely overlooks communicative competencies essential to clinical and public health practice, indicating a gap in the current evidence base.

Health workers are stepping into a world powered by artificial intelligence, yet the tools to measure their understanding of this technology are missing. A recent look at current training shows that most studies focus on medical and nursing students, leaving public health practice largely ignored. The research found that clear, theory-based definitions for AI literacy are rare, and connections to digital health literacy are only hinted at rather than proven.

To check how well people understand AI, researchers mostly used self-reported surveys. Common tools like the Artificial Intelligence Literacy Scale were used in a few studies, but many relied on custom-made questionnaires. Only one study out of twelve tried to measure algorithmic literacy separately. The skills being tested mostly covered basic awareness and ethics, while important communication skills were rarely checked.

This gap means we do not have a solid way to know if health workers are truly ready for AI-enabled tools. The findings suggest we need clearer definitions, better measurement methods, and a focus on the communication skills essential for patient care. As AI grows in healthcare, we must build systems that properly prepare the workforce without relying on shaky or incomplete data.

What this means for you:
Current AI literacy measures for health workers are weak and need better definitions and tools.

Study Details

Study typeSystematic review
EvidenceLevel 1
PublishedApr 2026
View Original Abstract ↓
IntroductionArtificial intelligence (AI) and algorithmic systems influence health workers’ access, interpretation, and action on clinical and public health information, positioning them as intermediaries between algorithmically mediated outputs and patients, communities, and decision makers. This study examines how AI and algorithmic literacy are conceptualized and measured among health workers through a digital health literacy (DHL) lens.MethodsUsing Arksey and O’Malley’s scoping review framework, we searched Ovid MEDLINE, Ovid Embase, Scopus, IEEE Xplore, ACM Digital Library, Europe PMC, and arXiv for English language sources published between January 2020 and May 2025. Two reviewers screened records and extracted data using a theory informed charting framework grounded in Nutbeam’s model (functional: basic understanding and use; critical: evaluation and ethics; communicative: interacting with AI systems and explaining AI-mediated information). We synthesized findings using descriptive statistics and a narrative synthesis.ResultsTwelve studies published between 2021 and 2025 met inclusion criteria. Evidence was concentrated in health professions education (10/12), primarily among medical (6/12) and nursing students (2/12), with no studies exploring public health practice. Explicit, theory-grounded definitions of AI literacy were uncommon, and links to DHL were only implied. AI literacy was frequently operationalized through self-reported instruments, commonly the Artificial Intelligence Literacy Scale (AILS; 3 studies), Meta Artificial Intelligence Literacy Scale (MAILS; 2 studies) and the Scale for the Assessment of Non-Experts’ AI Literacy (SNAIL), alongside self-developed tools. Only one study explicitly defined and measured algorithmic literacy as a distinct construct; in other studies, algorithmic considerations appeared indirectly through recognizing AI presence in systems or evaluating AI generated content. Across studies, competencies aligned mainly with functional and critical dimensions of DHL, particularly awareness, use, evaluation, and ethics, while communicative literacies were infrequently assessed.DiscussionAI and algorithmic literacy among health workers is underdeveloped, weakly integrated with digital health literacy, and inconsistently measured. Research prioritizes AI literacy using non–health-specific self-report tools and largely overlooks communicative competencies essential to clinical and public health practice. These findings point to the need for clearer conceptual alignment, health-specific measurement, and systems-based approaches to workforce readiness as AI-enabled tools expand across healthcare and public health.
Free Newsletter

Clinical research that matters. Delivered to your inbox.

Join thousands of clinicians and researchers. No spam, unsubscribe anytime.