This pilot randomized controlled trial included 64 adults scheduled for cataract surgery at a university hospital in Brussels, Belgium. Patients were randomized to receive standard information brochures plus access to a hospital-specific chatbot (Mina) (n=33) or brochures alone (n=31). The primary outcomes were knowledge gain, change in preinformation to postinformation anxiety, and satisfaction.
No significant differences were detected between groups for knowledge gain, anxiety change, or satisfaction (all P > .05). However, knowledge increased significantly in both groups after receiving information (P < .001). Among the chatbot group, 17 of 33 (52%) did not engage with the chatbot. Participants who engaged tended to be younger (mean age 64.1 ± 10.9 years) than those who did not (74.1 ± 10.5 years). Of the questions submitted, 63% matched validated answers. Chatbot usability was rated high (System Usability Scale score 83.1 ± 12.1).
Safety and tolerability were not reported. Key limitations include chatbot engagement barriers, particularly among older adults, and the use of validated-only content. The study was a pilot trial, so results should be interpreted cautiously.
In practice, these findings indicate the need for a hybrid approach that balances safety and flexibility in digital patient education. The chatbot did not improve outcomes over brochures alone, and engagement was low, especially in older patients.
View Original Abstract ↓
PURPOSE: To evaluate the effectiveness and usability of a safety-first, clinician-validated conversational artificial intelligence (AI) chatbot for cataract surgery education compared with standard brochures.
SETTING: University Hospital (UZ Brussel), Brussels, Belgium.
DESIGN: Prospective, single-center, randomized controlled trial.
METHODS: Adults scheduled for cataract operation were randomized to receive either standard information brochures alone (control group) or brochures plus access to a hospital-specific chatbot ("Mina"). Primary outcomes were knowledge gain, change in preinformation to postinformation anxiety, and satisfaction. Those outcomes were measured with questionnaires. Secondary outcomes included chatbot usability (measured with the System Usability Scale [SUS]) and engagement with the chatbot.
RESULTS: 64 patients were randomized (chatbot group 33, control group 31). Postoperative questionnaires were completed by 35 patients (14/33 chatbot, 21/31 control). No significant differences were detected in knowledge gain, anxiety change, or satisfaction ( P > .05). Knowledge increased in both groups after receiving information ( P < .001). In the chatbot group, 17 of 33 (52%) did not engage with the chatbot. Participants engaging with the chatbot tended to be younger (mean age: 64.1 ± 10.9 years) than those who did not (mean age: 74.1 ± 10.5 years). Among users, 63% of submitted questions matched validated answers. The SUS mean score indicated high usability (83.1 ± 12.1).
CONCLUSIONS: A custom-built chatbot with only clinician-validated responses showed high usability but did not improve knowledge, reduce anxiety, or increase satisfaction compared with brochures. Chatbot engagement barriers, particularly among older adults, and limits of validated-only content indicate the need for a hybrid approach of those models, to balance safety and flexibility in digital patient education.