Evaluating a large language model in translating patient instructions to Spanish using a standardized framework

Evaluating a large language model in translating patient instructions to Spanish using a standardized framework
JAMA Pediatrics; by Mondira Ray, Daniel J. Kats, Joss Moorkens, Dinesh Rai, Nate Shaar, Diane Quinones, Alejandro Vermeulen, Camila M. Mateo, Ryan C. L. Brewster, Alisa Khan, Benjamin Rader, John S. Brownstein, Jonathan D. Hron; 7/25
Patients and caregivers who use languages other than English in the US encounter barriers to accessing language-concordant written instructions after clinical visits. Large language models (LLMs), such as OpenAI’s GPT-4o ... can translate Spanish translations of real-world personalized written patient instructions that are comparable in quality to those generated by professional human translators. Independent professional medical translators preferred the GPT-4o–generated translations over the human translations, and error analysis revealed a higher rate of mistranslation errors in the human translations. These findings demonstrate GPT-4o’s ability to produce quality translations in Spanish ...