DCU researchers call for safer, more trustworthy AI language tools in high-stakes settings

AI language tools — like Google Translate or ChatGPT — are now being used in high-stakes situations like hospitals and disaster zones. But they can produce errors, inconsistent results, or outright fabrications, often while sounding perfectly fluent. This is dangerous when someone's health or safety depends on an accurate translation.

The paper introduces a framework called HCAILT (Human-Centered AI Language Technology), built on the idea that AI language tools should be designed with people — not just algorithms — at the centre. The framework borrows from an existing approach by computer scientist Ben Shneiderman and applies it specifically to language technology. It has three pillars: reliability, safety, and trustworthiness.

The authors illustrate the framework with two scenarios: a multilingual hospital setting (where a patient can't communicate with their doctor) and crisis communication (like disaster relief or a pandemic). In both cases, AI language tools could genuinely help — but only if properly designed with these safeguards.

They have also created a working prototype for the healthcare scenario: an Irish tourist in Spain with chest pain receives medical documents in Spanish. Three AI agents handle the task in sequence — one translates, one scores the quality, and one rewrites the result in plain language for a non-specialist reader.