Correct translations can save lives

“We have not yet thought enough about how we can help users,” said Carpuat in her lecture. But who are these users and in what situations do they resort to machine translations? Various studies conducted in recent years provide information on this: migrant workers use Machine Translation when they are looking for new jobs, want to contact potential employers or need information on health care. When young people want to translate something, they primarily use Generative AI. As a rule, users are looking for initial information, not the perfect translation.

Dangerous translation errors

Nevertheless, Carpuat uses a few examples from around the world to show that a Machine Translation error in just one word can have dramatic consequences. For example, a fatal argument broke out between a Chinese man living in Korea and his local friend because the Chinese man unintentionally referred to the Korean man’s wife as a nightclub employee due to a translation error. An Arab Israeli was arrested and interrogated because the automatic translation of the Arabic “good morning”, which he added to his photo on a social network, incorrectly translated the Hebrew “attack them” when translated into Hebrew using Machine Translation. But serious misunderstandings can also occur in a medical context, for example if a doctor recommends to her patient in English that she hold her medicine, but the patient, through Machine Translation, understands this as meaning that she should keep takingit.

How can doctors be supported in deciding whether a Machine Translation is good enough to communicate with patients who do not speak their language? Carpuat and her research group had English-language medical instructions and their automatic translations into Chinese first evaluated by a quality assessment model. Then the Chinese texts were automatically retranslated. The most important findings: the quality assessment was a great help for the doctors in deciding whether they could trust the automatic translation. The back-translation did not fulfill this purpose, but was better at identifying critical errors.

To make communication mediated by Natural Language Processing tools reliable and trustworthy, Carpuat therefore proposed a triple approach: explainable quality assessment, Machine Translation as a mediator between native and non-native speakers, and better public education on Machine Translation and AI in general.

The role of cultural context

Accurate translations can be vital, but they may not be enough. This is where explicitations come into play. Carpuat cited the example of Charlie Hebdo: most French people are familiar with the name of the magazine. However, when translating a French-language text into other languages, the explicitation that this is a French satirical magazine must be added.

In another project, Carpuat is investigating whether explicitations can be automatically generated and how this creation can be evaluated. To do this, she had professional translators evaluate and, if necessary, correct explicitations from the English-language Wikipedia with French, Polish and Spanish source texts. The same translators then assessed whether the respective explicitation was necessary from their point of view, whether it was appropriate and well integrated into the text.

“To support people in communicating across languages with Natural Language Processing, we need much more than just Machine Translation,” Carpuat summarized. We need tools that, for example, help them to rely on the results, to adapt the results to the context of use, and to use text and speech in many languages. And last but not least: “We need a broader range of tests to evaluate the usefulness in practical applications as well as in studies with humans.”