Forget the hype – machines still no match for humans when translating literary texts
- DCU study also shows translators prefer to 'start from scratch' than tidy up machine-translated text
Despite the hype surrounding Artificial Intelligence and machine translation, technology still struggles to match human expertise in accurately translating literary texts, according to a new study carried out by researchers at Dublin City University (DCU).
Dr Joss Moorkens, Assistant Professor at DCU’s School of Applied Language and Intercultural Studies (SALIS), who carried out the study, said: "Machine Translation is the use of computers to estimate how to best translate words and sentences from one language to another based on how they've been translated in the past. It's quick and effective, particularly on short, clear sentences requiring a literal, word-for-word translation. However, the limitations of machine translation can be exposed when processing longer, creative sentences typically found in a literary novel, for example.”
Explaining the rationale for the study, he added: “Recent years have seen the emergence of Neural Machine Translation (NMT), a system that usually produces more fluent translations, fewer errors and less literal translations. Little if any research has been published to date on translators' attitudes to using this new state-of-the-art software when post-editing or 'tidying up', particularly on challenging, literary text. Our study focuses on the translators’ perceptions of such a task, whether they found the machine translation suggestions useful, and whether they would consider post-editing literary texts in the future, and contrasts these task perceptions with our technical and temporal measurements of their activity."
To effectively test the latest translation software – and how translators use it for post-editing – a challenging, literary novel* was selected as the centrepiece of the study. Translators with experience ranging from between 1-4 years up to 30 years were chosen to translate (from English-to-Catalan) parts of the book’s Chapter 1 using three methods – translation from scratch, and post-editing of two forms of machine translation (neural and statistical).
Participating translators provided feedback before and after the translation via questionnaires and interviews. Asked to rank the translation methods according to the translation speed, all participants responded that the latest [Neural] machine translation software enabled them to do their job properly, but without exception, all translators still stated a preference for translation from scratch.
The study on professional translators' perceptions of machine translation has revealed that despite the technological advances enabling them to do their job faster, translators feel that 'post-editing' machine-translated text constrains their work, limits their creativity and conditions them to produce a literal translation.
In conclusion, Dr Moorkens said: "With a lot of hype surrounding technological advances in machine translation, it is a timely exercise to analyse how well state-of-the-art systems can deal with the most challenging and creative of translation tasks, along with human translators' perceptions of machine translation and their experiences of using machines for post-editing. Comparing human translation to a computer algorithm is problematic, of course, as machine translations systems merely attempt to replicate patterns found in their training data. While the latest systems can sometimes translate ambiguous words and phrases correctly where previous systems would have failed, participants in the study identified broader knowledge required to create an equivalent reading experience. So while the recent hype on new machine translation systems has been unnerving for some translators - with one participating translator remarking that it was 'scary and frightening to see that the machines are getting better' - it is safe and reasonable to conclude that machine translation systems are 'still far' from being a threat."
The study, 'Translators' perceptions of literary post-editing using statistical and neural machine translation', is to be published in the November edition of ‘Translation Spaces’ (https://benjamins.com/
KEY FINDINGS
- Participants stated that latest machine translation systems would be useful if they need to produce a translation “in a hurry”, but that they would not post-edit by choice as they found the process difficult and found the translations to be bland.
- All participating translators in this study responded that the latest machine translation software enabled them to do their job properly, yet, without exception, all translators still stated a preference for translation from scratch.
- The findings also showed that for this study, the use of latest machine translation technology represents an improvement in terms of productivity and number of edits required.
- Experienced translators who become familiar with latest machine translation technology are more likely to identify its limitations.
- Translators with less experience find machine translation suggestions useful.
RESEARCH TEAM
- Dr Joss Moorkens, Assistant Prof. of Translation Studies at the School of Applied Language and Intercultural Studies in DCU and researcher at the ADAPT Centre and Centre for Translations and Textual Studies;
- Dr Antonio Toral, Assistant Prof. at Groningen University;
- Dr Sheila Castilho, Post-doctoral Researcher at the ADAPT Centre in DCU;
- Prof Andy Way, Prof. of Computing and Deputy Director of the ADAPT Centre in DCU.
* Warbreaker, a fantasy novel written by US author, Brandon Sanderson.