• Citer

NMT/LLM PE and Video game Localisation: Can Visual Novels Benefit from Machine Translation?

par Copet, Simon ;De Faria Pires, Loïc
Référence Espaces Linguistiques, 10, page (76-100)
Publication Publié, 2025-12-01

Article révisé par les pairs

• Accès en ligne
• Détails
• Contenu
• Statistiques

Résumé :

This exploratory paper presents one of the first comparative studies of PE (post‑editing) and HT (human translation) quality in the field of video game localisation. We used an excerpt from the KillerTrait free visual novel, selected for its particular features in terms of gender, puns and oral language. This excerpt was submitted to DeepL and ChatGPT engines, and these versions were respectively post‑edited from English into French by nine and ten last-year translation students in the framework of their EN > FR post‑editing class at the University of Mons. Nine other students from the group translated the text from scratch. In these productions, we analysed fidelity and fluency. We compared the results from human quality evaluation to the HTER and BLEU automatic metrics. Human evaluation shows that the texts post‑edited using ChatGPT display better PE quality scores than those post‑edited using DeepL in terms of fidelity. Human translation was evaluated differently by both evaluators, thus leading to an impossibility to identify any trend. In terms of fluency, no clear trend can be found. Automatic metrics show that 1) no clear trend emerges in terms of raw MT quality, 2) PE effort is slightly lower (and quality supposedly higher) in the case of the raw ChatGPT output, and 3) quality does not seem to vary when comparing the PE texts produced using each MT engine, but HT displays a higher internal variability, which could indicate that HT provides for less homogeneous productions than PE.