Neural Machine Translation Post-Editing: Productivity, Edits and Knowledge

Project background

Productivity is a top priority for the language services industry, which demands ever faster turnaround times for translation and localisation projects. However, the concept of productivity in this sector has not been explicitly defined and there is a lack of clarity as to how productivity should be measured. Previous research has largely overlooked this issue, probably due to a lack of collaboration between industrial partners and academic researchers in the field of translation studies.

This project aims to fill this gap in knowledge and gain a better understanding of post-editing productivity by analysing the projects of a language service provider (LSP), in which adaptive neural machine translation and full human post-editing  – i.e. professional human linguists correct the outputs from state-of-the-art machine translation, to achieve translations of the highest possible quality – are used to translate advertising and marketing materials. 

Research team

• Lead Researcher: Ms Silvia Terribile, Translation and Intercultural Studies, The University of Manchester. 
• Academic Supervisor: Prof Maeve Olohan, Centre for Translation and Intercultural Studies, School of Arts, Languages and Cultures, The University of Manchester. 
• Industry Supervisor: Mr Matt Train, Head of Strategy & Operations at TranslateMedia.

Research approach

This research will employ a mixed-methods approach, combining findings from quantitative and qualitative analyses of productivity and post-editing datasets, to strengthen their reliability and validity. The qualitative analysis will also allow for an in-depth examination of trends identified at the quantitative analysis stage. A statistical analysis of productivity reports provided by TranslateMedia will be followed by a comprehensive categorisation of the types of edits that human linguists carry out during post-editing, based on a detailed qualitative analysis of English-to-Italian post-editing jobs. The final analytical stage will result in a classification of source features that are problematic for machine translation, based on analysis of English source texts. 

Expected impact

As machine translation assumes an increasingly prominent role in professional translation practice, the need for humanities-based research into the nature and consequences of human-machine interaction in the workplace, through activities such as post-editing, becomes ever more pressing. This project is firmly centred on these issues and thus helps to extend the research expertise of the Centre for Translation and Intercultural Studies in directions that are highly relevant. It also provides a valuable blueprint for future collaborative research between industry partners and humanities scholars in an area where such collaboration has hitherto been relatively rare.

The results of this research will inform recommendations on effective post-editing for TranslateMedia, including: 

• providing research-informed training on effective post-editing to their linguists;
• enabling managers to evaluate post-editing productivity in a more comprehensive manner and, as a result, to manage time and resources more effectively;
• gaining a better understanding of the ways in which a wide range of factors can affect post-editing productivity, which will enable them to make more informed decisions.

“By working on this project, I am developing a wide range of skills, such as deepening my knowledge of the topic, gaining great insights from my industrial partner, presenting my work clearly and effectively, learning new research methods – such as statistical analysis and text annotation and tools such as IBM SPSS Statistics and @nnotate. This project allows me to bridge the gap between academia and industry, and I truly feel that I am getting the most out of both worlds. Carrying out research on real-life projects increases the validity and reliability of my findings, and it is preparing me for multiple career paths both in academia and in industry”. Silvia Terribile / PhD Researcher