Artificial Intelligence Tools as Quality Assessment Copilots

We are seeing an explosion of interest in Artificial Intelligence (AI) driven tools that are paired as “copilots” with human workers. This paper describes such a tool, named AQuA (for Augmented Quality Assurance), which helps stakeholders in the translation process to efficiently and thoroughly evaluate certain qualities of a draft translation, even when the process that has been used to produce the translation is not well understood. After describing the aspects of translation quality that this AI copilot seeks to measure, we turn to the problem of how to anchor the abstract metrics generated by the AI in the concrete reality of the stakeholders. We propose two strategies: grounding the evaluation of metrics for a new translation in the aspirational qualities expressed in its translation brief and using the metrics for each new translation to place it within the landscape of known translations. The paper concludes by presenting some findings from case studies of applying AQuA in two field projects.