Two essays concerning complexity, language and machine learning

In this first essay, we present our research project concerning the extraction of information from the Translation Quality Assessment (TQA) process, in which the quality of a translation conducted by a human translator from one language to another, is evaluated by another human translator. We take advantage of the dataset provided by the professional translation service provider Translated SRL, consisting of thousand of translations, produced by human translators and edited (with error annotations) by human reviewers. We deal with subjectivity that raises from the linguists involved in the process and we aim to understand which are the features able to catch translators’ behaviour. We applied Bayesian Networks methods to build a probabilistic framework that helps us to understand the patterns of the translation process, assessing the difficulty of the source texts, the skill of the translators, and the strictness of the reviewers, together with the consistency of both the last two. We run three validation methods in which we test the two Bayesian models created, comparing them, and showing that they can reasonably fit the data and retrieve significant patterns in the behaviour of the linguists involved. We designed an experiment to add new data to our dataset to check the predictability of the quality of the individual translated texts, and despite the single estimate of the quality of the specific translation has been shown to be poorly predictable, the best of our models has proven to be able to predict the mutual relations between reviewers, showing the possibility to represent a useful tool in assessing linguists' behaviour and therefore in establishing their reliability.