Márta Lesznyák: A longitudinal investigation of translation trainees’ translation and post-editing competence development I
›
4 Summary and conclusions
›
4.1 A summary of the main findings

4.1 A summary of the main findings

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In this chapter, the design and the most important findings and conclusions of the longitudinal investigation reported above will be summarised, followed by some implications for translator training. Finally, the limitations of the research will be discussed, and suggestions for further research will be offered.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The longitudinal study reported here forms part of a larger research project investigating the role the elements of translation competence, as defined by the PACTE group, play in human translation (HT) and in the post-editing of machine translation (PE). The aims of the longitudinal study included describing students’ translation and post-editing competence at the beginning and at the end of their studies and identifying and analysing changes between the points in time. Furthermore, an important objective of the study was to compare post-editors and translators on performance and on several measures related to performance (detailed in the next paragraph). 31 students entered the longitudinal investigation. Nevertheless, there were instances of data loss and participant dropout, which have led to varying amounts of information available for analysis across the different research questions. In the final analysis of development, target texts from 23 students (10 HT and 13 PE) were included.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

At the beginning of their studies, students were assigned to two groups: post-editors and translators. Translators had to translate a 127-word-long text from English into Hungarian at the beginning and at the end of their studies. Post-editors worked with the same source text, but they were required to post-edit the initial draft of the translation generated by DeepL. Time on task was measured for both groups. Translations and post-edited target texts were evaluated by two raters using an MQM-based error classification system that comprised the categories of accuracy, fluency (linguistic conventions), style and terminology. Additionally, data were collected from students on the following factors: English language competence (reading and use of English), thematic knowledge (linguistics: bilingualism and pragmatics), beliefs about translation, and task perception (perceived task difficulty, terminological problems, post-editors’ specific experiences)

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In the next paragraphs, the most important findings of the study will be summarised.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In the first data collection wave, no significant differences in the time spent on task were revealed between students engaged in human translation (HT) and those performing post-editing (PE). Despite this, students in the PE condition demonstrated better overall performance, which was reflected in lower total error numbers and in lower accuracy error numbers. Accuracy errors were the most prevalent error type in both groups, suggesting that the most challenging aspect of translation and post-editing for novice students is exact information transfer.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Regarding the subjective experience of the task, translators and post-editors found it equally challenging. The most difficult aspect for both groups was syntactic complexity, indicating that structural issues posed greater difficulty than other linguistic factors. Interestingly, when students identified problematic terms, most of these were related to pragmatics rather than bilingualism.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In addition, PE students’ approach to terminology showed a marked contrast to that of HT students. Post-editors struggled with identifying terminological problems and seemed to have difficulties with distinguishing between terms and ordinary lexical items in the source text.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

As for correlations, in the PE condition, source language reading skills emerged as a key factor, showing significant correlations with a number of performance measures. For students working in the HT condition, heightened awareness of terminological and syntactic difficulties was linked to a lower number of accuracy errors. Conversely, in the PE group, the (false) belief that post-editing helps with terminology appeared to contribute to a higher number of accuracy errors, suggesting that overreliance on the MT output may lead to reduced awareness concerning terminological precision.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In the second data collection wave, no significant differences were found between HT and PE students concerning the time they spent on producing the TT. However, PE students showed better overall performance as they made fewer errors in total and scored higher in both accuracy and fluency. Accuracy errors remained the most common error type in both groups, though their dominance was slightly less pronounced than at the beginning of the training. In the PE group, fluency errors occurred significantly less frequently than other error types, suggesting that fluency is the easiest aspect of post-editing.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Additionally, PE students rated the task as generally easier compared to HT students, although there were no significant differences between the two groups regarding specific difficulty aspects such as terminology, thematic knowledge, and syntactic complexity. Neither group showed a dominant difficulty type, as all three dimensions were perceived as equally challenging.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Similar to the first year, students most often identified terms related to pragmatics as problematic. Second-year students in the PE condition still struggled with identifying terminological challenges, as they often seemed unable to distinguish between terms and general vocabulary in the source text. Despite this, post-editing students reported that they found their work mode faster and easier than translating from scratch. According to student responses, the main advantage of post-editing is that it reduces typing effort, though it does not offer any help with terminology. Interestingly, the loss of creativity was not considered a disadvantage of post-editing.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Turning to correlations, it was revealed that by the end of the second year, beliefs about translation emerged as a key subcompetence in both the HT and PE groups, showing significant correlations with performance. In the PE group, thematic knowledge also played an important role in certain aspects of performance, while grammar competence was mainly linked to overall achievement, as measured by the total number of errors. Time investment proved beneficial for both HT and PE students, though in different ways: for HT students, spending more time led to better accuracy, whereas for PE students, it contributed to improved terminological correctness.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Correlations in the HT group suggest that recognising the importance of thematic knowledge is more critical than scoring high on thematic knowledge tests. In the PE group, however, an inability to judge whether a solution offered by MT is acceptable was associated with lower accuracy scores, indicating competence gaps or deficiencies. A correlation pattern also revealed an unsuccessful approach among certain students: those who subscribed to the belief that post-editing was faster tended to work at a higher speed and made more terminology errors.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The comparison of the data in the first and the second data collection wave revealed growth in several areas and sub-competencies. One of the most remarkable developments was observed in “Beliefs about translation”. Among the examined sub-competencies, beliefs about translation showed significant improvement, which can be attributed to both the training program and the practical experience gained throughout the training. This suggests that exposure to translation tasks and systematic instruction contribute to a more refined understanding of what translation is and how it should be performed.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

An unexpected but notable improvement was found in reading competence. Since reading was not explicitly trained as part of the program, its development can be interpreted as a positive backwash effect of extensive translation practice. Engaging with various source texts and critically assessing them as part of translation tasks may have contributed to improved reading skills. This finding may have relevance for advanced language teaching methodology, too.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

By the end of the training, both human translators and post-editors worked faster compared to the first year, but the increase in speed did not reach the expected level of statistical significance in either group. Interestingly, human translators gained more time than post-editors, which may be explained by the nature of post-editing itself. The time gain related to post-editing comes mainly from preparing the first draft with an MT, which preceded the post-editing task assessed in this study.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Another key finding relates to translation and post-editing performance, more specifically, to accuracy. Human translators showed significant improvement in accuracy between the two data collection waves. However, even by the end of the second year, their average error numbers were higher than those of PE students at the beginning of the training. This suggests that post-editing inherently offers an advantage in terms of accuracy, possibly due to the presence of an initial MT output that prevents students from committing certain accuracy-related mistakes.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In general, PE students outperformed HT students and demonstrated greater improvement, particularly in fluency errors, but in overall error numbers, too. This suggests that traditional translator training also contributes to PE competence development, even if PE-specific training is not explicitly incorporated into the curriculum.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Interpreting these findings in light of the PACTE (2020) translation competence model, we may conclude that both the HT and PE groups demonstrated development in bilingual subcompetence and knowledge of translation subcompetence. Relevant elements of extra-linguistic subcompetence did not change. The persistence of terminology errors suggests that instrumental subcompetence still requires further development.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

When analysing the same findings relying on the framework of Nitzke et al.’s post-editing competence model (2019), we could observe progress in bilingual competence and translation competence and possibly revision and post-editing competencies. As the present study was not designed on the basis of the Nitzke model, the data do not allow us to determine whether the improvements in PE performance are the outcome of the development of revision competence, translation competence, post-editing competence or a combination of them. Further research would be needed to clarify this point. Students’ problems with handling terminology, as evidenced by high error numbers, suggest that both instrumental and research competencies need to be further developed.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Students’ perceived difficulty of the tasks decreased in both groups, but the nature of this reduction differed. In the PE group, the decrease in perceived difficulty was more pronounced. One area where improvement was less apparent was terminology identification. The number of ordinary lexical items falsely identified as terms did not significantly decrease between the two data collection waves. This shortcoming was particularly noticeable in the PE group, suggesting that students struggled to apply the concepts and techniques taught in terminology training when working on real translation or post-editing tasks.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The study also revealed important changes in correlations between time on task and performance measures. By the end of the second year, significant correlations emerged, suggesting that some students had developed the ability to utilise their time more effectively.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Changes in the number of correlations between sub-competencies and performance indicators were also observed between the two data collection waves. In more exact terms, the number of significant correlations increased. These findings suggest a strengthening of relationships between different sub-competencies, as well as a shift from the dominance of linguistic competencies towards the dominance of beliefs about translation. This may indicate that with growing competence, students’ conceptual understanding of translation becomes a stronger predictor of performance than just their linguistic skills.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The study also highlighted changes in correlations between post-editing specific experiences and performance measures. Over time, students’ opinions and perceptions about post-editing became more stable and pronounced, leading to a larger number of significant correlations.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

However, misconceptions and misrepresentations of post-editing processes were also observed. One correlation pattern indicated that a superficial approach to post-editing was linked to a higher number of terminology errors. This suggests that students who underestimated the complexity of post-editing may have paid less attention to certain aspects of the task, resulting in more terminology mistakes. However, it is possible that this is only a temporary stage in competence development, and with further training, these misconceptions could be corrected.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Some of the misconceptions observed in students' approach to post-editing may be due to the lack of PE instruction. While traditional translator training appears to support the development of PE competence, explicit PE training, which the students in our study did not receive, could help students develop a more accurate understanding of what post-editing is and avoid common pitfalls.
chevron_left
chevron_right
Tartalomjegyzék navigate_next
Keresés a kiadványban navigate_next
A kereséshez, kérjük, lépj be!
Könyvjelzőim navigate_next
A könyvjelzők használatához
be kell jelentkezned.
Jegyzeteim navigate_next
Jegyzetek létrehozásához
be kell jelentkezned.
    Kiemeléseim navigate_next
    Mutasd a szövegben:
    Szűrés:
    Kiemelések létrehozásához
    MeRSZ+ előfizetés szükséges.
      Útmutató elindítása
      delete
      Kivonat
      fullscreenclose
      printsave