Perhaps the most exciting part of this stage of our investigation of references in RAs is checking the reliability of the analysis. The reason why it is so interesting is that comparison of the results of reference in longer texts has been nearly impossible up to this point. As pointed out in Chapter 5 of this book, Halliday and Hasan’s (1976) analysis was not possible to work with when it came to longer texts, comparing several analyses was even more complicated. In the present form of the analytical tool for Referential Cohesion Analysis, chains of reference are represented by columns in an Excel table. This representation made the comparison of several analyses of the same text easily comparable. A fellow researcher from the PhD program, Ildikó Szendrői, was kind enough to devote a great amount of time to analyzing RA3 in my RA corpus, which made it possible to check inter-coder reliability. The text of RA3 is 153 sentences long; one of us identified 205, the other 236 cohesive ties are organized in reference chains, of which 26 were exactly the same in our analyses. In the remaining 13 chains there were a number of differences. The two analyses were copied into one table with identical or similar chains in parallel, thus the ratio of identical ties could be counted, the result of which is an 85.2% match. This result is much more convincing than results at Stage 1 with the abstracts, where only 54% of the ties were identified by both raters and only 18% of the ties were analyzed consistently in the same way.

 

Lexical relations were again problematic, especially when it came to part-whole relationships. In the list of examples below, the referring items in one analysis were considered cohesive, and in the other, they were not:

Besides the above differences, items with determiners were occasionally missing from one analysis: most enrolled students, each student. A new problem that arose here was the status of proper names with a definite article: one analysis identified a proper name with a definite article as a cohesive tie with its previous mention, the other did not. Another issue is whether items such as the present study refer to the manuscript in the same way as this paper or this study.

 

 

Occasional differences occurred with items like the computer which was ambiguous between exophoric and cohesive.