At the first stage of the research process, Halliday and Hasan’s (1976) original taxonomy was taken as a starting point. In their book, Cohesion Analysis they suggest “a method for the analysis of cohesion in text” in Chapter 8: The analysis of cohesion (ibid., 329–355). Based on my previous experience in analyzing text using this method and on the theoretical underpinnings detailed in the literature review, it was hypothesized that the taxonomy has its weaknesses regarding the applicability of the taxonomy for the analysis of long texts and the clarity of distinction between types of reference items. Therefore, at this phase of the study I concentrated on testing the validity and reliability of this taxonomy. For these purposes I and a fellow researcher (Krisztina Zsova) analyzed a mini corpus of 10 Research Article (RA) abstracts and checked inter- and intra-rater reliability. She was also familiar with the taxonomy, but she received a summary of the principles of the analysis to refer to (Appendix A). A think-aloud session was held to reveal problems that surfaced during the analysis. The validity of the construct of reference was rethought based on this data and theoretical research (Chapter 5 in this book).

The second stage of the research process was mainly concerned with designing an analytical tool for Referential Cohesion Analysis. The resulting tool was based partly on the data obtained during the analysis of abstracts in the previous validation process at Stage 1 and partly on earlier research (Jenei, 2008) using Halliday and Hasan’s (1976) instrument. The transformation of this framework into an analytical tool for Referential Cohesion Analysis was also grounded in theoretical research (e.g., Lautamatti, 1978; Hasan, 1985) in discourse analysis instruments and methods. The tool (see Table 8 in Chapter 3 in this book) was first piloted with the help of another researcher (Ildikó Szendrői). We analyzed a full research article independently with promising results and discussed problems related to the analytical procedures. This provided useful insights into necessary modifications in the instructions related to the application of the analytical tool. In this phase the empirical research reached the stage where the Referential Cohesion Analysis instrument could be trialed on 10 research articles (RAs 1–10).

Building on the experience gained during the Referential Cohesion Analysis of the 10 RAs in the second stage, the third stage was initially devoted to backing up the tool with its negative version by defining all those language items in a text that may look referential but are not (this will be clarified later, in Chapter 7). The decision to extend the analysis to this complementary aspect resulted from the observation that in order to establish the reference of an item clearly, we need to be able to explain whether its reference is endophoric, exophoric, or refers to an item in the same sentence. This reversed process which finally led to the taxonomy of non-cohesive items, builds on the categorization of empirically obtained data and on the relevant theoretical background for further explanation. As in the previous two stages, inter- and intra-rater reliability were checked. Again, Ildikó Szendrői was my brave companion and spent a great deal of time with the meticulous analysis that this pilot study entailed. This third stage of the research also ventured to expand the corpus of 10 RAs with 10 Master’s (MA) theses and to analyze it with the refined instrument. The comparison of these two corpora again served a dual purpose; besides piloting the analytical tool, this study set out to reveal differences between the two genres in terms of referential cohesion (Chapter 7 in this book).

The fourth stage of this research comprised of the analysis of MA theses and compares 10 higher and 10 lower rated papers. The previous comparative study had revealed that in order to formulate meaningful pedagogical implications, it was necessary to find a way to treat student errors in their uses of reference. This part of the research did not contribute directly to the development of the analytical tool; nevertheless, as the taxonomy of non-cohesive items was needed for a description of referential items, at this point, a taxonomy of typical learner errors was proposed for the analysis of student texts. Whereas Referential Cohesion Analysis is probably applicable to similar academic genres in English, the taxonomy of errors is only valid for describing this particular group of learners: university students who are advanced learners of English for Academic Purposes. Therefore, while this part of the study was highly relevant for our present purposes (formulating pedagogical implications for instruction in academic writing), the resulting taxonomy is not generalizable and is only applicable in similar contexts (see Chapter 8). In constructing this analytical tool, I found it important to ensure that it could be used for pedagogical purposes. Therefore, although it is firmly grounded in theory, the analytical tool should be easy to apply by anyone without extensive training. This time I asked a friend of mine who is an advanced English speaker (Judit Andruskó) to analyze an RA based on a summary of the instructions (Appendix I). She did not report any major difficulties.

The fifth stage is the final empirical study, which investigates referential features in the corpus of 40 papers (20 RAs and 20 MA theses) gradually built up during the stages of this research. This time the analysis of the whole corpus involved all approaches of the Referential Cohesion Analysis instrument: the description of cohesive reference supplemented with the categorization of non-cohesive items, as well as errors. Besides the discovery of writing techniques and strategies for reference that expert writers use and novice writers may lack, this study had another aim. It put to test the validity and reliability of the final version of the proposed analytical tool for Referential Cohesion Analysis. Triangulation of the data is ensured by the analysis itself, the number of cohesive items and that of non-cohesive items should add up to 100% of the total number of items in the texts. Besides, four texts were analyzed more than once, proving that the intra-coder reliability percentage is above 94%. The validity of the Referential Cohesion Analysis method and categories is reflected in its ability to represent cohesive chains in a structure that is easy to grasp and allows the researcher to not only get an insight into the cohesive referential structure of texts but also to compare different text types. Using this analysis produces results that give a summary of the analyzed paper, which in expert writers’ articles largely corresponds with their abstracts. This feature of the method further proves its validity. The method also predicts distinctions between higher- and lower-rated papers (see Chapter 9).

These five studies together provide the grounds for answering the main research questions. In order to make sure that pedagogical implications on the basis of the findings are relevant and are formulated in a way that will be useful for teachers and/or learners of academic writing. The Hungarian university context in which the studied theses were produced will also be explored.

Information will be gathered concerning the Hungarian academic discourse context in which the Hungarian MA theses were written. Besides, we will collect information on the teaching materials used, the types of tasks and text types used in teaching academic writing, manuals available to students. This will make it possible to develop materials for classes (see Chapter 10: Pedagogical implications) based on our Referential Cohesion Analysis. Other existing typologies of student errors and specialist information on teaching EAP will also be relevant here. This part of the paper, together with a text analysis supported by inter-coder reliability check will hopefully provide enough information for the triangulation of data (Hyland, 2000).