Mátyás Bánhegyi: The Effects of Politics and Ideology on the Translation of Argumentative Political Newspaper Articles
›
Chapter 7: Research design
›
7.10 Reliability of the constituent models of the Political Bias Screener
›
7.10.2 Reliability of macrostructure analysis

7.10.2 Reliability of macrostructure analysis

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The above-described procedure of macrostructure formation has been tested for reliability. The researcher and a co-coder carefully trained in the terminology and application of Tirkkonen-Condit’s (1985) Macrostructure Model first worked independently on establishing the macrostructure affected by the superstructure of the two source and the eight target texts. This was done as follows.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The starting level of the macrostructure reliability analysis is the Microlevel. At the Microlevel, every sentence is present and is represented by a number assigned to it as stated above. At Macrolevel 1, however, only certain sentences are present, while certain other sentences present at the Microlevel are not maintained at Macrolevel 1. Out of the sentences present at Macrolevel 1, only given sentences are maintained at Macrolevel 2 and so on with reference to each macrolevel. The macrostructure analysis continues until the highest macrolevel is reached.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In the table representing the macrostructure, the sentences present at the different macrostructure levels are recorded at the given level by indicating their numbers at the level concerned. After the macrostructure analyses were completed by both coders, the resulting macrostructure patterns for Coder 1 and Coder 2 were recorded in a tabular format. This is illustrated in Table 7.3 with reference to an imaginary seven-sentence string of text.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 7.3: Illustration of the reliability analysis of the macrostructure analysis: macrostructure patterns established by the two coders
Coder 1
Coder 2
Macrolev. 2
5
5
Macrolev. 1
1
4
5
6
2
4
5
6
Microlevel
1
2
3
4
5
6
7
1
2
3
4
5
6
7
Number of sentence
1
2
3
4
5
6
7
1
2
3
4
5
6
7
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

As a next step, each sentence is provided with a number according to the highest macrolevel it appears at plus 1 to compensate for the fact that the microlevel also constitutes a level, where all sentences are present. This number will be called the structure index of the given sentence. In our example, the respective structure indexes of the sentences of the seven-sentence sample text established on the basis of the coding effected by Coders 1 and 2 are indicated in Table 7.4.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 7.4: The structure indices of the sentences of the example string based on the two coders’ analysis
Coder 1
Coder 2
Structure index
2
1
1
2
3
2
1
1
2
1
2
3
2
1
Number of sentence
1
2
3
4
5
6
7
1
2
3
4
5
6
7
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The structure indices of Coders 1 and 2 are then compared. For the comparison, binary evaluation is used: if the given sentence has the same structure index for both Coders, the resulting agreement value will be 1. If such a relationship does not hold, the value with reference to the given sentence will be 0. In our example, the resulting agreement values are illustrated in Table 7.5.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 7.5: The resulting agreement values
Example string
Agreement value
0
0
1
1
1
1
1
Number of sentences
1
2
3
4
5
6
7
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

As a final step, the agreement values are added up and then divided by the total number of sentences within the text. In the example in Table 7.5, this works as follows: 0+0+1+1+1+1+1=5. Five is divided by 7 (as we have 7 sentences in the text string used in the example), which gives the agreement index 5/7= 71%. Using this method, agreement indices have been calculated with reference to the two source and the eight target texts, and the resulting values have been rounded up. The results are shown in Table 7.6.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 7.6: The agreement indices with reference to the two source texts and the eight target texts
Text
Agreement index
Source Text 1
82%
Source Text 2
77%
Target Text 1
82%
Target Text 2
86%
Target Text 3
82%
Target Text 4
82%
Target Text 5
85%
Target Text 6
82%
Target Text 7
84%
Target Text 8
84%
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The indices vary between 77% and 86%, therefore the results of the macrostructure analyses may be regarded as reliable.
 
chevron_left
chevron_right
Tartalomjegyzék navigate_next
Keresés a kiadványban navigate_next
A kereséshez, kérjük, lépj be!
Könyvjelzőim navigate_next
A könyvjelzők használatához
be kell jelentkezned.
Jegyzeteim navigate_next
Jegyzetek létrehozásához
be kell jelentkezned.
    Kiemeléseim navigate_next
    Mutasd a szövegben:
    Szűrés:
    Kiemelések létrehozásához
    MeRSZ+ előfizetés szükséges.
      Útmutató elindítása
      delete
      Kivonat
      fullscreenclose
      printsave