3.4.2.2. Pilot Study and the Validation of the Questionnaire

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The data collection took place in January-February 2021 when the questionnaire for secondary school teachers was posted to three groups on a popular social media platform and the participants of the interview study were asked to distribute it among their colleagues. As indicated at the beginning of the questionnaire (see in Appendix F), its completion was voluntary, and it took approximately 20-25 minutes to fill it in. After data collection, data were transferred to SPSS 22 and subjected to analysis.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Establishing internal validity and especially content validity was an essential step of the validation process. First, the questionnaire constructs and then the first draft of the questionnaire were given to experts, including my two group mates in the Language Pedagogy PhD Programme at ELTE and my supervisor, to obtain expert judgement on the relevance of the items to the issue, possible wording problems and the clarity of the instructions. Some changes were implemented as a result of the content validity and face validity check; the instructions were made clearer; some questions were deleted, and some new ones were added. The first instrument for secondary school teachers was created in Hungarian, the target population’s first language. The instrument was then translated to English and slightly modified to suit the university tutors’ population (e.g., in the case of frequency, considering that university tutors have maximum 2 sessions a week with their students, the labelling of the scales had to be changed). The English version was also subjected to expert judgement, and moderately modified based on their remarks. After the piloting phase and redrafting of the items in section 5, expert review was sought once again, and after the final minor modifications were implemented, the instrument was ready for being used in the main study.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The external validity of the study was ensured by the sampling procedure. Even though it was a case of convenience sampling (and snowball sampling), and data were collected online, mostly among the author’s teacher friends, but to involve participants teaching in different contexts and representing a variety of age groups, the questionnaire was circulated in specialized social media groups and via email by the participants of the interview study. Respondent self-selection based on topic preference was levelled off by giving the questionnaire a title that did not reveal much about what I intended to measure.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The reliability of the questionnaire was established by testing its internal consistency by calculating the Cronbach’s Alpha internal consistency reliability coefficients of the different scales, the results of which can be found in Table 3.7.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 3.7 The Reliability Analysis of the Scales in the Pilot Study
Scales
No of items
Cronbach’s alpha
1. Teachers’ likeliness to include certain global issues in their lessons
22
.96
2. Teachers’ likeliness to include certain local issues in their lessons
15
.94
3. Teachers’ likeliness to include certain intercultural issues in their lessons
17
.94
4. The frequency of the inclusion of global, local, and intercultural topics in the EFL class
5
.79
5. Teachers’ preferences in the inclusion of global content
6
.89
6. The importance of the inclusion of global, local, and intercultural issues
4
.83
7. Aspects influencing the inclusion of global content
23
.75
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In all cases, the Cronbach’s Alpha value reached the .6 threshold, which was an important first step, because as it was suggested by Dörnyei and Csizér (2012), above that threshold the scale is considered reliable. After calculating the Cronbach’s Alpha reliability coefficients, a principal component analysis (PCA) was run to test whether the grouped items measure the same dimension. In the case of the first three scales, it was not sensible to run this test, however by running factor analysis on them, there could be some new groups of topics observed, which teachers think similarly about. The items on scales 4, 5, and 6 seemed to be on the same dimension, so based on their reliability analysis and the PCA, they were deemed reliable and ready to be used for the main study. The seventh scale turned out to be problematic and the reason was that there should have been constructs created with an almost equal number of items so that this scale could work. A Varimax rotation factor analysis was run as an exploratory tool to identify the items that loaded together and constituted separate dimensions. Eight dimensions were identified in this way, however, after testing them again with PCA and computing their Cronbach’s Alpha, none of them emerged as reliable constructs. What became evident after the reliability check was that the variables influencing the teachers’ decisions about incorporating global content in their lessons cannot be analysed as one dimension as there are some underlying constructs and these constructs needed to be properly developed, with enough items. As detailed in the previous section (3.4.2.1), seven new constructs were created for the secondary school context (for 30 items) and eight constructs were created for the university context (for 33 items), and the modified questionnaire was considered as the final questionnaire for the main study.
 
Tartalomjegyzék navigate_next
Keresés a kiadványban navigate_next

A kereséshez, kérjük, lépj be!
Könyvjelzőim navigate_next
A könyvjelzők használatához
be kell jelentkezned.
Jegyzeteim navigate_next
Jegyzetek létrehozásához
be kell jelentkezned.
    Kiemeléseim navigate_next
    Mutasd a szövegben:
    Szűrés:

    Kiemelések létrehozásához
    MeRSZ+ előfizetés szükséges.
      Útmutató elindítása
      delete
      Kivonat
      fullscreenclose
      printsave