Annamária Kótay-Nagy: DIFFERENTIATING INSTRUCTION WITH TECHNOLOGY IN THE HUNGARIAN EFL CLASSROOM
›
3 Research Design and Methods
›
3.2 Phase 1: Exploring Hungarian Primary and Secondary School EFL Teachers’ Beliefs and Self-Reported Practices of DI and TEDI (Studies 1 and 2)
›
3.2.2 The Questionnaire Study (Study 2)

3.2.2 The Questionnaire Study (Study 2)

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.1 Methods of Data Collection. The aim of Study 2, in line with the exploratory sequential design, was to “refine and extend the qualitative findings” (Creswell, 2015, p. 546) of Study 1 and to test those findings in terms of their distribution in the population (Dörnyei, 2007). A questionnaire was deemed to be the most suitable instrument to use for this study, as it allowed for the collection of numerical or directly quantifiable data from a large group of people, which could then be used to determine the relationship between predefined categories (Dörnyei & Taguchi, 2010).

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The instrument was specifically designed for the purposes of the present study. The development, piloting, and validation processes are detailed in Section 3.2.2.2, while information about the participants, data collection procedures, and methods of data analysis is provided in Sections 3.2.2.3 and 3.2.2.4. Portions of the following sections appeared in a prior publication (Kótay-Nagy, 2025), with certain parts reproduced here verbatim and extended with additional analysis.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.2 The Development, Piloting and Validation of the Questionnaire

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.2.1 Development of the Instrument. The questionnaire, written in Hungarian, began with an introduction inviting participants to share their DI and TEDI beliefs. They were assured of anonymity, voluntariness, and that there were no right or wrong answers.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The questionnaire was organised around three main topics: teachers’ beliefs and self-reported practices of DI and TEDI, as well as the presence of certain enablers of DI. The latter, which covered themes such as professional preparedness and cooperation with stakeholders, both identified as enablers in Study 1, was included in the measurement to determine if their presence significantly influences teachers’ reported practices of DI.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The said topics were examined through 11 constructs presented in the form of multi-item scales. Participants were asked to respond to the items, worded as statements, on a 5-point Likert-scale, depending on the extent to which they felt that the items were true for them (1 meaning ‘not true at all’, 5 meaning ‘perfectly true’). Some items were adapted from previous questionnaire studies examining teachers’ beliefs and practices of DI (Letzel et al., 2020; Melesse, 2015; Santangelo & Tomlinson, 2012; Tzanni, 2018; Zelalem et al., 2022), while other items were developed specifically for this study, based on the themes identified in Study 1. Below is a list of the 11 constructs, each presented with an operational definition and a sample item:

Jegyzet elhelyezéséhez, kérjük, lépj be.!

  1. Beliefs: Importance of DI (5 items; Letzel et al., 2020, Tzanni, 2018; newly developed items): The extent to which participants consider the concept of DI to be important in TEFL. Sample item: I believe it is very important for EFL teachers to be able to differentiate effectively.
  2. Beliefs: Importance of TEDI (5 items; newly developed items): The extent to which participants consider the potentials of ICT for DI to be important in TEFL. Sample item: I believe that ICT tools make differentiation easier for EFL teachers.
  3. Beliefs: DI self-efficacy (4 items; Zelalem et al., 2022; newly developed items): The extent to which EFL teachers believe that they can effectively use DI in their lessons. Sample item: I believe I can effectively cater for my students’ individual needs.
  4. Practices: Differentiation of the learning environment (6 items; Santangelo & Tomlinson, 2012; Tzanni, 2018; Zelalem et al., 2022): The extent to which EFL teachers report to create a climate in the classroom that is conducive to differentiated teaching. Sample item: I take deliberate efforts to create a welcoming, safe atmosphere in my classes.
  5. Practices: Differentiation by readiness (7 items; Santangelo & Tomlinson, 2012; Tzanni, 2018; Zelalem et al., 2022): The extent to which EFL teachers report to adjust the content, process and product of teaching based on students’ readiness levels. Sample item: I strive to present new material at diverse levels of complexity to cater for my students’ varying English proficiency.
  6. Practices: Differentiation by interests (5 items; Santangelo & Tomlinson, 2012; Tzanni, 2018; Zelalem et al., 2022): The extent to which EFL teachers report to adjust the content, process and product of teaching based on students’ individual interests. Sample item: I strive to tailor materials to my students’ individual interests.
  7. Practices: Differentiation by learning profile (6 items; Santangelo & Tomlinson, 2012; Tzanni, 2018; Zelalem et al., 2022): The extent to which EFL teachers report to adjust the content, process and product of teaching based on students’ learning profiles (e.g., grouping orientation, learning styles). Sample item: I use various formats (e.g., video, audio, online texts, books) to present new material.
  8. Practices: TEDI (8 items; newly developed items): The extent to which EFL teachers report to incorporate technology into their teaching for DI purposes. Sample item: I use ICT tools to create differentiated activities.
  9. Presence of enablers: Professional preparedness (3 items; newly developed items): The extent to which EFL teachers perceive to be prepared for implementing DI. Sample item: I feel confident about my methodological readiness to apply differentiation in my teaching.
  10. Presence of enablers: ICT (3 items; Illés & Csizér, 2018; newly developed items). The extent to which participants report ICT to be present in their teaching contexts. Sample item: I find the classrooms where I teach English to be well-equipped with ICT tools.
  11. Presence of enablers: Cooperation with stakeholders (5 items; Letzel et al., 2020; newly developed items): The extent to which participants report to cooperate with third parties (e.g., teaching staff, school administrators, parents) to enhance the implementation of DI. Sample item: Among the teaching staff, we share tips and advice with each other on the methodology of differentiation.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

In addition, the questionnaire aimed to understand what importance EFL teachers attach to various learner differences in TEFL as well as to certain challenges and enablers commonly associated with the implementation of DI. As my main objective with this measurement was to obtain a general understanding of participants’ overall judgement of these constructs (Poon et al., 2002), and these constructs were specific, narrow in scope (Fuchs & Diamantopoulos, 2009), and factual rather than non-factual (Dörnyei, 2007), I opted for the use of single-item scales instead of multi-item scales, as their use is considered as an acceptable form of measurement under the mentioned circumstances. Respondents rated the importance of each of these aspects on a 5-point scale, and they could also mention further aspects in open-ended questions following each of these three topics, which were as follows:

Jegyzet elhelyezéséhez, kérjük, lépj be.!

  1. Beliefs about addressing learner differences (Santangelo & Tomlinson, 2012; Tzanni, 2018; newly developed items): Measured through 10 items, participants rated the importance of attending to various learner differences in TEFL (e.g., learners’ interests, learning profile, level of English proficiency).
  2. Perceived challenges of DI (Melesse, 2015; newly developed items): Measured through 6 items, teachers evaluated factors that may hinder the implementation of DI (e.g., increased preparation time, multitasking in class, acquainting students with differentiated learning).
  3. Perceived enablers of DI (Letzel et al., 2020; newly developed items): Involving 8 items, this section assessed factors that teachers believe facilitate the implementation of DI (e.g., in-service training on DI, access to differentiated materials, ICT tools in the classroom, support from school administrators).
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

While the above single questionnaire items could have been treated as background variables, in the present study I opted for treating them deliberately as thematic elements as they directly address two sub-research questions. Beliefs about addressing learner differences corresponds to RQ 1.2 and focuses on participants’ conceptualisation of DI by identifying which learner differences they believe are most important to address, while Perceived challenges of DI and Perceived enablers of DI correspond to RQ 2.2, as they measure the importance participants attach to specific challenges and enablers in implementing DI. The grouping of the questionnaire items was informed both by the existing literature (e.g., Letzel et al., 2020; Melesse, 2015; Santangelo & Tomlinson, 2015; Tzanni, 2018) and by the findings of Study 1, which addressed the same sub-research questions (RQ 1.2 and 2.2).

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The final section of the questionnaire included some demographic questions about the participants’ background, such as their gender, age, qualification, years of teaching experience, the type of school they teach at and the region where their school is located.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

At the first stage of the validation of the questionnaire, I sought feedback from my supervisor and from one of my peers in the Language Pedagogy PhD Programme on the relevance and wording of the questionnaire items and the clarity of the instructions. Based on their feedback, I made some modifications to improve both the content and face validity, including the clarification of the instructions, the reformulation of certain items, and the addition of new items. Once the draft was finalised and deemed ready to be piloted, I created an online version of the instrument using Google Forms to make the distribution and administration of the questionnaire easier.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.2.2 Piloting. The instrument was piloted in the spring of 2023. I employed a combination of non-probability sampling methods, specifically convenience and snowball sampling as suggested by Dörnyei (2007), to capture as wide a range of perspectives as possible from the Hungarian primary and secondary school EFL teacher population. First, I sent the link to the questionnaire to my teacher colleagues and then asked them to recruit further participants by distributing the link in their professional circles.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Altogether 50 participants filled in the pilot questionnaire, all of whom were Hungarian EFL teachers. Of these participants, 12% (n = 6) identified as male and 88% (n = 44) as female. They represented various school contexts: 30 of them taught in secondary education, 19 participants taught in primary education, while 1 respondent taught at both levels of education. The teachers’ ages ranged from 26 to 65, with a mean value of 43.82 (SD = 10.77). The average years of teaching experience was 15.53 (SD = 11.52). The least experienced respondent had 6 months of experience, while the most experienced participant had 42 years of teaching practice. The geographical distribution of the participants appears balanced. 50% (n = 25) taught in the capital, Budapest, while the rest were spread across the other 7 statistical regions of Hungary.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.2.3 Reliability Analysis. The aim of the pilot study was to check if the questionnaire constructs produce reliable results. To this end, for each of the 11 scales, the Cronbach’s Alpha internal consistency reliability coefficients were calculated and unrotated principal component analyses (PCA) were ran to see if the grouped items loaded onto the same dimension (Székelyi & Barna, 2002). In all cases, the Cronbach’s Alpha value reached the .6 threshold (Dörnyei and Csizér, 2012). Regarding factor loadings, in the first round of PCA, 10 scales produced a single component, while in the case of one scale (differentiation of the learning environment) I had to remove one item from the scale (“I strive to ensure that my students always work with different partners”), so that the remaining items would load onto a single dimension. After deleting this item, the second round of PCA revealed uni-dimensionality for all scales. Table 8 presents the results of the reliability analysis.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 8 The Cronbach’s Alpha Reliability Coefficients and the Results of the Second Round PCA in the Pilot of Study 2
Scale (number of items)
Cronbach’s alpha
Number of components extracted by PCA
Beliefs: Importance of DI (5)
.83
1
Beliefs: Importance of TEDI (5)
.90
1
Beliefs: Self-efficacy beliefs in DI (4)
.88
1
Practices: Differentiation of the learning environment (5, upon deleting 1 item)
.66
1
Practices: Differentiation by readiness (7)
.83
1
Practices: Differentiation by interests (5)
.76
1
Practices: Differentiation by learning profile (6)
.65
1
Practices: TEDI practices (8)
.89
1
Enablers: Professional preparedness (3)
.72
1
Enablers: ICT (3)
.64
1
Enablers: Cooperation with stakeholders (5)
.72
1
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

As one item had to be deleted from the construct of differentiation of the learning environment, I decided to include two new items for this scale for the main study (“I take deliberate efforts to make myself approachable and available to my students”, “I follow up privately on conflicts and problems”), adapted from Santangelo and Tomlinson’s (2012) questionnaire measuring teachers’ DI practices. With this addition, the modified questionnaire was now considered as the final version to be used in the main study. The English translation of the questionnaire items is attached in Appendix F.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.3 Participants and Data Collection Procedures. The data collection in the main questionnaire study was administered online from May to July 2023. Similar to the pilot study, a mixture of non-probability sampling methods (Dörnyei, 2007), namely convenience and snowball sampling were used to make sure that the participants reflect the diversity of the population of Hungarian primary and secondary school EFL teachers as much as possible. First a database of publicly available email addresses of EFL teachers from primary and secondary schools in Hungary was compiled (comprising altogether almost 1000 addresses), and calls for participation, along with the link to the questionnaire, were sent out to these teachers (the text is attached in Appendix G). The link was also shared on social media platforms, and the author requested their former colleagues, some of whom were the heads of the English departments at their respective schools, to distribute the link among their colleagues as well.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

A total of 212 participants completed the questionnaire, all of whom were Hungarian EFL teachers. Of these participants, 14.2% (n = 30) identified as male and 85.8% (n = 182) as female. They represented various school contexts: 153 of them taught in secondary education, 49 participants taught in primary education, while 10 respondents taught at both levels of education. The teachers’ ages ranged from 24 to 67, with a mean value of 48.33 (SD = 8.74).

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The average years of teaching experience was 19.55 (SD = 9.90). The least experienced respondent had 8 months of experience, while the most experienced participant had 40 years of teaching practice. One participant did not provide information regarding their years of teaching experience. To facilitate comparison based on teaching experience, participants were grouped into four categories: novice teachers (n = 13) with 1–3 years of experience, proficient teachers (n = 26) with 4–8 years, expert teachers (n = 35) with 9–15 years, and master teachers (n = 137) with 16+ years. This classification was chosen because it corresponds to the teacher experience categories used in Hungary’s compulsory performance evaluation system for primary and secondary school teachers in public education (Educational Authority, 2019) and it also accords with categorisations used in the international literature (e.g., Berliner, 2004; Day et al., 2006).

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The geographical distribution of the participants appears balanced. 28.3% (n = 60) taught in the capital, Budapest, while the rest were spread across the other 7 statistical regions of Hungary. For the analysis, three groups were created: participants’ responses from Central Hungary (the capital and Pest county, n = 88), the Great Plain and Northern Hungary (i.e., Eastern Hungary, n = 65), and Transdanubia (i.e., Western Hungary, n = 59) were compared.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

3.2.2.4 Methods of Data Analysis. Data were analysed using the Statistical Package for the Social Sciences (SPSS) version 29. Before running descriptive and inferential statistical procedures, similar to the pilot study, the reliability of the multi-item scales was checked. To this end, Cronbach’s Alpha coefficients were calculated for each of the 11 scales to determine internal consistency, and unrotated principal component analyses (PCA) were performed (Székelyi & Barna, 2002) to check whether the items of the scale loaded onto the same dimension. With one exception (differentiation by learning profile), all scales met or exceeded the minimum acceptable Cronbach’s Alpha threshold of .6, as suggested by Dörnyei and Csizér (2012). The first round of PCA indicated that seven scales formed a single factor, while four scales (differentiation of the learning environment, differentiation by readiness, differentiation by learning profile, cooperation with stakeholders) required the removal of one or two items to achieve a single dimension. After these items were deleted, a second round of PCA confirmed that all scales showed uni-dimensionality (Appendix F presents the questionnaire items, with the items deleted from the analysis highlighted in italics). The finalisation of the constructs is presented in Table 9 below.
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 9 Construct Finalisation in Study 2
Name
Original
Modification
Finalised
Item n.
α
Comp. n.
Item n.
α
Comp. n.
1. Beliefs: Importance of DI
5
.89
1
–
5
.89
1
2. Beliefs: Importance of TEDI
5
.89
1
–
5
.89
1
3. Beliefs: DI self-efficacy
4
.81
1
–
4
.81
1
4. Practices: Differentiation of learning environment
7
.65
2
2 items deleted
5
.61
1
5. Practices: DI by readiness
7
.77
2
1 item deleted
6
.78
1
6. Practices: DI by interests
5
.74
1
–
5
.74
1
7. Practices: DI by learning profile
6
.57
2
1 item deleted
5
.61
1
8. Practices: TEDI
8
.87
1
–
8
.87
1
9. Enablers: Professional preparedness
3
.72
1
–
3
.72
1
10. Enablers: ICT
3
.62
1
–
3
.62
1
11. Enablers: Cooperation with stakeholders
5
.70
2
1 item deleted
4
.72
1
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Next, descriptive statistical methods (measures of central tendency, skewness, kurtosis) were used to test the normality of the data. As can be seen in Table 10, most scales showed normal distribution with their skewness and kurtosis values being within the acceptability range of ± 2.0 (George & Mallery, 2016).
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Table 10 Normality Test Results of the Scales in Study 2
Scale
M
SD
Skewness
Kurtosis
1. Beliefs: Importance of DI
4.44
0.65
–1.87
4.84
2. Beliefs: Importance of TEDI
4.06
0.84
–.84
.26
3. Beliefs: DI self-efficacy
3.70
0.66
–.51
.53
4. Practices: Learning environment
4.61
0.39
–1.51
2.46
5. Practices: Readiness
4.06
0.65
–.63
.32
6. Practices: Interests
3.24
0.76
–.08
–.52
7. Practices: Learning profile
4.06
0.57
–.90
1.20
8. Practices: TEDI
3.68
0.87
–.55
–0.1
9. Enablers: Professional preparedness
3.09
0.98
–.13
–.60
10. Enablers: ICT
3.75
0.79
–.32
–.38
11. Enablers: Cooperation with stakeholders
3.26
0.85
–.23
–.44
Note. Standard error of kurtosis: .17, standard error of skewness: .33
 

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Some scales did have values outside the ±2.0 range, but, as the sample size was above 200 (N = 212), these deviations from normality were not expected to significantly impact the results, in line with Tabachnick and Fidell’s (2007) observation that such non-normalities in large samples are unlikely to “make a substantive difference in the analysis” (p. 80). Besides, the kurtosis level of 4.84 in the scale Beliefs: Importance of DI is also qualitatively justifiable as it is in line with the expectation that teachers would generally consider DI to be an important element of their pedagogical practices. In light of these considerations, it was deemed appropriate to use parametric tests for the data analysis.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

Data were analysed with various statistical procedures. Sub-RQs 1.1, 1.2, 1.3, 2.1., and 2.2 were answered with descriptive statistics and group-related statistics, with the latter being used in two ways: paired-samples t tests were used to compare the mean values of different scales, while independent-samples t tests and ANOVAs were conducted to examine the differences in the participants’ mean values on various scales based on specific background variables. Sub-RQ 2.3, which focused on the variables that influence participants’ reported use of DI and TEDI, was examined through regression analysis. The level of significance was set for p < 0.05.

Jegyzet elhelyezéséhez, kérjük, lépj be.!

The responses given to the open-ended questions were subjected to structural coding (Saldaña, 2013). First, each response was examined to identify distinct phrases that pointed to either barriers or supportive factors related to DI. These were then assigned with initial codes that reflected the specific challenges or enablers mentioned by the participants. Through an iterative process, these codes were then reviewed and grouped, with similar or redundant codes merged into larger categories, thus making it possible to examine commonalities, differences and relationships between the responses given by the participants.
chevron_left
chevron_right
Tartalomjegyzék navigate_next
Keresés a kiadványban navigate_next
A kereséshez, kérjük, lépj be!
Könyvjelzőim navigate_next
A könyvjelzők használatához
be kell jelentkezned.
Jegyzeteim navigate_next
Jegyzetek létrehozásához
be kell jelentkezned.
    Kiemeléseim navigate_next
    Mutasd a szövegben:
    Szűrés:
    Kiemelések létrehozásához
    MeRSZ+ előfizetés szükséges.
      Útmutató elindítása
      delete
      Kivonat
      fullscreenclose
      printsave