3.3.4.1 Methods of Data Collection. As Mackey and Gass (2005) point out, when doing classroom research, apart from observing lessons, “additional data-gathering methods may be helpful in order to triangulate classroom data and provide multiple perspectives by accessing the learners’ insights into the events that have been observed” (p. 201). For this reason, besides conducting interviews with teachers, I also sought to investigate what the students think about their teachers’ TEDI practices, more specifically, to examine the extent to which they perceive TEDI as responsive to their individual needs, as well as the possible relationship between these perceptions of TEDI and students’ motivated learning behaviour, language learning experience and self-efficacy beliefs, i.e., cognitive and affective variables that have been shown to be associated with technology-enhanced differentiated learning contexts (e.g., Hustinx et al., 2019; Vargas-Parra et al., 2018). To this end, I opted for the use of a questionnaire as it facilitated the collection of self-reported data from all the students taught by the participants and offered the chance to gain descriptive, inferential, and explanatory information (Cohen et al., 2007). The instrument was constructed, piloted, and validated specifically for the purposes of the present study, details of which were published in an article (Kótay-Nagy, 2023b). Some parts of that publication are reproduced here verbatim, accompanied by additional commentary and analysis.

 

3.3.4.2 The Development, Piloting, and Validation of the Student Questionnaire .

3.3.4.2.1 Development of the Instrument. The questionnaire sought to examine the following two topics: (a) perceptions of TEDI, i.e., the extent to which students perceive that the use of ICT tools in the EFL lessons responds to their interests, learning profiles and readiness levels; (b) students’ motivated learning behaviour, language learning experience and self-efficacy beliefs with regard to learning English. The multi-item scales measuring students’ perceptions of TEDI were developed and validated for the purpose of this study based on the learner differences dimension of Tomlinson’s (1999, 2014) model of DI. In designing these questionnaire items, I aimed to focus on both content and modality but placed greater emphasis on the content (e.g., “The tasks that I complete with the help of ICT tools are close to my interests.”). My rationale behind this choice was that my primary focus was not on modality itself but on how the interplay between content and modality can create a new quality of learning material and the extent to which students perceive these materials as responsive to their needs.

1. Acceptance of ICT tools (4 items): The extent to which students accept the use of ICT tools in the EFL lessons. Example: I like using ICT tools in the English lessons.

2. Interest-based TEDI perceptions (4 items): The extent to which students perceive that the use of ICT tools in the EFL lessons responds to their interests. Example: The tasks that I complete with the help of ICT tools are close to my interests.

3. Readiness-based TEDI perceptions (5 items): The extent to which students perceive that the use of ICT tools in the EFL lessons responds to their readiness levels. Example: When I use an ICT tool to complete a task in the English lesson, I can do the task without major difficulties.

4. Learning profile-based TEDI perceptions (5 items): The extent to which students perceive that the use of ICT tools in the EFL lessons responds to how they learn best. Example: When I use an ICT tool to complete a task in the English lesson, I have every opportunity to do it in a way that I find most convenient.

5. Language learning experience (4 items): How positively students relate to their EFL learning experiences. Example: I really enjoy learning English.

6. Motivated learning behaviour (5 items): Students’ efforts and persistence in learning English. Example: I am willing to make a lot of effort in order to learn to speak English very well.

7. Self-efficacy beliefs (6 items): The extent to which students feel they have the ability to successfully perform foreign language related tasks. Example: I am confident that I can do the speaking tasks in the English lessons.

 

Participants were asked to respond to the items on a 5-point Likert-scale. The items were worded as statements, and respondents rated them depending on the extent to which they felt that the items were true for them (1 meaning ‘not true at all’, 5 meaning ‘perfectly true’). Besides the 33 items aiming to measure the above seven constructs, the first introductory section of the questionnaire invited students to indicate what ICT devices they use in the EFL lessons, how often, and for what purposes. This section had two objectives: to clarify the meaning of the term ‘ICT tools’ for the students and to identify the specific devices used by the students in the EFL lessons, along with their associated activities. The final section of the questionnaire included questions about the participants’ background, such as their age, the grade when they started learning English and their self-reported English proficiency.

 

3.3.4.2.2 Piloting. The pilot study was conducted with two main goals in mind: firstly, to evaluate the reliability of the multi-item scales in measuring the constructs, and secondly, to observe how the students reacted to the instrument, particularly in terms of the wording and length of the questionnaire. This second aspect was especially important as the participants were children (Bell, 2007).

3.3.4.2.3 Reliability Analysis and the Revision of the Questionnaire. As for construct validity, for each of the 7 scales, the Cronbach’s Alpha internal consistency reliability coefficients were calculated. The coefficients were all above the acceptability level of .7 (Dörnyei & Csizér, 2012) except for readiness-based TEDI perceptions and language learning experience. Following the deletion of two items from both scales, a second round of analysis found these constructs to be reliable. Next, unrotated principal component analyses (PCA) were ran to see if the grouped items loaded onto the same dimension (Székelyi & Barna, 2002). All the scales produced single components except for self-efficacy beliefs, where one item had to be deleted to ensure uni-dimensionality. After deleting this item, the second round of PCA revealed uni-dimensionality for all the scales. Table 18 presents the results of the reliability analysis.

 

 

During the administration of the questionnaire, the students did not ask for clarifications regarding the meaning of any items. However, comments were made about the length of the questionnaire. In response to this feedback, I chose to remove the introductory section that asked students about their use of ICT in the EFL lessons, including the frequency and purpose of use. Upon analysing the responses to this section, it became clear that it yielded descriptive data, with almost all the responses being the same. Since such data, I realised, could also be gathered from the teachers, I decided to shorten the questionnaire by deleting this section, and to focus only on the multi-item scales that assessed students’ opinions and attitudes, i.e., areas where variation in the responses were more likely to emerge.

3.3.4.3 Participants and Data Collection Procedures. The participants of the student questionnaire study (Study 5) were drawn from the same groups as those taught by the teachers examined in the observation study (Study 4). It is important to note, however, that except for Tímea’s group, the number of the student participants was not the same in Studies 4 and 5. This discrepancy can be explained by two reasons. Firstly, as the questionnaire was administered to the students by the teacher in a lesson following the observation, in certain cases (Bea’s and Emma’s group) some students were absent during filling in the questionnaire, while in Rebeka’s group, the opposite was true, i.e., more students were present when filling in the questionnaire than during the observation. Secondly, in Kamilla’s class, the parents of one student opted their children out from the questionnaire study. Although participant attrition is often viewed as a threat to the validity of quantitative research as it can potentially distort sample representativeness (Dörnyei, 2007), it is important to emphasise that the primary aim of this questionnaire study was to explore patterns within the examined sample, rather than to generalise findings to a larger population. Therefore, it is hoped that as long as the discrepancy between the student numbers in Studies 3 and 4 is taken into account during data analysis and interpretation, participant attrition does not considerably compromise the validity of the research.

3.3.4.4 Methods of Data Analysis. Data were analysed with the Statistical Package for the Social Sciences (SPSS) version 29. Before running descriptive and inferential statistical procedures, similar to the pilot study, the reliability of the multi-item scales was checked. For each of the 7 scales, the Cronbach’s Alpha internal consistency reliability coefficients were calculated and unrotated principal component analyses (PCA) were ran to see if the grouped items loaded onto the same dimension (Székelyi & Barna, 2002). In all cases, the Cronbach’s Alpha value reached the .6 threshold (Dörnyei & Csizér, 2012). In the first round of PCA, five scales produced a single component, while in the case of learning profile-based TEDI perceptions and motivated learning behaviour, I had to remove one item from each scale to ensure that the remaining items loaded onto single dimensions (the deleted items were “When I use an ICT tool to complete a task in the English lesson, learning feels easy”, and “Learning English is one of the most important things in my life”, respectively.) Upon deletion, the second round of PCA revealed uni-dimensionality for all the scales. Table 19 presents the final scales and item numbers included in the analysis.

 

 

As a next step, descriptive statistical methods (measures of central tendency, skewness, kurtosis) were used to test the normality of the data. As indicated in Table 20, the majority of the data showed normal distribution, with skewness and kurtosis values falling within the accepted range of ±2.0 (George & Mallery, 2016). Although two scales presented values slightly beyond this threshold, considering the relative normality of most scales as well as the robustness of parametric tests to smaller normality violations (Field, 2017), using parametric tests for data analysis was deemed acceptable.

 

 

Data were analysed with various statistical procedures. Sub-RQ 3.5, which measured students’ perceptions of TEDI, was answered with the help of descriptive statistics, while sub-RQ 3.6, which focused on the relationship between students’ perceptions of TEDI and the variables of motivated learning behaviour, language learning experience and self-efficacy beliefs, was examined through correlation and regression analyses. The level of significance was set for p < 0.05.