The focus of the validation phase of the learners’ questionnaire was to develop a tool that reliably measured the proposed constructs of the data collection instrument. The processes described in this subchapter had been published earlier as a validation study (Fekete, 2021), but were revised to be more detailed for the purposes of this monograph.

 

3.1.1.1 Research Questions. The pilot study was designed to answer five research questions, the first of which was a technical research question (Pilot1RQ1) focusing on the reliability of the designed instrument followed by some more traditional research questions (Pilot1RQs2-5) to collect preliminary results. The final research questions of the validation study were the following:

 

3.1.1.2 Developing the Data Collection Instrument. The data collection instrument was a questionnaire which was designed for the purposes of this study following the protocols proposed by Dörnyei (2003; 2007) and Dörnyei and Csizér (2012): before the pilot phase, a think-aloud pre-pilot was administered on two EFL teacher education major volunteers that resulted in the final wording of the items to ensure content validity. Besides the background variables, the piloted questionnaire consisted of nine constructs that altogether included 54 items (Appendix A) which were mixed-up to ensure that respondents did not simply repeat their answers when rating similar items (Dörnyei & Csizér, 2012). The questionnaire was administered in Hungarian, which was the first language of the participants to maximise that respondents understand the items and that the questionnaire produces reliable results. The questionnaire started with a message to the participants starting with a short description of the research, asking them to answer the questions honestly; they were reassured of their anonymity, and they were offered a brief definition of ICT. The definition was adapted and translated from Hew and Brush (2007), who defined ICT as “computing devices such as desktop computers, laptops, handheld computers, software, or internet” (p. 225).

 

 

The questionnaire consisted of the following constructs (with sample items, reference to the sources that provide the rationale for inclusion from the Empirical and theoretical background, and the perspectives of free time or learning use or both):

 

3.1.1.3 Participants. The study was carried out in the Hungarian university context involving 45 university students (37 females and eight males) of two universities of the capital city, Budapest, in the spring of 2019. All respondents forming a non-probability convenience sample were Hungarian English majors, 15 of them (33%) were English Studies BA students and 30 of them (67%) were enrolled in the EFL Teacher Education (MA) programme. Respondents were between the ages of 19 and 27, with a mean average of 22.27 years of age (SD = 1.95). The gender distribution of the sample was 37 females and eight males, which suggests that females greatly outnumber males in Hungarian English Studies programmes, which keys in with the observations of Csizér and Tankó, (2017) conducted in a very similar context. Participants were between their first and sixth year of studies, the mean was 3.16 (SD = 1.52).

3.1.1.4 Methods of Data Collection. For the validation of the instrument, a paper and pencil questionnaire was administered at two universities in the spring of 2019. Participation in the study was voluntary and the informants were granted anonymity. Filling out the questionnaires lasted for approximately 15 minutes. Data was first registered in an Excel document, and then the file was imported to Statistical Package for Social Sciences (SPSS) version 22.

 

3.1.1.5 Methods of Data Analysis. Data was analysed using Statistical Package for Social Sciences (SPSS) version 22. To answer the first, technical research question, the items of the questionnaire were reorganised into constructs and construct validity was checked. Constructs were only accepted as reliable if their Cronbach’s alpha value reached the minimum 0.60 cut-off value as well as if all individual items of the construct loaded to the same dimension applying principal component analysis (Dörnyei & Csizér, 2012). Furthermore, the scales were also tested for skewness and kurtosis following Curran et al.’s (1996 as cited by Ryu, 2011, p. 1073) cut-off margins of maximum level of skewness = 3 and kurtosis = 21. These values are justified because “Curran et al. (1996) examined a wider range of skewness and kurtosis values; they found that the inflation of the ML [maximum likelihood] test statistic was more severe with skewness = 3, kurtosis = 21 than with skewness = 2, kurtosis = 7” (Ryu, 2011, p, 1073). Thus, these cut-off values were accepted as normal distribution that allows for running parametric statistical tests on the dataset (Dörnyei & Csizér, 2012; O’Leary, 2017).