Measuring online learning strategies during the covid-19 pandemic: instrument validity and reliability

This study aims to describe the validity and reliability of the learning strategy questionnaire instrument used during the Covid-19 pandemic. By using a quantitative approach, measurement of the learning strategy questionnaire was developed based on six language learning strategies, namely: cognitive strategies, memory strategies, compensation strategies, metacognitive strategies, affective strategies, and social strategies. A total of 17 students were randomly selected to be the test subjects of the learning strategy instrument. By using SPSS 23, the bivariate product moment correlate test was used to determine the validity of the questionnaire items and Cronbach's Alpha test to determine the reliability of the questionnaire. The results showed that as many as 124 questionnaire items were tested on respondents, it was found that 92 questionnaire items were declared valid and overall the questionnaire items were declared reliable or consistent. Thus, as many as 92 items of this learning strategy questionnaire statement can be used to measure the types of language learning strategies used by students, both for online learning and face-to-face learning.


Introduction
Measurement of the validity and reliability of research instruments in addition to realizing measurable research results is also to obtain valid research results. To get valid research results, researchers need valid and reliable measuring tools. The use of valid and reliable measuring instruments to measure abstract concepts is an important factor in determining the quality of research. If the researcher pays attention to the validity and reliability throughout his research, the research findings will be valid and reliable. In the measurement of Learning Outcomes Tests, validity and reliability are also the main requirements to get good test results, in addition to the difficulty index and discrimination index (Wenno, Tuhurima, & Manoppo, 2021). Validity relates to the ability to test learning outcomes to measure the circumstances to be measured. Validity testing can be grouped into three, namely: content validity, criterion validity, and construct validity. Content validity testing can be done by examining the items, asking for expert judgment, and calculating the correlation of items with the total. Testing the validity of the criteria in the form of congruent and predictive validity. The construct validity test was conducted by examining the items, asking for expert judgment, convergence and discriminability, multitrait-multimethod, and factor analysis. Meanwhile, reliability is a coefficient that shows the ability of learning outcomes tests to provide relatively fixed and consistent measurement results. Reliability is seen as internal stability or internal consistency. Reliability decisions are made by confirming the reliability coefficient of the calculation results with certain limit criteria (Purwanto, 2011). Validity and reliability are concepts used to evaluate the nature of psychometric scales in research. Reliability is related to reproducibility while validity is related to scale accuracy (Mikaberidze, 2007) The measurement of the instrument accurately without any doubt with other features is defined as validity. Validity is the degree to which it serves the intended use of the scale. Reliability is a measurement instrument that provides consistent results under the same conditions. Validity and reliability are two critical characteristics that must exist in every measurement instrument. Conducting evaluations or research with a scale that does not have valid and reliable conditions is considered incorrect and unethical and the results of the research will be scientifically controversial (Sürücü & Maslakçı, 2020). Validity is the product of validation. Validation is a process carried out by the compiler or instrument user to collect empirical data to support the conclusions generated by the instrument score. Meanwhile, validity is the ability of a measuring instrument to measure its measuring target. A measuring instrument is said to have validity if the measuring instrument is suitable for measuring the object that should be measured and in accordance with certain criteria. This means that there is a match between the measuring instrument and the measurement function and the measurement target. Reliability is the state of the instrument that shows reliable measurement results (not changing, consistent). A reliable instrument is an instrument that, when used to measure the same subject or object at different times and measurements made by different people, the results remain the same (Amirono & Daryanto, 2016). Reliability testing using Alpha Cronbach (Purwanto, 2011). Cronbach's alpha is seen as evidence of instrument quality and is widely used by researchers to show that tests and scales are fit for purpose (Taber, 2018). Reliability concerns often arise due to external factors that can affect the strength and significance of the test. The external factors in question are: respondent's age, gender, education level, religion, rural/urban living environment, type of survey, and the relevance of the subject to the survey conducted (Ursachi, Horodnic, & Zait, 2015).
Several studies were conducted for the purpose of testing research instruments, namely: writing skills assessment instruments (Moses & Yamat, 2021), teacher-made test instrument (Agung Setiabudi, Mulyadi, 2019), 2013 Curriculum BSE evaluation instrument (Safi'i & Salamah, 2021), an instrument for measuring the level of satisfaction of health profession students with online learning during the Covid-19 pandemic (Kulsum & Suryadi, 2021), instrument for measuring the level of fear of the people of South Korea against Covid-19 (Han, Park, & Lee, 2021), and instruments for measuring student perceptions of the implementation of the 2013 Curriculum for Indonesian subjects (Oktavia, Irwandi, Rajibussalim, Mentari, & Mulia, 2018). Other studies were also conducted to test the validity of educational products, for example, genre-based text-writing teaching materials (Sari & Nurgiyantoro, 2020), short story writing learning module (Ramadhanti & Basri, 2014), and problem-based learning syntactic learning tools (Helda & Ramadhanti, 2019).
However, the development of instruments related to learning strategies used by students during online learning during the Covid-19 pandemic has not yet been carried out. Learning strategies are one of the important aspects of achieving learning objectives. To find out the achievement of learning objectives, it is necessary to measure the learning strategies used. To measure the instruments used both prior research and research to be conducted using the measurement of validity and reliability. Therefore, it is very important to check validity and reliability when using scales or questionnaires (Mikaberidze, 2007).
Similar to previous research, to determine the learning strategies used by students, a measuring instrument in the form of a questionnaire was used. The questionnaire is a list of questions given to other people who are willing to respond according to user requests. The questionnaire aims to collect complete information about a problem from respondents without feeling worried if respondents give answers that are not in accordance with reality in filling out the list of questions. In this study, the questionnaire used was a closed questionnaire. Through this closed questionnaire, respondents are asked to choose one answer that suits their characteristics by putting a checklist on the answer choices (Riduwan, 2007).
The answer choices were developed using a Likert scale. The Likert scale is used to measure attitudes, opinions, and perceptions of a person or group about social events or phenomena (Riduwan, 2007). The social phenomenon referred to in this study is the learning strategy used during online learning during the Covid-19 pandemic. By using a Likert scale, the variables to be measured are translated into dimensions, dimensions are translated into sub-variables, then the sub-variables are translated into indicators that can be measured. This measurable indicator is used as a starting point for making instrument items in the form of questions or statements that need to be answered by respondents. Each answer is associated with a form of a statement or attitude support expressed in words in the form of the respondent's answer.
Based on the things mentioned above, this study aims to explain two research formulations. First, how is the preparation of the learning strategy questionnaire? Second, how are the results of testing the validity and reliability of the learning strategy questionnaire instrument?

Method
This study was a quantitative study that aims to test the validity and reliability of language learning strategy research instruments used by students during online learning during the Covid-19 pandemic. A total of 17 students Institut Agama Islam Negeri (IAIN) Imam Bonjol Padang were randomly selected to respond to a learning strategy questionnaire. The instrument used in the form of a questionnaire compiled based on the theory of language learning strategies proposed by Oxford (1990). The data of this research are in the form of test scores of learning strategy questionnaires. Data collection is done online via a Google Forms. A total of 124 questionnaire items were given to the selected respondents. Respondents were given time to respond according to their learning experience.
After the data was collected, data analysis was carried out to determine the validity and reliability of the learning strategy questionnaire instrument. Testing the validity and reliability of the questionnaire instrument using Statistical Package Social Science (SPSS)-23. The validity of the questionnaire items was determined by performing a bivariate product-moment correlate test. The basis for making a valid questionnaire item decision is done by comparing the value of sig. (2-tailed) with a probability of 0.05. If the value of Sig. (2-tailed) < 0.05 and the Pearson Correlation is positive, the questionnaire item is declared valid. If the value of Sig. (2-tailed) < and the Pearson Correlation are negative, the questionnaire item is declared invalid. If the value of Sig. (2-tailed) > 0.05, it means that the questionnaire item is declared invalid. The reliability of the questionnaire was determined by performing Cronbach's Alpha test. The basis for making Cronbach's Alpha reliability test decisions, namely: if Cronbach's Alpha value > 0.60, it means that the questionnaire is declared reliable or consistent. Meanwhile, if Cronbach's Alpha value < 0.60 means that the questionnaire is declared unreliable or inconsistent. The basis for making Cronbach's Alpha reliability test decisions, namely: if Cronbach's Alpha value > 0.60, it means that the questionnaire/questionnaire is declared reliable or consistent. Meanwhile, if Cronbach's Alpha value < 0.60 means that the questionnaire is declared unreliable or inconsistent. The items of the learning strategy questionnaire statements that have been declared valid and reliable are presented in the appendix section.

Result and Discussion
In this section, two research formulations are described, namely: the preparation of a learning strategy questionnaire instrument and testing the validity and reliability of the learning strategy questionnaire.

Preparation of Learning Strategy Questionnaire Instruments
The preparation of the questionnaire instrument was carried out with the following steps, namely: identifying the research variables, describing the variables into sub-variables, looking for indicators/aspects of each sub variable, arranging the descriptors of each indicator, formulating each descriptor into instrument items, and completing the instrument. with filling instructions and preface (Riduwan, 2007). In accordance with the statement, the variables referred to in this study are learning strategies. The subvariables of the learning strategy variable consist of six, namely: cognitive strategies, memory strategies, compensation strategies, metacognitive strategies, affective strategies, and social strategies. (Oxford, 1990). Cognitive strategy sub variable indicators consist of four, namely: practicing, receiving and sending messages, analyzing and reasoning, and creating structure for input and output. The memory strategy sub-variable indicators consist of four, namely: creating mental linkages, applying images and sounds, reviewing well, and employing action. The compensation strategy sub-variable indicators consist of two, namely: guessing intelligently and overcoming limitations in speaking and writing. The indicator of the metacognitive strategy sub variable consist of three, namely: centering your learning, arranging and planning your learning, and evaluating your learning. The indicators of the affective strategy sub-variable consist of three, namely: lowering your anxiety, encouraging yourself, and taking your emotional temperature. The indicators of the social strategy sub-variable consist of three, namely: asking questions, cooperating with others, and empathizing with others. Each item is designed with a positive type of question (favorable) with a Likert scale, where each question has five answer choices with a score, namely: strongly disagree (score 1), disagree (score 2), unsure (score 3), agree. (score 4), and strongly agree (score 5) (Sugiyono, 2009). The details of the learning strategy questionnaire instrument grid according to the description are visualized in Table 1 below. Testing the Validity and Reliability of the Learning Strategy Questionnaire The validity of the questionnaire was tested using SPSS-23. The test is called the Correlate Bivariate Product Moment Test. The Pearson correlation product-moment validity test uses the principle of correlating or connecting each item or question score with the total score obtained from the respondents' answers. The basis for making product-moment validity test decisions is done by comparing the value of sig. (2-tailed) with a probability of 0.05. If the value of Sig. (2-tailed) < 0.05 and the Pearson Correlation is positive, the questionnaire item is declared valid. If the value of Sig. (2-tailed) < and the Pearson Correlation are negative, the questionnaire item is declared invalid. If the value of Sig. (2-tailed) > 0.05, it means that the questionnaire item is declared invalid. The results of the questionnaire item validity test for each sub variable, namely: cognitive strategies, memory strategies, compensation strategies, metacognitive strategies, affective strategies, and social strategies.

Cognitive Strategies
Cognitive strategy sub variable indicators consist of four, namely: practicing, receiving and sending messages, analyzing and reasoning, and creating structure for input and output. The number of items for this sub variable is 30 items. The sub-indicators of practicing consist of five, namely: repeating, formally practicing with sounds and writing systems, recognizing and using formulas and patterns, recombining, and practicing naturalistically. The number of items for this sub-indicator is 10 items. Of the 10 items, 6 items were used, one item was revised, and the other three items were eliminated. The results of the validation of the sub-indicator items are visualized in Table 2 below. The sub indicators of receiving and sening message consist of two, namely: getting the idea quickly and using resources for receiving and sending messages. The number of items for this sub indicators is 4 items. Of the 4 items, 2 items were used and 2 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 3 below. The sub indicators of analyzing and reasoning consist of five, namely: reasoning deductively, analyzing expressions, analyzing contrastively (across languages), translating, and transfering. The number of items for this sub indicators is 10 items. Of the 10 items, 6 items were used, 3 items were eliminated, and one item was revised. The results of the validation of the sub indicator items are visualized in Table 4 below. The sub indicators of creating structure for input and output consist of three, namely: taking note, summarizing, and highlighting. The number of items for this sub indicators is 6 items. Of the 6 items, 5 items were used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 5 below.

Memory Strategies
Memory strategy sub variable indicators consist of four, namely: creating mental linkages, applying images and sounds, reviewing well, and employing action. The number of items for this sub variable is 20 items. The sub indicators of creating mental linkages consist of three, namely: grouping, associating/elaborating, and placing new words into a context. The number of items for this sub indicators is 6 items. Of the 6 items, 5 items were used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 6 below. The sub indicators of applying images and sounds consist of four, namely: using imagery, semantic mapping, using keywords, and representing sounds in memory. The number of items for this indicators is 8 items. Of the 8 items, 6 items were used and 2 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 7 below. The sub indicator of reviewing well consits of one, nemaly structured reviewing. The number of items for this indicator is 2 items. Of the 2 items, one item was used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 8 below. The sub indicators of employing action consist of two, namely: using physical response or sensation and using mechanical techniques. The number of items for this indicators is 4 items. Of the 4 items, 3 items were used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 9 below.

Compensation Strategies
Compensation strategy sub variable indicators consist of two, namely: guessing intelligently and overcoming limitations in speaking and writing. The number of items for this sub variable is 20 items. The sub indicators of guessing intelligently consist of two, namely: using linguistic clues and using other clues. The number of items for this indicators is 4 items. Of the 4 items, 2 items were used and 2 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 10 below. The sub indicators of overcoming limitations in speaking and writing consist of eight, namely: switching to the mother tongue, getting help, using mime or gesture, avoiding communication partially or totally, selecting the topic, adjusting or approximating the message, coining words, and using a circumlocution or synonym. The number of items for this indicators is 16 items. Of the 16 items, 10 items were used, 2 items were revised, and 4 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 11 below.

Metacognitive Strategies
Metacognitive strategy sub variable indicators consist of three, namely: centering your learning, arranging and planning your learning, and evaluating your learning. The number of items for this sub variable is 22 items. The sub indicators of centering your learning consist of three, namely: overviewing and linking with already know material, paying attention, and delaying speech production to focus on listening. The number of items for this indicators is 6 items. Of the 6 items, all items were used. The results of the validation of the sub indicator items are visualized in Table 12 below. The sub indicators of arranging and planning your learning consist of six, namely: finding out about language learning, organizing, setting goals and objectives, identifiying the purpose of a language task (purposeful listening/reading/speaking/writing), planning for a language task, and seeking practice opportunities. The number of items for this indicators is 12 items. Of the 12 items, 8 items were used, 3 items were eliminated, and one item was revised. The results of the validation of the sub indicator items are visualized in Table 13 below. The sub indicators of evaluating your learning consist of two, namely: self-monitoring and self-evaluating. The number of items for this indicators is 4 items. Of the 4 items, all item were used. The results of the validation of the sub indicator items are visualized in Table 14 below.

Affective Strategies
Affective strategy sub variable indicators consist of three, namely: lowering your anxiety, encouraging yourself, and taking your emotional temperature. The number of items for this sub variable is 20 items. The sub indicators of lowering your anxiety consist of three, namely: using progressive relaxation, deep breathing, or mediation, using music, and using laughter. The number of items for this indicators is 6 items. Of the 6 items, 2 items were used, 2 items were revised, and 2 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 15 below. The sub indicators of encouraging yourself consist of three, namely: making positive statements, taking risks wisely, and rewarding yourself. The number of items for this indicators is 6 items. Of the 6 items, 5 items were used and one items was eliminated. The results of the validation of the sub indicator items are visualized in Table  16 below. The sub indicators of taking your emotional temperature consist of four, namely: listening to your body, using a checklist, writing a language learning diary, and discussing your feeling with someone else. The number of items for this indicators is 8 items. Of the 8 items, 4 items were used and 4 items were eliminated. The results of the validation of the sub indicator items are visualized in Table 17 below.

Social Strategies
Social strategy sub variable indicators consist of three, namely: asking questions, cooperating with others, and empathizing with others. The number of items for this sub variable is 12 items. The sub indicators of asking questions consist of two, namely: asking for clarification or verification and asking for correction. The number of items for this indicators is 4 items. Of the 4 items, all item were used. The results of the validation of the sub indicator items are visualized in Table 18 below. The sub indicators of cooperating with others consist of two, namely: cooperating with peers and cooperating with proficient users of the new language. The number of items for this indicators is 4 items. Of the 4 items, 3 items were used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 19  The sub indicators of empathizing with others consist of two, namely: developing cultural understanding and becoming aware of others' thoughts and feelings. The number of items for this indicators is 4 items. Of the 4 items, 3 items were used and one item was eliminated. The results of the validation of the sub indicator items are visualized in Table 20 below.   Table 21 Case Processing Summary informs about the number of samples or respondents (N) analyzed in the SPSS program, namely N as many as 17 students. Because there is no empty data (in the sense that all respondents' answers are filled in) then the number of valid answers is 100%. Thus, a learning strategy questionnaire that is declared valid and reliable can be used to measure the learning strategies used by students.

Conclusion
Valid and reliable research instruments are needed to obtain valid data about the object under study. Measurement of the validity and reliability of learning strategy instruments used by students during online learning during the Covid-19 pandemic. The learning strategy in question consists of direct and indirect learning strategies, namely: cognitive strategies, memory strategies, compensation strategies, metacognitive strategies, affective strategies, and social strategies. A total of 124 questionnaire items were tested on 17 research subjects. A total of 92 questionnaire statements were declared valid and reliable so that they could be used to collect data about the learning strategy.

28
I am used to mapping concepts using a concept map/mind mapping to make it easy to remember. 29 I remember information only from the keywords used. 30 I'm used to using keywords to remember something concrete. 31 I can remember information just by listening carefully.

32
I am used to listening to information directly from the lecturer or listening to information directly to understand the subject matter.

33
I study the subject matter in a structured and repetitive way so that I am familiar with the material. 34 I practice the subject matter I learn to make it easier to remember.

35
I change the learning materials that are concrete to things that are easy to remember.

49
I use presentation slides that present information points that I will convey in the discussion.

50
I paraphrase the main idea according to the concept so that it is easy for me to understand and convey meaning.
D. Metcognitive Strategies Strongly Agree Agree Doubtful Disagree Strongly Disagree 51 I connect the material I learn with the concepts I already know.

52
I review the material that I have learned by rereading it and finding out the importance of certain concepts I have learned. 53 I pay close attention to my study assignments and ignore distractions. 54 I select concepts that support my learning tasks.

55
I always listen carefully to any useful information to support my learning. 56 I talk less and listen more for my learning effectiveness.

57
I make various efforts for my learning, including reading many sources and finding a comfortable environment to support my learning. 58 I use a lot of information to improve my learning.

59
I organize my learning to be more optimal by setting a schedule, a comfortable study environment, and a diary to record my learning activities. 60 I can't study to the fullest if I don't manage my schedule well. 61 I set long term goals and short term goals for my learning.

62
I listen to all the information and read various sources to increase my knowledge.