Study of metacognitive awareness at the individual level of students across study programs using Rasch modeling

Moh. Irma  Sukarelawan; Panji Hidayat; Widodo Widodo

doi:10.12928/jrkpf.v11i1.748

Authors

Moh. Irma Sukarelawan Universitas Ahmad Dahlan https://orcid.org/0000-0002-3823-3964
Panji Hidayat Universitas Ahmad Dahlan
Widodo Widodo Universitas Ahmad Dahlan

DOI:

https://doi.org/10.12928/jrkpf.v11i1.748

Keywords:

Metacognitive awareness, Rasch Modeling, Logit Value of Person, Wright map

Abstract

Evaluating students' metacognitive awareness is necessary to identify weaknesses and strengths in managing their cognition, which can help develop effective cognitive regulation to overcome academic challenges. Therefore, this research aims to examine students' level of metacognitive awareness using Rasch modeling. Survey research was conducted on 122 students at FKIP. Metacognitive awareness was evaluated using the 18-item Jr. MAI, and each item uses a 5-point Likert rating scale. Jr. MAI in the Google form is distributed for 2–3 weeks. Metacognitive awareness data was analyzed using the Logit Value of Person (LVP) approach and the Person Wright Map in Rasch modeling. The analysis results show that the distribution of metacognitive awareness shows that the majority of physics education students tend to have a higher level of metacognitive awareness, while the distribution of metacognitive awareness among elementary school teacher education students is more even across all categories. The implication of this research is the need to develop effective cognitive regulation in overcoming academic challenges through evaluating students' metacognitive awareness.

References

K. Craig, D. Hale, C. Grainger, and M. E. Stewart, "Evaluating Metacognitive Self-Reports: Systematic Reviews of the Value of Self-Report in Metacognitive Research," Metacognition Learn., vol. 15, no. 2, pp. 155-213, Aug. 2020, doi: 10.1007/s11409-020-09222-y.

https://doi.org/10.1007/s11409-020-09222-y

R. Abdellah, "Metacognitive Awareness and its Relation to Academic Achievement and Teaching Performance of Pre-service Female Teachers in Ajman University in UAE," Procedia - Soc. Behav. Sci., vol. 174, pp. 560-567, Feb. 2015, doi: 10.1016/j.sbspro.2015.01.707.

https://doi.org/10.1016/j.sbspro.2015.01.707

H. K. Ning, "A rasch analysis of the junior metacognitive awareness inventory with singapore students," Meas. Eval. Couns. Dev., vol. 51, no. 2, pp. 84-91, Apr. 2018, doi: 10.1080/07481756.2017.1358061.

https://doi.org/10.1080/07481756.2017.1358061

B. A. Allen and E. Armour-Thomas, "Construct Validation of Metacognition," J. Psychol., vol. 127, no. 2, pp. 203-211, Mar. 1993, doi: 10.1080/00223980.1993.9915555.

https://doi.org/10.1080/00223980.1993.9915555

B. Çetin, "Metacognition and self-regulated learning in predicting university students' academic achievement in Turkey," J. Educ. Train. Stud., vol. 5, no. 4, pp. 132-138, Mar. 2017, doi: 10.11114/jets.v5i4.2233.

https://doi.org/10.11114/jets.v5i4.2233

A. M. Amin, A. D. Corebima, S. Zubaidah, and S. Mahanal, "The Correlation between Metacognitive Skills and Critical Thinking Skills at the Implementation of Four Different Learning Strategies in Animal Physiology Lectures," Eur. J. Educ. Res., vol. 9, no. 1, pp. 143-163, Jan. 2020, doi: 10.12973/eu-jer.9.1.143.

https://doi.org/10.12973/eu-jer.9.1.143

E. Ahdhianto, Marsigit, Haryanto, and N. N. Santi, "The Effect of Metacognitive-Based Contextual Learning Model on Fifth-Grade Students' Problem-Solving and Mathematical Communication Skills," Eur. J. Educ. Res., vol. 9, no. 2, pp. 753-764, Apr. 2020, doi: 10.12973/eu-jer.9.2.753.

https://doi.org/10.12973/eu-jer.9.2.753

Y. Herlanti, "Kesadaran Metakognitif dan Pengetahuan Metakognitif Peserta Didik Sekolah Menengah Atas dalam Mempersiapkan Ketercapaian Standar Kelulusan pada Kurikulum 2013," J. Cakrawala Pendidik., vol. 3, no. 3, pp. 357-367, Dec. 2015, doi: 10.21831/cp.v3i3.7343.

https://doi.org/10.21831/cp.v3i3.7343

A. Young and J. D. Fry, "Metacognitive awareness and academic achievement in college students," J. Scholarsh. Teach. Learn., vol. 8, no. 2, pp. 1-10, Jan. 2008, [Online]. Available: http://www.tandfonline.com/doi/full/10.3109/0142159X.2010.487711.

A. Vrugt and F. J. Oort, "Metacognition, Achievement Goals, Study Strategies and Academic Achievement: Pathways to Achievement," Metacognition Learn., vol. 3, no. 2, pp. 123-146, Aug. 2008, doi: 10.1007/s11409-008-9022-4.

https://doi.org/10.1007/s11409-008-9022-4

W. Tamsyani, "Pengaruh Model Pembelajaran dan Kesadaran Metakognitif Terhadap Hasil Belajar Peserta Didik SMA dalam Materi Pokok Asam Basa," J. EST, vol. 2, no. 1, pp. 10-25, 2016, doi: 10.26858/est.v2i1.1887.

https://doi.org/10.26858/est.v2i1.1887

T. Khoiriah, "Strategi Pembelajaran Metakognitif Terhadap Hasil Belajar Siswa pada Konsep Sistem Pencernaan pada Manusia," J. Pengajaran Mat. dan Ilmu Pengetah. Alam, vol. 6, no. 1, p. 177, Sep. 2015, doi: 10.18269/jpmipa.v20i2.581.

https://doi.org/10.18269/jpmipa.v20i2.581

N. Solikhah, E. R. Winarti, and A. W. Kurniasih, "Keefektifan Model Guided Inquiry dengan Pendekatan Keterampilan Metakognitif terhadap Kemampuan Pemecahan Masalah," J. Kreano, vol. 5, no. 1, pp. 18-25, 2014, doi: 10.15294/kreano.v5i1.3273.

S. A. Coutinho, "The Relationship Between Goals, Metacognition, and Academic Success," Educate, vol. 7, no. 1, pp. 39-47, 2007.

A. Muhlisin, H. Susilo, M. Amin, and F. Rohman, "The Effectiveness of RMS Learning Model in Improving Metacognitive Skills on Science Basic Concepts," J. Turkish Sci. Educ., vol. 15, no. 4, pp. 1-14, 2018, doi: 10.12973/tused.10242a.

M. I. Sukarelawan and S. Sriyanto, "Mapping of profile students' metacognitive awareness in yogyakarta, Indonesia," J. Res. Phys. Educ. Res. Ris. dan Kaji. Pendidik. Fis., vol. 6, no. 2, pp. 56-62, Oct. 2019, doi: 10.12928/jrkpf.v6i2.14556.

https://doi.org/10.12928/jrkpf.v6i2.14556

Kemendikbud, "Salinan Peraturan Menteri Pendidikan Dan Kebudayaan Republik Indonesia Nomor 37 Tahun 2018." Jakarta, 2018.

G. Taasoobshirazi, M. Bailey, and J. Farley, "Physics metacognition inventory part II: Confirmatory factor analysis and rasch analysis," Int. J. Sci. Educ., vol. 37, no. 17, pp. 2769-2786, Nov. 2015, doi: 10.1080/09500693.2015.1104425.

https://doi.org/10.1080/09500693.2015.1104425

G. Taasoobshirazi and J. Farley, "Construct validation of the physics metacognition inventory," Int. J. Sci. Educ., vol. 35, no. 3, pp. 447-459, 2013, doi: 10.1080/09500693.2012.750433.

https://doi.org/10.1080/09500693.2012.750433

H. Haeruddin, Z. K. Prasetyo, and S. Supahar, "The development of a metacognition instrument for college students to solve physics problems," Int. J. Instr., vol. 13, no. 1, pp. 767-782, Jan. 2020, doi: 10.29333/iji.2020.13149a.

https://doi.org/10.29333/iji.2020.13149a

U. Aydin and B. Ubuz, "Turkish Version of the Junior Metacognitive Awareness Inventory: An Exploratory and Confirmatory Factor Analysis Bilişüstü Yetiler Envanteri'nin Türkçeye Uyarlanması: Açımlayıcı ve Doğrulayıcı Faktör Analizi," Educ. Sci., vol. 35, no. 157, pp. 32-47, 2010.

B. Kim, B. Zyromski, M. Mariani, S. M. Lee, and J. C. Carey, "Establishing the factor structure of the 18-item version of the junior metacognitive awareness inventory," Meas. Eval. Couns. Dev., vol. 50, no. 1-2, pp. 48-57, Apr. 2017, doi: 10.1177/0748175616671366.

https://doi.org/10.1177/0748175616671366

H. K. Ning, "The Bifactor Model of the Junior Metacognitive Awareness Inventory (Jr. MAI)," Curr. Psychol., vol. 38, no. 2, pp. 367-375, Apr. 2019, doi: 10.1007/s12144-017-9619-3.

https://doi.org/10.1007/s12144-017-9619-3

M. I. Sukarelawan, J. Jumadi, H. Kuswanto, and M. A. Thohir, "The Indonesian Version of the Physics Metacognition Inventory: Confirmatory Factor Analysis and Rasch Model," Eur. J. Educ. Res., vol. 10, no. 4, pp. 2133-2144, Oct. 2021, doi: 10.12973/eu-jer.10.4.2133.

https://doi.org/10.12973/eu-jer.10.4.2133

M. I. Sukarelawan, D. Sulisworo, J. Jumadi, H. Kuswanto, and S. A. S. A. S. A. Rofiqah, "Heat and temperature metacognition awareness inventory: A confirmatory factor analysis," Int. J. Eval. Res. Educ., vol. 10, no. 2, p. 389, Jun. 2021, doi: 10.11591/ijere.v10i2.20917.

https://doi.org/10.11591/ijere.v10i2.20917

M. I. Sukarelawan, J. Jumadi, H. Kuswanto, S. Soeharto, and F. N. Hikmah, "Rasch Analysis to Evaluate the Psychometric Properties of Junior Metacognitive Awareness Inventory in the Indonesian Context," J. Pendidik. IPA Indones., vol. 10, no. 4, pp. 486-495, Dec. 2021, doi: 10.15294/jpii.v10i4.27114.

https://doi.org/10.15294/jpii.v10i4.27114

R. Hidayat, H. Zulnaidi, and S. N. A. Syed Zamri, "Roles of metacognition and achievement goals in mathematical modeling competency: A structural equation modeling analysis," PLoS One, vol. 13, no. 11, p. e0206211, Nov. 2018, doi: 10.1371/journal.pone.0206211.

https://doi.org/10.1371/journal.pone.0206211

D. Alindra, A. Fauzan, and A. Asmar, "Metacognitive Awareness and Its Effect on Students' Problem Solving Ability in Implementing RME Approach," in Proceedings of the 1st International Conference on Innovation in Education (ICoIE 2018), 2019, vol. 178, no. ICoIE 2018, pp. 318-322, doi: 10.2991/icoie-18.2019.70.

https://doi.org/10.2991/icoie-18.2019.70

M. Bahari, A. Widodo, and N. Winarno, "Using Self-Regulated Learning to Promote Students' Metacognitive Awareness in Learning Electricity," in Proceedings of the Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar, MSCEIS 2019, 12 October 2019, Bandung, West Java, Indonesia, 2020, no. July, doi: 10.4108/eai.12-10-2019.2296408.

https://doi.org/10.4108/eai.12-10-2019.2296408

A. Fauzi and W. Sa'diyah, "The Metacognition of Pre-Service Biology Teachers: Awareness, Skills, Understanding, and Practices," in Proceedings of the 6th International Conference on Community Development (ICCD 2019), 2019, no. January, pp. 26-32, doi: 10.2991/iccd-19.2019.8.

https://doi.org/10.2991/iccd-19.2019.8

M. Yasir, A. Fikriyah, N. Qomaria, and A. T. Al Haq, "Metacognitive skill on students of science education study program: Evaluation from answering biological questions," JPBI (Jurnal Pendidik. Biol. Indones., vol. 6, no. 1, pp. 157-164, Mar. 2020, doi: 10.22219/jpbi.v6i1.10081.

https://doi.org/10.22219/jpbi.v6i1.10081

P. Susongko, M. Kusuma, and H. Widiatmo, "Using Rasch Model to Detect Differential Person Functioning and Cheating Behavior in Natural Sciences Learning Achievement Test," J. Penelit. dan Pembelajaran IPA, vol. 5, no. 2, p. 94, 2019, doi: 10.30870/jppi.v5i2.5945.

https://doi.org/10.30870/jppi.v5i2.5945

T. Rusch, P. B. Lowry, P. Mair, and H. Treiblmaier, "Breaking Free from the Limitations of Classical Test Theory: Developing and Measuring Information Systems Scales using Item Response Theory," Inf. Manag., vol. 54, no. 2, pp. 189-203, Mar. 2017, doi: 10.1016/j.im.2016.06.005.

https://doi.org/10.1016/j.im.2016.06.005

J. M. Linacre, "Winsteps® (Version 4.6.1) [Computer Software]." 2021, [Online]. Available: http://www.winsteps.com.

F. N. Hikmah, M. I. Sukarelawan, T. Nurjannah, and J. Djumati, "Elaboration of high school student's metacognition awareness on heat and temperature material: Wright map in Rasch model," Indones. J. Sci. Math. Educ., vol. 4, no. 2, pp. 172-182, 2021, doi: 10.24042/ijsme.v4i2.9488.

https://doi.org/10.24042/ijsme.v4i2.9488

B. Setiawan, M. Panduwangi, and B. Sumintono, "A rasch analysis of the community's preference for different attributes of Islamic banks in Indonesia," Int. J. Soc. Econ., vol. 45, no. 12, pp. 1647-1662, Dec. 2018, doi: 10.1108/IJSE-07-2017-0294.

https://doi.org/10.1108/IJSE-07-2017-0294

A. Blanc and A. J. Rojas, "Use of Rasch Person-Item Maps to Validate a Theoretical Model for Measuring Attitudes toward Sexual Behaviors," PLoS One, vol. 13, no. 8, p. e0202551, 2018, doi: 10.1371/journal.pone.0202551.

https://doi.org/10.1371/journal.pone.0202551

A. A. Walker, J. K. Jennings, and G. Engelhard, "Using person response functions to investigate areas of person misfit related to item characteristics," Educ. Assess., vol. 23, no. 1, pp. 47-68, 2018, doi: 10.1080/10627197.2017.1415143.

https://doi.org/10.1080/10627197.2017.1415143

M. Dür et al., "Development and validation of a self-reported questionnaire to assess occupational balance in parents of preterm infants," PLoS One, vol. 16, no. 11, p. e0259648, Nov. 2021, doi: 10.1371/journal.pone.0259648.

https://doi.org/10.1371/journal.pone.0259648

B. Sumintono and W. Widhiarso, Aplikasi model rasch untuk penelitian ilmu-ilmu sosial [Rasch model application for social sciences research]. Cimahi: Trim Komunikata Publishing House, 2014.

J. M. Guzmán and L. C. Grajo, "Preliminary Rasch Construct and Reliability Validation of TeleWrite: A Telehealth-Based Handwriting Assessment," OTJR Occup. Ther. J. Res., vol. 43, no. 3, pp. 342-350, Jul. 2023, doi: 10.1177/15394492231151883.

https://doi.org/10.1177/15394492231151883

H. M. Antunes, L. de C. Magalhães, G. C. Vasconcelos, B. L. C. Trindade, A. C. M. Gonzaga, and R. P. G. Antunes, "Catquest-9SF questionnaire: validation of the Portuguese version using the Rasch analysis," Arq. Bras. Oftalmol., vol. 86, no. 2, pp. 156-63, 2023, doi: 10.5935/0004-2749.20230014.

K. Saghafi, S. M. R. Amirian, and M. E. Shirvan, "Differential Item Functioning Analysis of Persian Adaptation of Foreign Language Classroom Anxiety Scale Against Gender," Hum. Arenas, no. 0123456789, Jan. 2021, doi: 10.1007/s42087-020-00172-0.

https://doi.org/10.1007/s42087-020-00172-0

M. Ling, V. Pang, and C. C. Ompok, "Measuring Change in Early Mathematics Ability of Children Who Learn Using Games: Stacked Analysis in Rasch Measurement," in Pacific Rim Objective Measurement Symposium (PROMS) 2016 Conference Proceedings, Q. Zhang, Ed. Singapore: Springer Singapore, 2018, pp. 215-226.

https://doi.org/10.1007/978-981-10-8138-5_17

S. I. Hofer, R. Schumacher, and H. Rubin, "The test of basic Mechanics Conceptual Understanding (bMCU): using Rasch analysis to develop and evaluate an efficient multiple choice test on Newton's mechanics," Int. J. STEM Educ., vol. 4, no. 1, 2017, doi: 10.1186/s40594-017-0080-5.

https://doi.org/10.1186/s40594-017-0080-5

A. Ullenhag, L. Ek, A. Eliasson, and L. Krumlinde‐Sundholm, "Interrater and test-retest reliability of the Hand Assessment for Infants," Dev. Med. Child Neurol., vol. 63, no. 12, pp. 1456-1461, Dec. 2021, doi: 10.1111/dmcn.14967.

https://doi.org/10.1111/dmcn.14967

N. Geel, T. Passeron, A. Wolkerstorfer, R. Speeckaert, and K. Ezzedine, "Reliability and validity of the Vitiligo Signs of Activity Score (VSAS)," Br. J. Dermatol., vol. 183, no. 5, pp. 883-890, Nov. 2020, doi: 10.1111/bjd.18950.

https://doi.org/10.1111/bjd.18950

A. W. Sabekti and F. Khoirunnisa, "Penggunaan Rasch Model Untuk Mengembangkan Instrumen Pengukuran Kemampuan Berikir Kritis Siswa Pada Topik Ikatan Kimia," J. Zarah, vol. 6, no. 2, pp. 68-75, 2018, doi: 10.31629/zarah.v6i2.724.

https://doi.org/10.31629/zarah.v6i2.724

M. I. Sukarelawan, Jumadi, H. Kuswanto, T. Nurjannah, F. N. Hikmah, and M. F. Ramadhan, "Implementation of Rasch Model for Mapping Students' Metacognitive Awareness," J. Pendidik. Fis. Indones., vol. 17, no. 2, pp. 86-93, 2021, doi: 10.15294/jpfi.v17i2.27172.

https://doi.org/10.15294/jpfi.v17i2.27172

Y. M. Lu et al., "Measurement precision of the disability for back pain scale-by applying Rasch analysis," Health Qual. Life Outcomes, vol. 11, no. 1, pp. 1-10, 2013, doi: 10.1186/1477-7525-11-119.

https://doi.org/10.1186/1477-7525-11-119

J. Hincapié Pinzón, A. M. B. da Silva, M. C. S. da Silva, W. de Lara Machado, C. Moret-Tatay, and M. Z. de Oliveira, "Internal Structure, Invariance, and Rasch Analyses: A Work-Life Integration-Blurring Scale," Healthcare, vol. 10, no. 11, p. 2142, Oct. 2022, doi: 10.3390/healthcare10112142.

https://doi.org/10.3390/healthcare10112142

A. Sonia and Z. Rachid, "Use Of Rasch-andrich's Rating Scale Model In The Calibrating Of The Depression, Anxiety And Stress Scale (dass-42) On High School Students," J. Psychol. Educ. Sci., vol. 9, no. 2, pp. 133-147, 2023, [Online]. Available: https://www.asjp.cerist.dz/index.php/en/article/229774.

H. Baumeister, B. Abberger, A. Haschke, M. Boecker, J. Bengel, and M. Wirtz, "Development and calibration of an item bank for the assessment of activities of daily living in cardiovascular patients using Rasch analysis," Health Qual. Life Outcomes, vol. 11, no. 1, pp. 1-9, 2013, doi: 10.1186/1477-7525-11-133.

https://doi.org/10.1186/1477-7525-11-133

A. Joshi, S. Kale, S. Chandel, and D. Pal, "Likert Scale: Explored and Explained," Br. J. Appl. Sci. Technol., vol. 7, no. 4, pp. 396-403, Jan. 2015, doi: 10.9734/BJAST/2015/14975.

https://doi.org/10.9734/BJAST/2015/14975

B. Sumintono, "Pemodelan Rasch pada Asesmen Pendidikan: Suatu Pengantar," Konf. Guru dan Dosen Nas. 2015, no. November, pp. 1-14, 2015, [Online]. Available: http://eprints.um.edu.my/id/eprint/15603%0A.

M. A. A. Bakar and N. Ismail, "Exploring metacognitive regulation and students' interaction in mathematics learning: An analysis of needs to enhance students' mastery," Humanit. Soc. Sci. Rev., vol. 8, no. 2 Special issue, pp. 67-74, 2020, doi: 10.18510/hssr.2020.82e07.

https://doi.org/10.18510/hssr.2020.82e07

L. De Backer, H. Van Keer, and M. Valcke, "Variations in socially shared metacognitive regulation and their relation with university students' performance," Metacognition Learn., vol. 15, no. 2, pp. 233-259, 2020, doi: 10.1007/s11409-020-09229-5.

https://doi.org/10.1007/s11409-020-09229-5

Z. Zulfiani, Y. Herlanti, and R. Yunistika, "Investigating Metacognitive Attitude of High School Biology Teachers," J. Penelit. dan Pembelajaran IPA, vol. 6, no. 1, pp. 1-12, 2020, doi: 10.30870/jppi.v6i1.6240.

https://doi.org/10.30870/jppi.v6i1.6240