Rasch evaluation of matched positively and negatively worded CTS items among high school students
DOI:
https://doi.org/10.59672/ijed.v7i2.7023Keywords:
Computational Thinking Scale, Modified, Negatively worded, Positively worded, Rasch modelAbstract
Instruments composed of items worded in the same direction may be vulnerable to response bias. This study evaluated a modified Computational Thinking Scale (CTS) comprising 19 matched pairs of positively and negatively worded items. An instrument-development design adapted from ADDIE was used. The study population comprised science-track senior high school students in the Surakarta Residency. The sample included 393 Grade XI students from three purposively selected schools. Data were collected using a 38-item, five-category self-report questionnaire. Five experts assessed content validity, yielding coefficients of 0.80–0.95 and a mean of 0.89. Responses were analyzed using the Rasch model in Winsteps 5.7.3.0. Cronbach’s alpha and person reliability were 0.66, with person separation of 1.40, whereas item reliability was 1.00 and item separation was 14.67. Response categories showed ordered observed averages and Andrich thresholds, with outfit MNSQ values of 0.91–1.07. All items met the primary outfit MNSQ criterion, ranging from 0.82 to 1.21. Three algorithmic-thinking items showed statistically significant but small gender DIF contrasts of 0.30–0.35 logits. The hypothesis was partially supported because the instrument showed acceptable item-level functioning but limited person-level discrimination. Future studies should examine wording effects, matched-pair equivalence, dimensionality, and local dependence in broader, more diverse samples.
Downloads
References
Aho, A. V. (2012). Computation and computational thinking. The Computer Journal, 55(7), 832–835. https://doi.org/10.1093/comjnl/bxs074
Angeli, C., & Giannakos, M. (2020). Computational thinking education: Issues and challenges. Computers in Human Behavior, 105, 106185. https://doi.org/10.1016/j.chb.2019.106185
Barnette, J. J. (2000). Effects of stem and likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361–370. https://doi.org/10.1177/00131640021970592
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: what is Involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905
Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model. Psychology Press. https://doi.org/10.4324/9781410614575
Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer Netherlands. https://doi.org/10.1007/978-94-007-6857-4
Branch, R. M. (2009). Instructional design: The ADDIE approach. Springer US. https://doi.org/10.1007/978-0-387-09506-6
Chan, S. W. (2020). Computational thinking activities in number patterns: A study in a Singapore secondary school. ICCE 2020 - 28th International Conference on Computers in Education, Proceedings, 1, 171–176.
Cheng, L. (2023). The effects of computational thinking integration in STEM on students’ learning performance in K-12 education: A Meta-analysis. Journal of Educational Computing Research, 61(2), 416–443. https://doi.org/10.1177/07356331221114183
Chongo, S., Osman, K., & Nayan, N. A. (2021). Impact of the plugged-in and unplugged chemistry computational thinking modules on achievement in chemistry. Eurasia Journal of Mathematics, Science and Technology Education, 17(4), em1953. https://doi.org/10.29333/ejmste/10789
CSTA, & ISTE. (2011). Operational definition of computational thinking for K–12 education. http://www.iste.org/docs/pdfs/Operational-Definition-of-Computational-Thinking.pdf
Denning, P. J. (2009). The profession of IT beyond computational thinking. Communications of the ACM, 52(6), 28–30. https://doi.org/10.1145/1516046.1516054
Huda, N., & Rohaeti, E. (2024). Computational thinking skill level of senior high school students majoring in natural science. International Journal of Learning, Teaching and Educational Research, 23(1), 339–359. https://doi.org/10.26803/ijlter.23.1.17
Hurt, T., Greenwald, E., Allan, S., Cannady, M. A., Krakowski, A., Brodsky, L., Collins, M. A., Montgomery, R., & Dorph, R. (2023). The computational thinking for science (CT-S) framework: operationalizing CT-S for K–12 science education researchers and educators. In International Journal of STEM Education, 10(1). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1186/s40594-022-00391-7
Khine, M. S. (2020). Rasch measurement: Applications in quantitative educational research. Springer Singapore. https://doi.org/10.1007/978-981-15-1800-3
Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005
Li, Y., Schoenfeld, A. H., diSessa, A. A., Graesser, A. C., Benson, L. C., English, L. D., & Duschl, R. A. (2020). Computational thinking is more about thinking than computing. Journal for STEM Education Research, 3(1), 1–18. https://doi.org/10.1007/s41979-020-00030-2
Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.
Matlock, K. L., Turner, R. C., & Gitchel, W. D. (2018). A study of reverse-worded matched item pairs using the generalized partial credit and nominal response models. Educational and Psychological Measurement, 78(1), 103–127. https://doi.org/10.1177/0013164416670211
Matsumoto, P. S., & Cao, J. (2017). The development of computational thinking in a high school chemistry course. Journal of Chemical Education, 94(9), 1217–1224. https://doi.org/10.1021/acs.jchemed.6b00973
Ogegbo, A. A., & Ramnarain, U. (2022). A systematic review of computational thinking in science classrooms. Studies in Science Education, 58(2), 203–230. https://doi.org/10.1080/03057267.2021.1963580
Setiawati, F. A., Nurhayati, S. R., Amelia, R. N., & Darojat, A. A. (2022). Study on the threats of reverse-worded items to the psychometric properties of the marital quality scale. The Open Psychology Journal, 15(1), e187435012208150. https://doi.org/10.2174/18743501-v15-e2208150
Sondakh, D. E., Pungus, S. R., & Putra, E. Y. (2022). Indonesian undergraduate students’ perception of their computational thinking ability. CogITo Smart Journal, 8(1), 68–80. https://doi.org/10.31154/cogito.v8i1.387.68-80
Suárez-Álvarez, J., Pedrosa, I., Lozano, L., García-Cueto, E., Cuesta, M., & Muñiz, J. (2018). Using reversed items in Likert scales: A questionable practice. Psicothema, 2(30), 149–158. https://doi.org/10.7334/psicothema2018.33
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan (Application of Rasch modeling in educational assessment). Trim Komunikata.
Suryaningsih, N. M. A., Poerwati, C. E., Lestari, P. I., & Parwata, M. Y. (2025). Early childhood literacy skills: Implementation of the local genius-based STEM learning model. Indonesian Journal of Educational Development (IJED), 6(1), 160–172. https://doi.org/10.59672/ijed.v6i1.4627
Swain, S. D., Weathers, D., & Niedrich, R. W. (2008). Assessing three sources of misresponse to reversed likert items. Journal of Marketing Research, 45(1), 116–131. https://doi.org/10.1509/jmkr.45.1.116
Sweeney, C. T., Pillitteri, J. L., & Kozlowski, L. T. (1996). Measuring drug urges by questionnaire: Do not balance scales. Addictive Behaviors, 21(2), 199–204. https://doi.org/10.1016/0306-4603(95)00044-5
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798. https://doi.org/10.1016/j.compedu.2019.103798
van Sonderen, E., Sanderman, R., & Coyne, J. C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PLoS ONE, 8(7). https://doi.org/10.1371/journal.pone.0068967
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10.1007/s10956-015-9581-5
Widana, I. W., Sopandi, A. T., Suwardika, I. G. (2021). Development of an authentic assessment model in mathematics learning: A science, technology, engineering, and mathematics (STEM) approach. Indonesian Research Journal in Education, 5(1), 192-209. https://doi.org/10.22437/irje.v5i1.12992
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
Yudi Hartawan, I. G. N., Pujawan, I. G. N., & Wibawa, N. A. (2026). An exploration of teachers’ perspectives on computational thinking in mathematics learning. Indonesian Journal of Educational Development (IJED), 6(4), 1173–1188. https://doi.org/10.59672/ijed.v6i4.5594
Zeng, B., Wen, H., & Zhang, J. (2020). How does the valence of wording affect features of a scale? The method effects in the undergraduate learning burnout scale. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.585179
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Ahmad Fauzi, Hartono, Sunyoto Eko Nugroho, Arif Widiyatmoko
This work is licensed under a Creative Commons Attribution 4.0 International License.
This is an Open Access article distributed under the terms of Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.
