Validity and Reliability of Science Literacy Assessment Instruments for Measuring Science Competencies in the Context of PISA 2025 using the Rasch Model
DOI: https://doi.org/10.26618/n0sjn231
assessment instrument, science literacy, scientific reasoning, PISA 2025, rasch model
Abstract
Science literacy is a key competency for preparing students to reason scientifically, interpret evidence, and make informed decisions in real-world contexts. Yet Indonesian students’ performance in international assessments remains below the OECD average, underscoring the urgent need for assessment tools aligned with global standards. This study aimed to develop and validate a PISA 2025–oriented science literacy assessment instrument to support the measurement of senior high school students’ competencies in scientifically explaining phenomena; constructing and evaluating investigation designs; critically interpreting data and evidence; and researching, evaluating, and using scientific information for decision-making and action. Using a research and development approach with the 4-D model, ten context-rich essay items and an analytic scoring rubric (levels 1–4) were produced. Content validity was examined by three experts using Aiken’s V, and empirical validation was conducted with 50 Year 11 students from two senior high schools in Langsa City using the Rasch Model (Winstep). The instrument demonstrated strong content validity (Aiken’s V averages: construction = 0.84, relevance = 0.89, clarity = 0.90). Rasch results showed that 8 of 10 items met fit criteria, while two items displayed misfit on selected indices but remained positively correlated with the measured construct. Reliability was high (Cronbach’s alpha = 0.85; person reliability = 0.84; item reliability = 0.97), and item difficulty spanned a broad range, enabling discrimination across students’ ability levels. The novelty of this work lies in producing a Rasch-validated, constructed-response instrument explicitly aligned with the revised PISA 2025 science competency structure, including the new competency on evaluating and using scientific information in digital contexts. In conclusion, the instrument is suitable for limited classroom and research use. It provides a practical contribution to physics education by strengthening evidence-based assessment of scientific reasoning and decision-making in physics-related and environmental contexts.
References
Adhari, D., Santiani, S., & Fatmawati, S. (2025). Validity and reliability analysis of an environmentally integrated science literacy test using Rasch model. EduFisika: Jurnal Pendidikan Fisika, 10(1), 38–53. https://doi.org/10.59052/edufisika.v10i1.42136
Afriana, J., Permanasari, A., & Fitriani, A. (2016). Penerapan project-based learning terintegrasi STEM untuk meningkatkan literasi sains siswa ditinjau dari gender. Jurnal Inovasi Pendidikan IPA, 2(2), 202–212. https://doi.org/10.21831/jipi.v2i2.8561
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Alti, R. P., Lufri, L., Helendra, H., & Yogica, R. (2021). Pengembangan instrumen asesmen berbasis literasi sains tentang materi keanekaragaman hayati kelas X. Journal for Lesson and Learning Studies, 4(1), 53–58. https://doi.org/10.23887/jlls.v4i1.34270
Arikunto, S. (2013). Prosedur penelitian: Suatu pendekatan praktik. Rineka Cipta.
Azis, A. A., Lutfi, & Ismail. (2018). Pengaruh project-based learning terintegrasi STEM terhadap literasi sains, kreativitas, dan hasil belajar peserta didik. Prosiding Seminar Nasional Biologi dan Pembelajarannya, 189–194. https://ojs.unm.ac.id/semnasbio/article/view/6984
Bashooir, K., & Supahar, S. (2018). Validitas dan reliabilitas instrumen asesmen kinerja literasi sains pelajaran fisika berbasis STEM. Jurnal Penelitian dan Evaluasi Pendidikan, 22(2), 219–230. https://doi.org/10.21831/pep.v22i2.19590
Boone, W. J., Yale, M. S., & Staver, J. R. (2014). Rasch analysis in the human sciences. https://doi.org/10.1007/978-94-007-6857-4
Darman, D. R., Suhandi, A., Kaniawati, I., Samsudin, A., & Wibowo, F. C. (2024). Development and validation of scientific inquiry literacy instrument (SILI) using Rasch measurement model. Education Sciences, 14(3), 1-28. https://doi.org/10.3390/educsci14030322
Fuadi, H., Robbia, A. Z., Jamaluddin, J., & Jufri, A. W. (2020). Analisis faktor penyebab rendahnya kemampuan literasi sains peserta didik. Jurnal Ilmiah Profesi Pendidikan, 5(2), 108–116. https://www.neliti.com/publications/431701/analisis-faktor-penyebab-rendahnya-kemampuan-literasi-sains-peserta-didik
García-Ceberino, J. M., Antúnez, A., Ibáñez, S. J., & Feu, S. (2020). Design and validation of the instrument for the measurement of learning and performance in football. International Journal of Environmental Research and Public Health, 17(13), 1-22. https://doi.org/10.3390/ijerph17134629
Helendra, H., & Sari, D. R. (2021). Pengembangan instrumen asesmen berbasis literasi sains tentang materi sistem ekskresi dan sistem pernapasan. Jurnal Ilmiah Pendidikan Profesi Guru, 4(1), 17–25. https://doi.org/10.23887/jippg.v4i1.29860
Hidayah, N., & Muhtarom, M. (2023). Validity and reliability test of teaching materials using Aiken’s V formula and SPSS 22. Schola, 1(2), 75–82. https://doi.org/10.26877/schola.v1i2.342
Hidayah, N., Rusilowati, A., & Masturi, M. (2019). Analisis profil kemampuan literasi sains siswa SMP/MTs di Kabupaten Pati. Phenomenon: Jurnal Pendidikan MIPA, 9(1), 36–47. https://doi.org/10.21580/phen.2019.9.1.3601
Kamil, F. F., Permanasari, A., & Riandi, R. (2021). Studi profil literasi sains siswa dan pembelajarannya di SMP Kota Banda Aceh. Jurnal IPA & Pembelajaran IPA, 5(4), 353–363. https://doi.org/10.24815/jipi.v5i4.23446
Linacre, J. M. (2011). A user’s guide to Winsteps®: Rasch-model computer programs. Winsteps.com. www.winsteps.com/manuals.htm
Martinah, A. A., Mubarok, V., Miarsyah, M., & Ristanto, R. H. (2022). Pengembangan instrumen tes literasi sains berbasis kontekstual pada materi pencemaran lingkungan. Bioedusiana: Jurnal Pendidikan Biologi, 6(2), 192–218. https://doi.org/10.37058/bioed.v6i2.3251
Novitasari, L., & Handhika, J. (2018). Profil analisis kebutuhan pengembangan instrumen kognitif literasi sains untuk siswa SMA. Seminar Nasional Quantum, 25, 517–523. https://www.semanticscholar.org/paper/Profil-analisis-kebutuhan-pengembangan-instrumen-Novitasari-Handhika/b1e86830b70d9e27aa18f2fb2d6670b094e42721
OECD. (2019). PISA 2018 assessment and analytical framework. OECD Publishing. https://doi.org/10.1787/b25efab8-en
OECD. (2023). PISA 2025 science framework (Second draft). https://pisa-framework.oecd.org/science-2025/assets/docs/PISA_2025_Science_Framework.pdf
Oktaviani, C., Seprianto, S., & Putri, M. D. (2023). Creative thinking-oriented students’ scientific literacy skills: Preliminary study. Jurnal Penelitian Pendidikan IPA, 9(10), 8245–8250. https://doi.org/10.29303/jppipa.v9i10.5520
Permanasari, A. (2016). STEM education: Inovasi dalam pembelajaran sains. Prosiding Seminar Nasional Pendidikan Sains, 23–34. https://media.neliti.com/media/publications/173124-ID-stem-education-inovasi-dalam-pembelajara.pdf
Putri, M. D. (2021). Identifikasi kemampuan literasi sains siswa di SMP Negeri 2 Pematang Tiga Bengkulu Tengah. Gravitasi: Jurnal Pendidikan Fisika dan Sains, 4(1), 9–17. https://doi.org/10.33059/gravitasi.jpfs.v4i01.3610
Rostikawati, D. A., & Permanasari, A. (2016). Rekonstruksi bahan ajar dengan konteks socio-scientific issues pada materi zat aditif makanan untuk meningkatkan literasi sains siswa. Jurnal Inovasi Pendidikan IPA, 2(2), 156–164. https://doi.org/10.21831/jipi.v2i2.8814
Rusyati, R., Permanasari, A., & Ardianto, D. (2019). Rekonstruksi bahan ajar berbasis STEM untuk meningkatkan literasi sains dan teknologi siswa pada konsep kemagnetan. Journal of Science Education and Practice, 2(2), 10–22. https://doi.org/10.33751/jsep.v2i2.1395
Saidah, E. N., & Malichatin, H. (2023). Pengembangan instrumen literasi sains berbasis Asesmen Kompetensi Minimum (AKM) untuk peserta didik kelas VII SMP/MTs. NCOINS: National Conference of Islamic Natural Science, 3, 240–255. https://www.collegesidekick.com/study-docs/28017374
Saputra, I. G., P., E., Sukariasih, L., Nursalam, L. O., & Desa, S. S. (2022). The effect of scientific literacy approach with discovery learning model toward physics concepts understanding. Jurnal Pendidikan Fisika, 10(2), 144–153. https://doi.org/10.26618/jpf.v10i2.7769
Sari, D. N. A., Rusilowati, A., & Nuswowati, M. (2017). Pengaruh pembelajaran berbasis proyek terhadap kemampuan literasi sains siswa. PSEJ (Pancasakti Science Education Journal), 2(2), 114. https://scienceedujournal.org/index.php/PSEJ/article/view/85
Serdianus, S., & Saputra, T. (2023). Peran artificial intelligence ChatGPT dalam perencanaan pembelajaran di era Revolusi Industri 4.0. Masokan: Ilmu Sosial dan Pendidikan, 3(1), 1–18. https://doi.org/10.34307/misp.v3i1.100
Stacey, K. (2021). Assessing mathematical and scientific literacy: Reflections from PISA. Educational Studies in Mathematics, 108(1–2), 5–24. https://link.springer.com/book/10.1007/978-3-319-10121-7
Sugiyono. (2017). Metode penelitian kuantitatif, kualitatif, dan R&D. Alfabeta.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata Publishing House.
Sutrisna, N. (2021). Analisis kemampuan literasi sains peserta didik SMA di Kota Sungai Penuh. Jurnal Inovasi Penelitian, 1(12), 2683–2694. https://www.neliti.com/publications/469443/analisis-kemampuan-literasi-sains-peserta-didik-sma-di-kota-sungai-penuh
Tabacholly, F. A., Susongko, P., & Nafiati, D. A. (2025). Construct validation of science literacy instrument with Rasch modeling on students of Grade VIII junior high school. Journal of English Language and Education, 10(1), 378–388. https://doi.org/10.31004/jele.v10i1.652
Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children. Leadership Training Institute/Special Education, University of Minnesota. https://www.semanticscholar.org/paper/InstructionalDevelopment-for-Training-Teachers-ofThiagarajan/44a718a0c8e219b37aabb4c36117dcd695c895d0
Wirama, T. G. P. (2022). Asesmen literasi sains tema kimia hijau pada siswa. Indonesian Journal of Educational Development, 3(1), 1–15. https://doi.org/10.5281/zenodo.6557263
Yusmar, F., & Fadilah, R. E. (2023). Analisis rendahnya literasi sains peserta didik Indonesia: Hasil PISA dan faktor penyebab. LENSA (Lentera Sains): Jurnal Pendidikan IPA, 13(1), 11–19. https://doi.org/10.24929/lensa.v13i1.283
Zhang, L., Liu, X., & Feng, H. (2023). Development and validation of an instrument for assessing scientific literacy from junior to senior high school. Disciplinary and Interdisciplinary Science Education Research, 5(21), 1-15. https://doi.org/10.1186/s43031-023-00093-2
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Mentari Darma Putri, Dini Fitria, Nurlaini, Nur Jannah Berutu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright:
Authors who publish with this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licence:
Authors are free to:
1. Share: Copy and redistribute the material in any medium or format
2. Adapt: Remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as the authors follow the license terms, which include the following:
1. Attribution: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
2. ShareAlike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
3. No additional restrictions: You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Jurnal Pendidikan Fisika is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
