Validity and Reliability of Science Literacy Assessment Instruments for Measuring Science Competencies in the Context of PISA 2025 using the Rasch Model
DOI: https://doi.org/10.26618/n0sjn231
Science Literacy, Science Competencies, PISA 2025, Rasch Model
Abstract
This study aims to analyse the validity and reliability of a science literacy assessment instrument for measuring students’ science competencies in the context of PISA 2025 using the Rasch Model. Research was conducted using a development design based on the 4-D model, which consists of the define, design, develop and disseminate stages. The developed instrument comprised 10 essay questions integrating science and environmental science indicators aligned with the PISA 2025 framework. Content validity analysis yielded high Aiken’s V coefficients, with scores of 0.84 for construction, 0.89 for relevance, and 0.90 for clarity, indicating strong expert agreement. Empirical testing was conducted on 50 respondents, and data were analysed using the Rasch Model. The results showed that 8 out of 10 items met the criteria for Outfit MNSQ, ZSTD, and Pt Measure Corr. The reliability value exceeded 0.70, confirming that the instrument fulfilled the person–item reliability standards. Item difficulty levels were distributed from very difficult to very easy, showing good item spread. Person measure and Wright map analyses indicated that most students demonstrated moderate to high science literacy abilities, with logit values ranging from –2 to +5. The average student ability was slightly higher than the average item difficulty, suggesting that the test was relatively easy for the sample group. Therefore, further development is recommended by adding more challenging items to enhance the instrument’s discriminating power and improve its accuracy in measuring science literacy across the full range of student abilities.
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Mentari Darma Putri, Dini Fitria, Nurlaini, Nur Jannah Berutu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright:
Authors who publish with this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licence:
Authors are free to:
1. Share: Copy and redistribute the material in any medium or format
2. Adapt: Remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as the authors follow the license terms, which include the following:
1. Attribution: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
2. ShareAlike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
3. No additional restrictions: You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Jurnal Pendidikan Fisika is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
