A NEEDS ANALYSIS FOR DESIGNING A MODULE-BASED AUTOMATED WRITING EVALUATION (AWE) TOOL IN INDONESIAN EFL ACADEMIC WRITING
DOI: https://doi.org/10.26618/xv300v61
Academic Writing, Automated Writing Evaluation, AWE, Ethics
Abstract
Marhamah1, Hendri Yawan2, Litha Nesidekawati Dakka3
1,2,3 Universitas Sembilanbelas November Kolaka, Indonesia
Marhamahudri@gmail.com
Received: May 30, 2025 Revised: June 27, 2025 Accepted: August 16, 2025
ABSTRACT
The growing use of AI Automated Writing Evaluation (AWE) tools in higher education has raised concerns about students’ reliance, lack of critical engagement, and ethical uncertainties. However, limited research has examined how AI-supported writing is experienced in Indonesian EFL higher education contexts, particularly regarding its pedagogical and ethical implications in academic writing classrooms. This study, as part of a broader Design-Based Research (DBR) project, focused on a needs analysis through a qualitative case study to explore students’ and lecture’s perceptions of AI-supported academic writing in an Indonesian EFL context and to inform the design of a digital writing module. Data were collected through interviews with one instructor, focus group discussions with 15 students, and the analysis of student writing drafts. Reflexive thematic analysis was applied to capture patterned meanings and interpretive insights. Findings showed that while AI tools improved grammatical accuracy and text coherence, they often encouraged limited argumentation, homogenized structures, and confusion about academic integrity. These results suggest the importance of embedding scaffolding for reflection, feedback literacy, and digital ethics into instructional design.
References
Aini, N., Kurniarahman, I., Widiati, U., Cahyono, B. Y., & Basthomi, Y. (2024). Indonesian university students' perspectives on integrating AIEd into English language learning. Issues in Educational Research, 34(3), 803–824. https://www.proquest.com/docview/3104449889
Anisah, L., Yawan, H., & Marhamah, M. (2024). Artificial intelligence enhanced learning management system: Supporting Merdeka Belajar–Kampus Merdeka (MBKM) at a state university in Indonesia. International Journal of Education and Social Studies Management, 4(3), 917–931. https://doi.org/10.52121/ijessm.v4i3.414
Aspers, P., & Corte, U. (2019). What is qualitative in qualitative research. Qualitative Sociology, 42(2), 139–160. https://doi.org/10.1007/s11133-019-9413-7
Barab, S., & Squire, K. (2021). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 30(3), 1–15.
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Education and Information Technologies, 28(6), 7195–7211. https://doi.org/10.1007/s10639-023-11978-2
Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry and research design: Choosing among five approaches (4th ed.). SAGE.
Darwin, D., Rusdin, D., Mukminatien, N., Suryati, N., & Laksmi, E. D. (2024). Critical thinking in the AI era: An exploration of EFL student perception, benefits, and limitations. Cogent Education, 11(1), 2290342. https://doi.org/10.1080/2331186X.2023.2290342
Dwihadiah, D. L., Niyu, N., & Purba, H. (2024). Digital ethics model concerning the use of ChatGPT in Indonesian higher education. Information, Medium, Society: Journal of Public Studies, 23(1), 1–22. https://doi.org/10.18848/2691-1507/CGP/v23i01/1-22
Flick, U. (2022). The SAGE handbook of qualitative research design. SAGE.
Gozali, I., Wijaya, A. R. T., Lie, A., Cahyono, B. Y., & Suryati, N. (2024). ChatGPT as an automated writing evaluation tool: Feedback literacy development and AWE tools’ integration framework. JALT CALL Journal, 20(1), 1–20. https://doi.org/10.29140/jaltcall.v20n1.1200
Harrison, H., Birks, M., Franklin, R., & Mills, J. (2020). Case study research: Foundations and methodological orientations. Forum Qualitative Sozialforschung, 21(1).
Helmiatin, H., Hidayat, A., & Kahar, M. R. (2024). Investigating the adoption of AI in higher education: A study of public universities in Indonesia. Cogent Education, 11(1), 2380175. https://doi.org/10.1080/2331186X.2024.2380175
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. TESOL Quarterly, 57(4), 1105–1118. https://doi.org/10.1002/tesq.3318
Korstjens, I., & Moser, A. (2019). Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. European Journal of General Practice, 24(1), 120–124. https://doi.org/10.1080/13814788.2017.1375092
McKenney, S., & Reeves, T. C. (2019). Conducting educational design research (2nd ed.). Routledge.
Nguyen, H. T., Bui, T. T., & Vo, D. T. (2024). The impact of AI writing tools on academic integrity: Unveiling English-majored students' perceptions and practical solutions. International Journal of TESOL & Education, 4(1), 110–121. https://doi.org/10.54855/ijte.241110
Qin, C., Zhang, R., & Xiao, Y. (2022). A questionnaire-based validation of metacognitive strategies in writing and their predictive effects on the writing performance of English as foreign language student writers. Frontiers in Psychology, 13, 1071907. https://doi.org/10.3389/fpsyg.2022.1071907
Rafida, T., Suwandi, S., & Ananda, R. (2024). EFL students’ perception in Indonesia and Taiwan on using artificial intelligence to enhance writing skills. Jurnal Ilmiah Peuradeun, 12(3), 987–1016. https://doi.org/10.26811/peuradeun.v12i3.1520
Saifullah, S., Yawan, H., Syafitri, N., & Nurhaliza, S. (2024). Integrating AI chatbot into learning management system: Enhancing student engagement and learning outcomes. Dharmas Education Journal, 5(2), 1346–1359. https://doi.org/10.56667/dejournal.v5i2.1605
Selwyn, N. (2019). Should robots replace teachers? AI and the future of education. Polity Press.
Sharples, M. (2023). Towards social generative AI for education: Theory, practices and ethics. arXiv.
Smith, M. (2023). Artificial intelligence and the homogenization of student writing: Risks and responsibilities in higher education. Journal of Academic Ethics, 21(2), 211–229. https://doi.org/10.1007/s10805-022-09460-3
Stahl, B. C. (2025). Locating the ethics of ChatGPT—Ethical issues as affordances in AI ecosystems. Information, 16(2), 104. https://doi.org/10.3390/info16020104
Sun, Q., & Zhang, L. J. (2022). Understanding learners’ metacognitive experiences in learning to write in English as a foreign language: A structural equation modeling approach. Frontiers in Psychology, 13, 986301. https://doi.org/10.3389/fpsyg.2022.986301
Weldiningsih, I. (2024). Balancing AI and authenticity: EFL students’ experiences with ChatGPT in academic writing. Cogent Arts & Humanities, 11(1), 2392388. https://doi.org/10.1080/23311983.2024.2392388
Yang, Y., & Chen, X. (2023). Learning analytics of AI-supported academic writing in LMS: Patterns and challenges. Frontiers in Education, 8, 1126083. https://doi.org/10.3389/feduc.2023.1126083
Zheng, L., Bender, S., & Wang, Y. (2023). Design-based research on the development and implementation of an AI course integrating competition and education. Frontiers of Education in China, 18(3), 365–389. https://doi.org/10.3868/s110-008-023-0031-5
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 2026
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
In order to assure the highest standards for published articles, a peer review policy is applied. In pursue of the compliance with academic standards, all parties involved in the publishing process (the authors, the editors and the editorial board and the reviewers) agree to meet the responsibilities stated below in accordance to the Journal publication ethics and malpractice statement.
Duties of Authors:
- The author(s) warrant that the submitted article is an original work, which has not been previously published, and that they have obtained an agreement from any co-author(s) prior to the manuscript’s submission;
- The author(s) should not submit articles describing essentially the same research to more than one journal;
- The authors(s) make certain that the manuscript meets the terms of the Manuscript Submission Guideline regarding appropriate academic citation and that no copyright infringement occurs;
- The authors(s) should inform the editors about any conflict of interests and report any errors they subsequently, discover in their manuscript.
Duties of Editors and the Editorial Board:
- The editors, together with the editorial board, are responsible for deciding upon the publication or rejection of the submitted manuscripts based only on their originality, significance, and relevance to the domains of the journal;
- The editors evaluate the manuscripts compliance with academic criteria, the domains of the journal and the guidelines;
- The editors must at all times respect the confidentiality of any information pertaining to the submitted manuscripts;
- The editors assign the review of each manuscript to two reviewers chosen according to their domains of expertise. The editors must take into account any conflict of interest reported by the authors and the reviewers.
- The editors must ensure that the comments and recommendations of the reviewers are sent to the author(s) in due time and that the manuscripts are returned to the editors, who take the final decision to publish them or not.
Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
