Automated Tri Dharma Quality Assessment for Academic Accreditation Using Qwen, Mistral, and DeepSeek

Abstract

Internal Quality Audit (AMI) is critical for Indonesian higher-education accreditation, yet study 
programs still score 60+ indicators manually by cross-referencing a 200-page Self-Evaluation Report (LED) and a 
50-sheet Study Program Performance Report (LKPS), a process that takes 3–5 days per program and varies 
between assessors. This study develops an automated quality-assessment agent that ingests both documents 
and benchmarks three open-weight Large Language Models, Qwen 3.5 35B, DeepSeek-R1 32B, and Mistral-Small 
3.2 24B, on 19 LAM-Teknik indicators sampled across all nine accreditation criteria (9 qualitative, 6 quantitative, 
4 composite). The reference scores are taken from the certified human assessor's official record produced during 
the program's 2024 post-audit cycle. The results show that Qwen 3.5 attains the lowest MAE (0.605) and RMSE 
(0.769) with 100% within ±1 accuracy and 32.5 s per indicator; DeepSeek-R1 is the most cautious but the slowest 
(MAE 1.395; 79.9 s) and collapses on quantitative items (MAE 2.17); Mistral is the fastest (13.6 s).