Data-Driven Assessment Engineering Model Based on Log Automation and Predictive Analytics for the Transformation of Science Evaluation Systems

Abstract

The transformation of science evaluation systems toward intelligent assessment is a critical urgency in the realm of digital technology, utilizing digital exhaust as a massive source of behavioral data. However, conventional monolithic Learning Management Systems (LMS) lack the architectural capacity to process real-time time-series logs, functioning merely as static score repositories rather than proactive diagnostic tools. This study aims to develop and validate a Data-Driven Assessment Engineering (DDAE) model to address this infrastructural limitation. Utilizing the Design Science Research Methodology (DSRM), the computational architecture was engineered using a distributed microservices approach integrating Apache Kafka, Polyglot Persistence (PostgreSQL, MongoDB, InfluxDB), and Machine Learning algorithms (XGBoost and DBSCAN). The prototype was deployed for junior high school science evaluations involving 312 eighth-grade students across four public schools (936 examination sessions; 487,592 raw log events), and user acceptance was quantitatively assessed using the Technology Acceptance Model (TAM) via Multiple Linear Regression. Load testing demonstrated the DDAE architecture's stability, maintaining an optimal latency of 42.1 ms under a peak load of 1,000 concurrent users. Furthermore, the XGBoost predictive model, optimized via grid-search hyperparameter tuning with stratified 5-fold cross-validation, achieved an outstanding recall of 95.1% in classifying learning mastery, while DBSCAN precisely isolated rapid-guessing anomalies as spatial noise. Consequently, the automated processing of exam logs successfully converts passive raw data into actionable intelligence. This shifts the science evaluation paradigm into a proactive Early Warning System (EWS), heavily driven by its perceived usefulness among educational stakeholders.