Development of an Automated Descriptive Text-based Scoring System

Abstract

Computers and electronic technology today offer a very large number of ways to enrich educational assessment both in classroom and in large scale testing situations. Presently, in a large scale testing situation, scores are awarded manually. However, this system is characterized by inconsistency owing to emotional and cognitive human attributes. These can invariably damper students’ morals. Thus, a text-based scoring system based on computer technology is proposed in order to alleviate the limitations of the manual system in a large-scale testing situation. In this work, an automated descriptive text-based scoring system (ADTSS) is developed in the science and technology area. The ADTSS architecture consists three modules: the domain knowledge, text reviewer and scoring engine modules. The domain knowledge contains set of keywords that relate to terms in words, sentences that describe topic in question in the descriptive text-based system. The text reviewer appraises students’ responses, trim and format as well as maps students’ Identity to their corresponding expected responses Identity in the knowledge base. The scoring engine is divided into two components viz: the marker class and marks obtainable. The mark obtainable by student is based on Multivariate Bernoulli model. The proposed ADTSS was evaluated using the responses of 50 students in software engineering examination in Federal University of Technology Akure (FUTA). The results obtained shows 73.7% accuracy of the proposed system using mean divergence metric. The results shows that the proposed system can be used for text-based scoring because the comparative analysis between the proposed the manual scoring shows a little divergence and the problem examiner’s bias is removed.