Could On-line Testing have the Same Effects on Scoring as Paper-and-Pencil Testing?

Author: Heng-Li Yang(Dept. of MIS, NCCU), Ming-Hsiung Ying(Dept. of MIS, Chinmin Institute of Technology)

Vol.&No.：Vol. 50, No.2
Date：October 2005
Pages：85-107
DOI：10.6300/JNTNU.2005.50(2).04

Abstract：

With the rapid development of the Internet, computer-based or online testing has become an important issue in information education. Currently, most on-line tests only have selection-type items (single or multiple choice). Though some tests provide short-answer （completion-type） items, they can only recognize answers as being either “all correct” or “all wrong” given the computer’s simple binary-pattern matching system. Thus, in order to achieve the same precision of evaluation as in traditional paper-and-pencil testing, the first stage of this research project has adopted the concepts of fuzzy theory, the thesaurus, the set, and artificial intelligence to develop a “fuzzy scoring” mechanism. The proposed on-line testing system has true-false, multiple choice, and completion-type items. The latter will be graded by means of the naturally“fuzzy judgment” of human teachers.
The main purpose of this research is to evaluate the degree of equivalence of paper-and-pencil testing and on-line testing based on the fuzzy-scoring mechanism. The results demonstrate that different scoring mechanisms have a significant effect on test scores. At the beginning, though our fuzzy on-line testing system is significantly better than the usual on-line testing system, it could not achieve the same effects as paper-and-pencil testing. After we expanded our semantic vocabulary based on feedback, however, our fuzzy scoring mechanism is now equivalent, in terms of effects on scoring, to traditional paper-and-pencil testing.

Keywords：Computer On-Line Testing, Completion-Type Items, Fuzzy Scoring Mechanism, E-Learning, Test Score Equation

《Full Text》