線上測驗是否有可能具備與紙筆測驗相同評分效力？

作者:楊亨利(國立政治大學資訊管理學系)、應鳴雄(親民技術學院資訊管理系)

卷期：50卷第2期
日期：2005年10月
頁碼：85-107
DOI：10.6300/JNTNU.2005.50(2).04

摘要：

由於線上測驗系統提供填充題及問答題等題性測驗時，會產生嚴重的測驗評分等化（Equation）問題，因此目前大部分的網路測驗系統的考試題形便以是非題、單選題及複選題為主。
　　為了瞭解線上測驗是否能具備與傳統測驗相同的評分效力，本研究在第一階段發展了一個具備模糊評分機制的線上測驗系統。該系統除了包括一般測驗系統所提供的是非、單選、複選等題型外，也能利用智慧評分機制來進行填充題評分。在第二階段，本研究將傳統紙筆測驗、一般型評分機制、本研究提出之智慧型評分機制進行評分效力比較。
　　研究結果顯示在包含填充題型的測驗中，不同的評分機制在測驗成績的評分結果上會有顯著差異，而智慧型評分機制運作初期雖然可以減少與紙筆評分間的差異，並改善一般型評分機制的評分效力，但仍無法在統計上或得具有相同評分效力的結果。但是智慧型評分機制在擴充詞彙語意後，其評分結果會比未擴充前更加提升。此外，「已擴充語意後的智慧型評分機制」與「紙筆評分」的評分結果並無顯著差異，此結果顯示出透過智慧型模糊評分機制，線上評分系統將有可能具有與紙筆評分相同的評分效力來處理具有填充題型的測驗評分工作。

關鍵詞：測驗等化、測驗評分等化、電腦線上測驗、填充題、模糊評分機制　

《詳全文》

Could On-line Testing have the Same Effects on Scoring as Paper-and-Pencil Testing?

Author: Heng-Li Yang(Dept. of MIS, NCCU), Ming-Hsiung Ying(Dept. of MIS, Chinmin Institute of Technology)

Vol.&No.：Vol. 50, No.2
Date：October 2005
Pages：85-107
DOI：10.6300/JNTNU.2005.50(2).04

Abstract：

With the rapid development of the Internet, computer-based or online testing has become an important issue in information education. Currently, most on-line tests only have selection-type items (single or multiple choice). Though some tests provide short-answer （completion-type） items, they can only recognize answers as being either “all correct” or “all wrong” given the computer’s simple binary-pattern matching system. Thus, in order to achieve the same precision of evaluation as in traditional paper-and-pencil testing, the first stage of this research project has adopted the concepts of fuzzy theory, the thesaurus, the set, and artificial intelligence to develop a “fuzzy scoring” mechanism. The proposed on-line testing system has true-false, multiple choice, and completion-type items. The latter will be graded by means of the naturally“fuzzy judgment” of human teachers.
The main purpose of this research is to evaluate the degree of equivalence of paper-and-pencil testing and on-line testing based on the fuzzy-scoring mechanism. The results demonstrate that different scoring mechanisms have a significant effect on test scores. At the beginning, though our fuzzy on-line testing system is significantly better than the usual on-line testing system, it could not achieve the same effects as paper-and-pencil testing. After we expanded our semantic vocabulary based on feedback, however, our fuzzy scoring mechanism is now equivalent, in terms of effects on scoring, to traditional paper-and-pencil testing.

Keywords：Computer On-Line Testing, Completion-Type Items, Fuzzy Scoring Mechanism, E-Learning, Test Score Equation