Information-based methods for evaluating the semantics of automatically generated test items

Abstract

Multiple-choice questions are the popular type of test items that are used for testing the knowledge of health-science students innorth America and elsewhere. The motivation of this article is to present the recent advances in the automatic item generation(AIG) and to propose a novel unsupervised approach that extends the information-based Compositional Distributional SemanticModel (CDSM) to measure the semantic relatedness among the pool of automatically generated items. We have used operationalitem bank from the medical science domain for developing the CDSM and demonstrated our approach using the concepts fromAIG research. We illustrated our approach using eleven item models from the medical education domain, and discussed thepossible applications to advance the AIG research.