Estimating solving probability of test items using psychometric models and physiological signals

OData support
Dr. Forstner Bertalan
Department of Automation and Applied Informatics

In education, it is important for students to report the knowledge they acquired, which is mostly done by measuring their learned abilities. For the measurement, written tests containing multiple questions are often used. When assembling these tests, it is nec-essary to know the parameters of the tasks we use. Psychometric models, including item response theory (IRT) models use the parameters of the tasks and the answers of the measured individual to estimate their ability level, utilizing methods of mathematical statis-tics. The ability level is then used to estimate the solving probability of a task with fixed parameters.

Classic IRT models, however, take only the quality of the individual answers into account when making the estimate, even though recent scientific research found that the influence of the mental state of a person is an equally important factor in their performance on a test. Nowadays, the measurement of physiological signals needed to determine men-tal state is feasible using cost-efficiently and simply obtainable physiological sensors available in retail, which can be used to assess the mental condition of a person during a test. Nevertheless, according to the common opinion of many researchers, the extension of psychometric models using mental state is still an open question in need of interdisci-plinary scientific examination.

In my thesis, I describe the basics of traditional IRT models and the possibilities for measuring psychological signals using EEG and ECG sensors. I present the extended IRT model I have developed, which takes the momentary mental state of the measured subject into account as a time-dependent parameter. In order to establish and validate the extended, closed model, I developed a simulation environment which is also presented in my paper. To apply the model in practice, I conducted measurements using an educational game based on my model and a partly self-developed, modern, extensible measurement framework for mobile devices. In my thesis, I present the details of the architecture and the design and implementation process of the framework made with software and hard-ware integration. I also thoroughly describe and evaluate the results of the measurements I made using the framework.


Please sign in to download the files of this thesis.