With online courses, how can you assess academic achievements accurately and reliably? In other words, how can you distinguish the online students who know their stuff from those who are guessing? Researchers from the University of Leuven and the Higher School of Economics improved a model that assesses academic achievements. The National Research University Higher School of Economics (HSE) is in Moscow, Russia, and the University of Leuven is in Flanders, Belgium.
The researchers wrote about their work and results in the journal Heliyon (citation below).
Factors making it hard to assess academic achievements
Many systemic factors make it hard for those who develop online courses to assess the proficiency of students accurately. Below are three factors:
- The average test has ten to fifteen questions, which are too few to accurately gauge how much students know.
- Multiple-choice questions lead to guessing, which subsequently distorts the results.
- The use of proportion of correct answers as a measure of proficiency makes it difficult to compare students when the test is updated even slightly.
Expanding the classic Rasch model
In this study, the researchers managed to solve these problems. They expanded the classic Rasch model by adding parameters.
Dmitry Abbakumov, Head of eLearning at the HSE Centre for Psychometrics, said:
“First, our expanded approach includes the effect of multiple attempts, making it possible to distinguish between students who guess and those who know the answers.”
“Second, because the knowledge metrics obtained with this expanded approach are expressed on a single scale, they can be compared even when the test questions are changed significantly.”
“And finally, we calculate metrics based not only on test results, but also by taking into account the student’s experience – their activity when watching videos and performance in hands-on sessions – providing a more comprehensive understanding of the student’s competence.”
‘E-learning‘ stands for ‘electronic learning,’ which means learning online, i.e., doing an online course.
The authors believe that online educators could use their approach to assess students’ academic achievements more accurately. The metrics could also be built into the navigation and recommendation solutions of online courses.
Citation
“Measuring student’s proficiency in MOOCs: multiple attempts extensions for the Rasch model,” Dmitry Abbakumov, Piet Desmet, and Wim Van den Noortgate. Heliyon 4 (2018) e01003. DOI: https://doi.org/10.1016/j.heliyon.2018.e01003.