Home > Published Issues > 2022 > Volume 8, No. 2, June 2022 >

The Study on Automated Evaluation of Online Discussion Quality Based on Semantics

Wen-Yan Qin 1, Shuang Qi 2, Shu-Yue Zhou 1, and Chao-Jun Xu 1
1. School of Education Science, Nanjing Normal University, Nanjing, Jiangsu, China
2. Affiliated to Hangzhou No.14 High School, Hangzhou, Zhejiang, China

Abstract—High-quality discussions are of great significance to promote learning and achieve teaching goals. In this study, content analysis is used to perform semantic analysis on the content of the discussion posted by learners on online learning platform from the perspective of semantic similarity. The semantic similarity of different topics under the course is weighted to obtain the online discussion quality score of the learner. Then the evaluation results from experts discussed with the learner are divided into four levels. The Kappa consistency test and the McNemar paired chi-square test are performed. The experimental results show that there is consistency between the two evaluation methods, and the accuracy rates of the two ranked lower rankings are better. Therefore, the automated evaluation of the online discussion quality of learners based on semantics will play an indispensable role in the selection and sticking of comments on online learning platforms.
 
Index Terms—automated evaluation, discussion quality, semantic analysis, semantic similarity

Cite: Wen-Yan Qin, Shuang Qi, Shu-Yue Zhou, and Chao-Jun Xu, "The Study on Automated Evaluation of Online Discussion Quality Based on Semantics," International Journal of Learning and Teaching, Vol. 8, No. 2, pp. 80-85, June 2022. doi: 10.18178/ijlt.8.2.80-85

Copyright © 2022 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.