Abstract—High-quality discussions are of great significance to promote learning and achieve teaching goals. In this study, content analysis is used to perform semantic analysis on the content of the discussion posted by learners on online learning platform from the perspective of semantic similarity. The semantic similarity of different topics under the course is weighted to obtain the online discussion quality score of the learner. Then the evaluation results from experts discussed with the learner are divided into four levels. The Kappa consistency test and the McNemar paired chi-square test are performed. The experimental results show that there is consistency between the two evaluation methods, and the accuracy rates of the two ranked lower rankings are better. Therefore, the automated evaluation of the online discussion quality of learners based on semantics will play an indispensable role in the selection and sticking of comments on online learning platforms.