Predicting engagement in computer-supported collaborative learning groups using natural language processing

Abstract

Collaborative group engagement is a key factor of success in learning groups. This work explores the development of an innovative natural language processing method for predicting collaborative group engagement. To this end, we identified linguistic markers based on an established observation-based scheme for rating collaborative group engagement, such as, semantic similarity to task instructions, verbal mimicry, sentiment, and use of jargon. We evaluated the predictive power of the linguistic markers on the data of an observational study in which 38 learning groups were instructed to perform a collaborative learning task. Overall, the data consisted of 2588 expert ratings on collaborative group engagement. We relied on machine learning to the predict collaborative group engagement ratings using informative subsets of linguistic markers. The results showed above-baseline predictive accuracy for all four dimensions of collaborative group engagement. Moreover, the analysis of feature importance points to quantity of utterances, responsiveness and uniformity of participation as the most important markers for collaborative group engagement. By harnessing natural language processing, this work extends traditional qualitative analysis and delivers nuanced quantitative metrics suitable for capturing the complexity and dynamics of contemporary Computer Supported Collaborative Learning (CSCL) environments. Thereby, it contributes to the evolving landscape of CSCL research and demonstrates the potential of novel analytic techniques to support and enrich qualitative analysis in multiple domains.