2024-03-20, Jeitziner, Loris Tizian, Paneth, Lisa, Rack, Oliver, Zahn, Carmen, Wulff, Dirk

Collaborative group engagement is a key factor of success in learning groups. This work explores the development of an innovative natural language processing method for predicting collaborative group engagement. To this end, we identified linguistic markers based on an established observation-based scheme for rating collaborative group engagement, such as, semantic similarity to task instructions, verbal mimicry, sentiment, and use of jargon. We evaluated the predictive power of the linguistic markers on the data of an observational study in which 38 learning groups were instructed to perform a collaborative learning task. Overall, the data consisted of 2588 expert ratings on collaborative group engagement. We relied on machine learning to the predict collaborative group engagement ratings using informative subsets of linguistic markers. The results showed above-baseline predictive accuracy for all four dimensions of collaborative group engagement. Moreover, the analysis of feature importance points to quantity of utterances, responsiveness and uniformity of participation as the most important markers for collaborative group engagement. By harnessing natural language processing, this work extends traditional qualitative analysis and delivers nuanced quantitative metrics suitable for capturing the complexity and dynamics of contemporary Computer Supported Collaborative Learning (CSCL) environments. Thereby, it contributes to the evolving landscape of CSCL research and demonstrates the potential of novel analytic techniques to support and enrich qualitative analysis in multiple domains.

2023, Jeitziner, Loris Tizian, Paneth, Lisa, Rack, Oliver, Zahn, Carmen, Wulff, Dirk U., Damşa, Crina, Borge, Marcela, Koh, Elizabeth, Worsley, Marcelo

This study takes a NLP approach to measuring social engagement in CSCL-learning groups. Specifically, we develop linguistic markers to capture aspects of social engagement, namely sentiment, responsiveness and uniformity of participation and compare them to human ratings of social engagement. We observed small to moderate links between NLP-markers and human ratings that varied in size and direction across the different groups. We discuss measurement and prediction of social collaborative group engagement using natural language processing.