2024-09-27, Jeitziner, Loris Tizian, Paneth, Lisa, Rack, Oliver, Dirk U. Wulff, Zahn, Carmen

We investigate the Quality of Collaborative Group Engagement (QCGE) in Computer Supported Collaborative Learning (CSCL) employing a multimethod approach. Analyzing 38 triad groups the study combines advanced methods such as video analysis (verbal and nonverbal behavior self assessment, trained observer ratings, and natural language processing (NLP )). The results produced key insights into QCGE . First, t he observer rating s and self assessments exhibited limited variance and considerable skewness in most QCGE dimensions , significantly limiting their usefulness. Second, no nverbal behavior s and linguistic markers extracted using NLP showed small to moderate correlations with QCGE ratings, suggesting opportunities for measuring QCGE in an automatized fashion . Our study emphasizes the importance of multimethod approaches for understanding QCGE and highlights a potential to refine these methodologies using artificial intelligence to increase the accuracy and reliability of QCGE assessment.

2024-03-20, Jeitziner, Loris Tizian, Paneth, Lisa, Rack, Oliver, Zahn, Carmen, Wulff, Dirk

Collaborative group engagement is a key factor of success in learning groups. This work explores the development of an innovative natural language processing method for predicting collaborative group engagement. To this end, we identified linguistic markers based on an established observation-based scheme for rating collaborative group engagement, such as, semantic similarity to task instructions, verbal mimicry, sentiment, and use of jargon. We evaluated the predictive power of the linguistic markers on the data of an observational study in which 38 learning groups were instructed to perform a collaborative learning task. Overall, the data consisted of 2588 expert ratings on collaborative group engagement. We relied on machine learning to the predict collaborative group engagement ratings using informative subsets of linguistic markers. The results showed above-baseline predictive accuracy for all four dimensions of collaborative group engagement. Moreover, the analysis of feature importance points to quantity of utterances, responsiveness and uniformity of participation as the most important markers for collaborative group engagement. By harnessing natural language processing, this work extends traditional qualitative analysis and delivers nuanced quantitative metrics suitable for capturing the complexity and dynamics of contemporary Computer Supported Collaborative Learning (CSCL) environments. Thereby, it contributes to the evolving landscape of CSCL research and demonstrates the potential of novel analytic techniques to support and enrich qualitative analysis in multiple domains.