Quality and students' perception of feedback generated by GPT-4 on a complex group task

Abstract

AI in education is a significant and widely debated subject, present at various educational levels. Students utilise AI for easy referencing of concepts to complete university assignments. It is also integrated into lecture content, facilitating discussions on effective prompting within specific contexts. Moreover, AI can act as a peer, aiding in the interactive development of assignment solutions. Instructors employ AI for task development and student outcome evaluation.

This study explores how generative AIs can support or even supplant the feedback process for complex group tasks within the classroom. It focuses on a project management group task centred around developing project objectives according to IPMA standards, a task aimed at achieving multiple learning objectives. Students must comprehend an interview with a client, grasp project objective theory, categorise objectives, formulate them SMART, and ideally align them with company goals described in a case study. After working on the task during class time, groups receive guidance from instructors who provide coaching and support. Following completion, solutions are uploaded for individual feedback from instructors.

Using the GPT-4 AI model, we trained it with past semester feedback and provided the theoretical context of the group task. With these inputs, we analysed, evaluated, and provided feedback generated by GPT-4 to groups. Subsequently, we surveyed groups to gauge the feedback's usefulness and compared this with lecturer assessments. This task forms part of a module for undergraduate business students focusing on business process and project management, developed in line with evidence-based university teaching criteria. The module is taught uniformly across the university's three locations, comprising seven classes in Spring 2024 (5 in German, 2 in English), utilising the same translated case study, exercises, and evaluation exam.

The module's methodology combines case study teaching and the flipped classroom concept. A case study, based on a start-up company's real-life experiences, was developed. Over 12 weeks, students acquainted themselves with business process and project management theories, engaging in group exercises during class sessions.

This study demonstrates how GPT-4-generated feedback can be utilised for complex group tasks and provides insights into its requirements in terms of theory, training data, and prompt structure.