Comparative analysis of generative AI models in educational exercise performance
Loading...
Authors
Author (Corporation)
Publication date
2024
Typ of student thesis
Course of study
Collections
Type
04B - Conference paper
Editor (Corporation)
Supervisor
Parent work
EDULEARN24 Proceedings
Special issue
DOI of the original publication
Link
Series
Series number
Volume
Issue / Number
Pages / Duration
5181-5190
Patent number
Publisher / Publishing institution
International Academy of Technology, Education and Development (IATED)
Place of publication / Event location
Palma de Mallorca
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
The integration of artificial intelligence (AI) in educational settings presents great opportunities for enhancing learning experiences. Within the context of a Swiss University of applied Sciences employing a flipped classroom methodology and in class exercises, a noticeable shift in student behaviour has been observed. The quality of using generative AI for solving case study based exercises in Business Process and Project Management was analysed using OpenAI’s ChatGPT Model 3.5. However, it is unknown how the different models are performing compared to each other when solving exercises in business education. This paper aims to extend the discourse of solving class exercises by conducting a comparative analysis of various generative AI models in the context of educational exercises within the flipped classroom setting, particular in Business Process and Project Management. The study systematically assesses the performance of different AI models, such as GPT-4, BERT, BART, T5, LLM API, etc, in answering selected exercises derived from real-world business scenarios.
This study analyzes the accuracy, relevance, completeness, and contextual understanding exhibited by each AI model in response to a series of exercises. These exercises are designed to mimic real-world business challenges in Business Process and Project Management, thereby providing a meaningful evaluation of each model's use in an educational context. The study further delves into the nuances of prompt construction, examining how variations in prompt design influence the performance of AI models, thereby shedding light on the critical role of effective communication in leveraging AI for educational purposes.
The findings of this research provide educators, researchers, and practitioners with a comprehensive understanding of the comparative strengths and weaknesses of various generative AI models in the context of business education, particularly in Business Process and Project Management. By highlighting key differences in selecting and deploying AI tools for educational exercises, the paper aims to contribute insights into the optimization of AI-assisted learning environments.
Keywords
Subject (DDC)
Event
Edulearn24. 16th annual International Conference on Education and New Learning Technologies
Exhibition start date
Exhibition end date
Conference start date
Conference end date
Date of the last check
ISBN
978-84-09-62938-1
ISSN
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
Peer review of the complete publication
Open access category
Closed
License
Citation
Meyer, L., & Dannecker, A. (2024). Comparative analysis of generative AI models in educational exercise performance. In L. Gómez Chova, C. González Martínez, & J. Lees (Eds.), EDULEARN24 Proceedings (pp. 5181–5190). International Academy of Technology, Education and Development (IATED). https://doi.org/10.21125/edulearn.2024.1273