Towards a sustainable operating concept of LLM generated examination support in higher technical education
Loading...
Author (Corporation)
Publication date
2024
Typ of student thesis
Course of study
Collections
Type
04B - Conference paper
Editor (Corporation)
Supervisor
Parent work
17th annual International Conference of Education, Research and Innovation
Special issue
DOI of the original publication
Link
Series
ICERI Proceedings
Series number
Volume
Issue / Number
Pages / Duration
1064-1069
Patent number
Publisher / Publishing institution
IATED Academy
Place of publication / Event location
Valencia
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
At the University of Applied Science Northwestern Switzerland (FHNW) it can be observed that the usage of preferred learning material shifted in the past years from books and articles to videos, summaries, flashcards and test exams. In a specialization module of the programme BSc Business Information Technology (BIT) at FHNW a Python-based prototype was created to semi-automatically generate MCQ (Multiple Choice Question) based exams and summaries using the - since 2023 - very popular large language models (LLM). The current prototype software is based on the Python libraries langchain for the LLM framework, and streamlit for the user interface. As vector database ChromaDB is used and as LLM Vertex AI from Google. Currently the software is tested with selected documents in order to find out if the quality of generated exams and summaries match the expectations of the lecturers and students. The results will help to answer important questions regarding the practical usage of LLM in education in general. Currently the AI community is flooded with an immense number of software prototypes for nearly every domain. However, a sustainable operational concept for AI in general and LLM in particular is rarely discussed. Our contribution seeks to address this issue for the domain of examination support in higher technical education. Specifically, the following topics are discussed: prompt engineering, testing, quality management, confidentiality assurance and didactic strategy. These issues have to be answered before using the software in a broader context, namely other modules and programmes.
Keywords
Event
ICERI 2024
Exhibition start date
Exhibition end date
Conference start date
11.11.2024
Conference end date
13.11.2024
Date of the last check
ISBN
978-84-09-63010-3
ISSN
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
Peer review of the complete publication
Open access category
Closed
License
Citation
Telesko, R., & Wilke, G. (2024). Towards a sustainable operating concept of LLM generated examination support in higher technical education. In L. Gómez Chova, C. González Martínez, & J. Lees (Eds.), 17th annual International Conference of Education, Research and Innovation (pp. 1064–1069). IATED Academy. https://doi.org/10.21125/iceri.2024.0350