Towards a sustainable operating concept of LLM generated examination support in higher technical education
Lade...
Autor:innen
Autor:in (Körperschaft)
Publikationsdatum
2024
Typ der Arbeit
Studiengang
Typ
04B - Beitrag Konferenzschrift
Herausgeber:innen
Herausgeber:in (Körperschaft)
Betreuer:in
Übergeordnetes Werk
17th annual International Conference of Education, Research and Innovation
Themenheft
DOI der Originalpublikation
Link
Reihe / Serie
ICERI Proceedings
Reihennummer
Jahrgang / Band
Ausgabe / Nummer
Seiten / Dauer
1064-1069
Patentnummer
Verlag / Herausgebende Institution
IATED Academy
Verlagsort / Veranstaltungsort
Valencia
Auflage
Version
Programmiersprache
Abtretungsempfänger:in
Praxispartner:in/Auftraggeber:in
Zusammenfassung
At the University of Applied Science Northwestern Switzerland (FHNW) it can be observed that the usage of preferred learning material shifted in the past years from books and articles to videos, summaries, flashcards and test exams. In a specialization module of the programme BSc Business Information Technology (BIT) at FHNW a Python-based prototype was created to semi-automatically generate MCQ (Multiple Choice Question) based exams and summaries using the - since 2023 - very popular large language models (LLM). The current prototype software is based on the Python libraries langchain for the LLM framework, and streamlit for the user interface. As vector database ChromaDB is used and as LLM Vertex AI from Google. Currently the software is tested with selected documents in order to find out if the quality of generated exams and summaries match the expectations of the lecturers and students. The results will help to answer important questions regarding the practical usage of LLM in education in general. Currently the AI community is flooded with an immense number of software prototypes for nearly every domain. However, a sustainable operational concept for AI in general and LLM in particular is rarely discussed. Our contribution seeks to address this issue for the domain of examination support in higher technical education. Specifically, the following topics are discussed: prompt engineering, testing, quality management, confidentiality assurance and didactic strategy. These issues have to be answered before using the software in a broader context, namely other modules and programmes.
Schlagwörter
Veranstaltung
ICERI 2024
Startdatum der Ausstellung
Enddatum der Ausstellung
Startdatum der Konferenz
11.11.2024
Enddatum der Konferenz
13.11.2024
Datum der letzten Prüfung
ISBN
978-84-09-63010-3
ISSN
Sprache
Englisch
Während FHNW Zugehörigkeit erstellt
Ja
Zukunftsfelder FHNW
Publikationsstatus
Veröffentlicht
Begutachtung
Peer-Review der ganzen Publikation
Open Access-Status
Closed
Lizenz
Zitation
Telesko, R., & Wilke, G. (2024). Towards a sustainable operating concept of LLM generated examination support in higher technical education. In L. Gómez Chova, C. González Martínez, & J. Lees (Eds.), 17th annual International Conference of Education, Research and Innovation (pp. 1064–1069). IATED Academy. https://doi.org/10.21125/iceri.2024.0350