Creation of RAG Systems for Managing Massive Data in Vector Databases

Abstract

This master’s thesis explores the development and optimization of a Retrieval Augmented Generation (RAG) pipeline designed to extract contextually rich, accurate, and detail-oriented responses from extensive, multilingual technical documents stored in a vector database. Grounded in a design science research methodology, the study employs an iterative, artifact-centric approach that not only builds and refines the RAG pipeline but also systematically evaluates its effectiveness. A comprehensive literature review provided the theoretical basis for the choice of embedding models, evaluation metrics, and prompt templates. Based on these theoretical insights, a first conceptual design was created prior to coding to ensure that the practical implementation was closely aligned with the best practices, new techniques, and recognized knowledge gaps identified in the literature.