Toward a holistic framework for human-AI collaboration in safety-critical systems

The integration of artificial intelligence (AI) into safety-critical systems, where human operators remain central to decision-making, introduces various challenges that existing AI frameworks struggle to address comprehensively. Key concerns involve designing a socio-technical system that balances AI transparency, trust, and explainability with the imperative for robust and reliable decision-making. Presently, while numerous sector-specific solutions exist, a holistic framework that effectively integrates human expertise with AI capabilities remains absent, leaving critical gaps in system design, deployment, and oversight. This chapter proposes a multidisciplinary conceptual framework to enhance human-AI collaboration in critical infrastructures such as power grids, railways, and air traffic management. The different design steps were guided by the requirements of these industrial domains. The framework combines key design principles that support human cognition, leveraging insights from decision theory, mathematics, and specialized engineering domains to optimize AI-assisted decision-making. Furthermore, it embeds trustworthiness and risk assessment methodologies, using tools such as the Assessment List for Trustworthy Artificial Intelligence (ALTAI) tool to ensure compliance with ethical and regulatory requirements.

Schlagwörter

Fachgebiet (DDC)

004 - Computer Wissenschaften, Internet

Projekt

Veranstaltung

Startdatum der Ausstellung

Enddatum der Ausstellung

Startdatum der Konferenz

Enddatum der Konferenz

Datum der letzten Prüfung

ISBN

978-3-032-10560-8
978-3-032-10561-5

ISSN

Sprache

Englisch

Während FHNW Zugehörigkeit erstellt

Ja

Zukunftsfelder FHNW

Publikationsstatus

Veröffentlicht

Begutachtung

peer-reviewed

Open Access-Status

Gold

Lizenz

Zitation

Bessa, R. J. G. d. S. B., Leyli-Abadi, M., Yagoubi, M., Boos, D., Borst, C., Castagna, A., Chavarriaga, R., Dias, D., Egli, A., Eisenegger, A., Ellerbroek, J., Fedorova, A., Felix, C., Fuxjäger, A., Geraldes, J., Hamouche, S., Hassouna, M., Kop, S., Lemetayer, B., et al. (2026). Toward a holistic framework for human-AI collaboration in safety-critical systems. In E. Curry, P. Piatkiewicz, F. Heintz, H. Vornhagen, A. Nabil Belbachir, E. Girardi, M. Schoenauer, & J. Röning (Eds.), Artificial Intelligence, Data and Robotics. Foundations, Transformations and Future Directions (pp. 343–402). Springer. https://doi.org/10.1007/978-3-032-10561-5_13

Komplettanzeige