Toward a holistic framework for human-AI collaboration in safety-critical systems

The integration of artificial intelligence (AI) into safety-critical systems, where human operators remain central to decision-making, introduces various challenges that existing AI frameworks struggle to address comprehensively. Key concerns involve designing a socio-technical system that balances AI transparency, trust, and explainability with the imperative for robust and reliable decision-making. Presently, while numerous sector-specific solutions exist, a holistic framework that effectively integrates human expertise with AI capabilities remains absent, leaving critical gaps in system design, deployment, and oversight. This chapter proposes a multidisciplinary conceptual framework to enhance human-AI collaboration in critical infrastructures such as power grids, railways, and air traffic management. The different design steps were guided by the requirements of these industrial domains. The framework combines key design principles that support human cognition, leveraging insights from decision theory, mathematics, and specialized engineering domains to optimize AI-assisted decision-making. Furthermore, it embeds trustworthiness and risk assessment methodologies, using tools such as the Assessment List for Trustworthy Artificial Intelligence (ALTAI) tool to ensure compliance with ethical and regulatory requirements.

Keywords

Subject (DDC)

004 - Computer Wissenschaften, Internet

Project

Event

Exhibition start date

Exhibition end date

Conference start date

Conference end date

Date of the last check

ISBN

978-3-032-10560-8
978-3-032-10561-5

ISSN

Language

English

Created during FHNW affiliation

Yes

Strategic action fields FHNW

Publication status

Published

Review

peer-reviewed

Open access category

Gold

License

Citation

Bessa, R. J. G. d. S. B., Leyli-Abadi, M., Yagoubi, M., Boos, D., Borst, C., Castagna, A., Chavarriaga, R., Dias, D., Egli, A., Eisenegger, A., Ellerbroek, J., Fedorova, A., Felix, C., Fuxjäger, A., Geraldes, J., Hamouche, S., Hassouna, M., Kop, S., Lemetayer, B., et al. (2026). Toward a holistic framework for human-AI collaboration in safety-critical systems. In E. Curry, P. Piatkiewicz, F. Heintz, H. Vornhagen, A. Nabil Belbachir, E. Girardi, M. Schoenauer, & J. Röning (Eds.), Artificial Intelligence, Data and Robotics. Foundations, Transformations and Future Directions (pp. 343–402). Springer. https://doi.org/10.1007/978-3-032-10561-5_13

Full item page