Psychometric evaluation of the AI-based career matching platform meinBeruf.ch. Evidence for reliability, construct validity, and gender fairness

Abstract

Abstract: Objective: This study examined the preliminary psychometric quality and construct validity of the meinBeruf.ch assessment system and evaluated gender-related fairness at the level of input variables and algorithmic recommendations. Methods: The sample comprised 914 users of the meinBeruf.ch platform. Participants completed standardized personality (BFI-2), interest (RIASEC), cognitive (HMT), and domain-specific skill measures, followed by algorithmic career recommendations. Reliability and construct validity were assessed using internal consistency coefficients and correlations between psychological scales and industry-fit scores. Gender fairness was examined by testing whether self-reported gender added incremental predictive value beyond psychological profiles, and by reporting supplementary input-variable analyses of gender differences, DIF, measurement invariance, and a technical counterfactual audit of the ranking layer. Results: The psychological scales demonstrated adequate to high internal consistencies, and correlations with algorithmic recommendations showed theoretically consistent patterns. Adding gender to random-forest prediction models changed predictive accuracy only minimally ( M | ΔR 2| = .011), indicating that gender provided little systematic predictive value beyond the psychological profile. Supplementary analyses documented expected gender differences in some input variables and no counterfactual rank changes when only the gender label was varied. Limitations: The study used a self-selected online sample, relied primarily on cross-sectional platform data, and lacked long-term criterion outcomes such as later career choice, satisfaction, or persistence. Conclusion: The meinBeruf.ch assessment system provides preliminary evidence for psychometric quality and construct validity. The algorithmic ranking layer behaved technically gender-fair, while gender-related differences in personality, interests, and self-estimated skills remain important for interpretation and future research, especially in STEM-related vocational guidance.