Letting the AI decide what the AI should code? On responsibility and accountability in requirements-driven software evolution

Abstract

AI agents can now take software change requests from a natural-language description to code changes. Capability alone, however, does not answer whether an agent should be allowed to act. We frame that upstream decision as a requirements engineering problem: before implementation starts, a team must decide whether a request may be delegated to AI, handled collaboratively, or kept under human control, all while humans remain accountable for the outcome. Existing work mainly automates implementation or isolated RE tasks, but offers little guidance for assigning responsibility. We therefore propose RADAR (Responsibility-Aware Delegation with Auditable Reasoning), a framework that treats responsibility allocation as a requirements artifact. The framework evaluates each incoming request along observable dimensions: clarity, traceability, scope, risk (including domain criticality), and compliance. It then routes the request through four stages: input reception, information assessment, compliance checking, and responsibility assignment. The result is one of four responsibility levels, ranging from autonomous AI execution to human-only handling. Every decision produces an explainable, auditable package with evidence, rationale, uncertainty notes, and required approvals. We present the framework, describe an initial prototype, report preliminary findings from expert interviews (N=4), and outline a twopart evaluation that studies whether the allocation criteria are defensible and whether the outputs support validation, override, and audit.