Module 7: Governing AI Deployment · BoK IV.A
The agentic risk landscape
Autonomy brings four new risk families: goal misalignment (right goal, wrong way), compounded systemic impact (errors spread across departments and platforms), root cause complexity (failures span platforms and data sources), and human disempowerment (over-reliance erodes expertise and dignity).
Four new risk families come with the autonomy. Each has a recognisable scenario shape.
- Goal misalignment → agents achieve the goal but not in the way creators intended (a cost-savings system harming customer experience or violating ethics) → reduce by aligning agents with organisational values and documenting intended functions.
- Compounded systemic impact → a single error spreads quickly across departments and platforms (a chatbot pushing wrong policy through multiple channels) → worse without monitoring and escalation processes.
- Root cause complexity → failure causes are hard to pin down when decisions span multiple platforms and data sources → the airline incident needed extra scrutiny to separate training-data gaps from user-input misinterpretation.
- Human disempowerment → prolonged reliance erodes human expertise, risking skill atrophy → potential negative psychological effects on self-worth and dignity.
Key terms - quick answers
What is “Goal misalignment”?
When an agent achieves its goal but not in the way creators intended.
What is “Compounded systemic impact”?
When a single agent error spreads quickly across departments and platforms.
What is “Root cause complexity”?
Difficulty pinning down failure causes when decisions span multiple platforms and data sources.
What is “Human disempowerment”?
Erosion of human expertise (skill atrophy) and dignity from over-reliance on agents.