Module 6: Governing AI Development · BoK III.A
The Six Risk Assessment Strategies, In Order
A repeatable sequence - Use Smart Methods, Handle Big Projects - to identify, evaluate, treat and mitigate risk: use case evaluation, stakeholder mapping, harms matrix, mitigation hierarchy, benchmarking, pre-deployment pilots. Stage placement is exactly what gets tested.
A repeatable sequence to identify, evaluate, treat and mitigate risk across the life cycle. Stage placement is exactly what gets tested.
Mnemonic
Use Smart Methods, Handle Big Projects → Use case evaluation · Stakeholder mapping · harms Matrix · mitigation Hierarchy · Benchmarking · Pre-deployment pilots.
| Strategy | When · who | What it does |
|---|---|---|
| 1 · Use case evaluation | Planning & design · all models | Determines if the need warrants AI at all and informs the model type → evaluate ease of implementation, strategic alignment, required expertise → flag risks |
| 2 · Stakeholder mapping | Planning & design · all models | Project-management step ensuring the correct parties are in the decision-making process → map interests, keep communication open, spot risks early |
| 3 · Probability/severity harms matrix | Design & development · all models | Rate severity and probability → multiply the two scores for the risk score |
| 4 · Risk mitigation hierarchy | Design, development & implementation · all models | Used in tandem with the matrix → the "now what" → avoid, minimise, remediate and/or offset a risk's impact |
| 5 · Benchmarking | Planned late-design · executed once a candidate model exists · esp. ML, neural networks, reinforcement learning | Standardised tests comparing systems on accuracy, speed, complex-task handling → particularly useful for "black box" models → broad or aspect-specific |
| 6 · Pre-deployment pilots | Planned late-design · executed immediately before go-live · all models | Trial phase with settings matching production as closely as possible → confirms the AI works as expected, with a chance to update before deployment |
Key terms - quick answers
What is “Use case evaluation”?
Risk strategy determining whether the need warrants AI at all and informing the model type.
What is “Stakeholder mapping”?
Risk strategy ensuring the correct parties are in the decision-making process and spotting risks early.
What is “Probability/severity harms matrix”?
Risk strategy rating severity and probability and multiplying them for a risk score.
What is “Risk mitigation hierarchy”?
The 'now what' used with the matrix - avoid, minimise, remediate and/or offset a risk's impact.