Incidents, consequences and accountability
Treat every occurrence as an incident, keep records in an AI registrar, and know the five usual causes - brittleness, lack of robustness, lack of quality data, insufficient testing, and model or data drift. Forecast downstream consequences (resentment, false safety, a roadmap for malicious actors). Tools like AI Verify (11 ethics principles) automate accountability.
Treat every occurrence as an incident, learn why it happened, and keep the receipts in one place.
- Incident discipline → identify the issue and who it must be reported to, inside and outside the organisation → keep incident and issue information in an AI registrar → document the mitigation and communication.
- Why incidents happen → the five usual suspects → brittleness · lack of robustness · lack of quality data · insufficient testing · model or data drift. An incident's impact rarely stops at one tool, so notify partner and internal groups.
- Resentment → poorly implemented interventions and opacity read as unfairness, arbitrariness or ideological influence; superficial policies meet the letter but not the spirit of the law.
- False sense of safety → assuming all risks were addressed, likelier when there are incentives to obscure or reframe risks; one-time evaluation vs continuous monitoring is the failure pattern.
- Unintended consequences → asking developers to reflect on misuse can itself create a "roadmap" for malicious actors
Timing → review downstream consequences early in the research and development pipeline · Risk levels → categorise research and consequences by risk · Normalise discussions of downstream consequences, negative and positive · be fully transparent and proactive in identifying negatives · develop common protocols for responsible development, deployment and continuous improvement. Updates should be freely available in plain language, with use guidance and mitigation strategies.
- Accountability mechanisms → governments increasingly require audits and assessments by use case and risk level.
- Human review → respect automated decision-making rules for personal data; reviewers must be trained and empowered to override automated decisions (beware automation bias).
- Automation tools → manual validation is slow, costly and error-prone, so automation institutionalises processes and continuously collects evidence → AI Verify (Singapore) validates systems against 11 ethics principles; the Model Card Regulatory Check app automates compliance checks from model cards.
1) Review policies (privacy, security, IP, model ops, open source) · 2) Evaluate options → cloud scales, on-prem controls, edge localises · 3) Manage third-party risk → two contexts, five riding risks, one screening strategy · 4) Prioritise ethics → document competing-value calls, keep humans in the loop · 5) Assess readiness → Well-Tested Code Deserves Model-cards · 6) Monitor continuously → baselines, drift detection, snapshots, red teaming, challenger models · 7) Document incidents → AI registrar, the five causes, human shutdown · 8) Ensure transparency → disclosure that AI is in place is the global threshold. Mnemonic bank → Cloud scales · On-prem controls · Edge localises · Foundation → Risk → Society · Well-Tested Code Deserves Model-cards.