Life cycle policies and the use case assessment
Policies must create oversight across nine areas of the AI life cycle. The Use case assessment is the front door, running NIST's Map (NIST) · Measure (NIST) · Manage (NIST) → conducted before implementation AND continuously.
Policies must create oversight and accountability across every life cycle stage. The use case assessment is the front door.
- Use case assessment
- Risk management
- Ethics by design
- Data acquisition & use
- Model & system development
- Training & testing
- Deployment & monitoring
- Documentation & reporting
- Incident management
A use case assessment is a structured process evaluating the viability, risks and ethical implications of applying AI to a specific problem, ensuring responsible, effective, compliant deployment. It follows NIST's map · measure · manage.
- Map (Phase 1) → establish context and identify risks in that context → document intended purposes, beneficial uses and deployment settings; identify stakeholders and impacts; categorise the system; map limits, risks and benefits including third-party software and data; document knowledge limits and human oversight of output.
- Measure (Phase 2) → assess, analyse and track the mapped risks → apply metrics for trustworthy characteristics (accuracy, robustness, fairness); rate severity, likelihood and scope incl. bias and security vulnerabilities; continuously track risks and gather feedback on measurement efficacy.
- Manage (Phase 3) → prioritise and act → plan responses by projected impact → mitigate, transfer, avoid or accept; implement security controls and safeguards; monitor behaviour, update controls, run continuous improvement plans.
- Before implementation → strategic alignment, feasibility, risk identification
- Early in the life cycle → mapping is the foundational first step of risk management
- For any new initiative → especially significant-impact or high-risk systems
- Throughout the life cycle + for regulatory compliance → risks and performance evolve, so reviews and checkpoints are ongoing
| Scenario | When to assess | Why |
|---|---|---|
| AI cancer detection for radiologists | During the design phase → choosing algorithms, data sources, training plans | Map life-critical false positives and negatives, bias from undiverse training data, protected health information security, and the need for a radiologist always making the final diagnosis |
| Third-party sentiment tool bought off the shelf | Before integrating the third-party solution | Evaluate the vendor's governance, transparency, performance on relevant data, compliance with internal policy → understand the risks of a black-box system |
| Bank customer service chatbot | Before allocating significant resources or buying a vendor solution | Check a chatbot is genuinely the best solution vs improved FAQs or agent training · risks of misread intent, wrong answers, sensitive financial data, job impacts |
If asked when a use case assessment happens, the answer is before implementation AND continuously → never a one-off. And the medical example's control is human oversight, the radiologist makes the final call.