Life cycle policies and the use case assessment
Last reviewed:
Policies must create oversight across nine areas of the AI life cycle. The Use case assessment is the front door, running NIST's Map (NIST) · Measure (NIST) · Manage (NIST) → conducted before implementation AND continuously.
Policies must create oversight and accountability across every life cycle stage. The use case assessment is the front door.
- Use case assessment
- Risk management
- Ethics by design
- Data acquisition & use
- Model & system development
- Training & testing
- Deployment & monitoring
- Documentation & reporting
- Incident management
A use case assessment is a structured process evaluating the viability, risks and ethical implications of applying AI to a specific problem, ensuring responsible, effective, compliant deployment. It follows NIST's map · measure · manage.
- Map (Phase 1) → establish context and identify risks in that context → document intended purposes, beneficial uses and deployment settings; identify stakeholders and impacts; categorise the system; map limits, risks and benefits including third-party software and data; document knowledge limits and human oversight of output.
- Measure (Phase 2) → assess, analyse and track the mapped risks → apply metrics for trustworthy characteristics (accuracy, robustness, fairness); rate severity, likelihood and scope incl. bias and security vulnerabilities; continuously track risks and gather feedback on measurement efficacy.
- Manage (Phase 3) → prioritise and act → plan responses by projected impact → mitigate, transfer, avoid or accept; implement security controls and safeguards; monitor behaviour, update controls, run continuous improvement plans.
- Before implementation → strategic alignment, feasibility, risk identification
- Early in the life cycle → mapping is the foundational first step of risk management
- For any new initiative → especially significant-impact or high-risk systems
- Throughout the life cycle + for regulatory compliance → risks and performance evolve, so reviews and checkpoints are ongoing
| Scenario | When to assess | Why |
|---|---|---|
| AI cancer detection for radiologists | During the design phase → choosing algorithms, data sources, training plans | Map life-critical false positives and negatives, bias from undiverse training data, protected health information security, and the need for a radiologist always making the final diagnosis |
| Third-party sentiment tool bought off the shelf | Before integrating the third-party solution | Evaluate the vendor's governance, transparency, performance on relevant data, compliance with internal policy → understand the risks of a black-box system |
| Bank customer service chatbot | Before allocating significant resources or buying a vendor solution | Check a chatbot is genuinely the best solution vs improved FAQs or agent training · risks of misread intent, wrong answers, sensitive financial data, job impacts |
If asked when a use case assessment happens, the answer is before implementation AND continuously → never a one-off. And the medical example's control is human oversight, the radiologist makes the final call.
Key terms - quick answers
What is “Use case assessment”?
What is “Map (NIST)”?
What is “Measure (NIST)”?
What is “Manage (NIST)”?
Keep going - free
Every study note on this site is free. So are these: