Module 7: Governing AI Deployment · BoK IV.B
The vendor / open-source agreement checklist
Eight areas to evaluate before signing a vendor or open-source agreement: data considerations, security/safety, bias metrics, product type, technical specs, performance results, monitoring & maintenance, and terms of use. Each row is a potential scenario question.
Eight areas to evaluate before signing. Each row is a potential scenario question.
| Area | What to ask |
|---|---|
| Data considerations | Do they have legal rights to the data used? Was personal data minimised and deidentified before training or testing? If they collect data the organisation uses with the model, how will they use it? |
| Security / safety | The model's identified risks · potential to fail, be misused, be attacked or be used for a high-risk activity · are incident response plans in place? |
| Bias metrics | Steps taken to minimise bias · evidence the AI does not produce statistical inaccuracies, bias or discrimination affecting people |
| Product type | Internal use or external-facing? Does it generate content? |
| Technical specs | The model types provided · the types of datasets used to train |
| Performance results | What ensures model stability and prevents inaccurate outputs? |
| Monitoring & maintenance | How will the model be monitored and maintained? |
| Terms of use | Is fine-tuning allowed? For content generators, what do the terms say about intellectual property and model outputs? |
Key terms - quick answers
What is “Vendor agreement checklist”?
Eight areas to evaluate before signing: data, security/safety, bias, product type, technical specs, performance, monitoring, terms of use.