16. An Organisational AI Governance Framework

Organisations should establish clear rules before AI use becomes widespread. Governance should enable beneficial experimentation while managing legal, ethical, operational, security, and reputational risks.

16.1 Core governance elements

1. Purpose and scope: identify approved use cases, prohibited uses, and high-risk activities.
2. Accountability: assign owners for tools, data, outputs, review, incidents, and policy updates.
3. Tool approval: review security, privacy, terms, data location, integration, accessibility, and vendor controls.
4. Risk classification: categorise uses by potential impact, reversibility, sensitivity, and scale.
5. Human oversight: define when review, sign-off, or professional judgement is mandatory.
6. Transparency: require disclosure to affected people or audiences where appropriate.
7. Validation: test outputs against representative tasks and monitor errors over time.
8. Training: teach staff how to prompt, verify, protect data, and report incidents.
9. Monitoring: track benefits, complaints, errors, bias, costs, and changes in vendor terms.
10. Incident response: establish how to pause use, preserve evidence, notify responsible teams, and remediate harm.

16.2 A simple risk classification

Low-risk examples may include brainstorming public campaign themes or improving the formatting of non-sensitive text. Moderate-risk examples may include summarising internal documents approved for the tool or generating code that will be reviewed before use. High-risk examples include processing identifiable participant data, making employment or student decisions, giving medical or legal advice, controlling safety-critical systems, or automatically sending consequential communications. These categories are illustrative and should be adapted to applicable law and institutional policy.

16.3 Procurement questions

Before approving a tool, ask what data it collects, how long it retains data, whether inputs are used to train models, where data is processed, whether administrators can control access, what happens when an account is closed, how incidents are reported, whether outputs can be exported, what audit logs exist, and what contractual protections apply. Obtain legal, security, privacy, and procurement advice when the risk warrants it.

16.4 Measuring value

Measure more than speed. Evaluate accuracy, rework, user satisfaction, accessibility, quality, risk incidents, cost per completed task, and whether staff retain the skills needed to judge the output. A system that saves ten minutes but creates expensive errors may not be beneficial.