Operations
Council Health Check
A maturity model and self-assessment for evaluating your AI governance program.
A council that never evaluates itself will drift. This page provides a structured health check you can run at the 6-month mark, annually, or whenever the program feels stuck.
How to Use This
Run the health check as a council exercise. Have each member score independently, then discuss the results together. The gaps between members' scores are often more revealing than the scores themselves.
Maturity Levels
The health check uses four maturity levels:
| Level | Name | Description |
|---|---|---|
| 0 | Not started | No capability in place |
| 1 | Initial | Ad-hoc or informal. Some activity, but no consistent process |
| 2 | Established | Defined process in place, consistently followed |
| 3 | Optimizing | Process is measured, reviewed, and actively improved |
You do not need to be at level 3 everywhere. A council in its first year should aim for level 2 across the foundations and level 1-2 in operations. Level 3 is a target for mature programs.
Assessment
Governance Foundations
| Capability | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Charter. The council has a written charter defining mission, scope, authority, and accountability | No charter | Draft exists but not formally approved | Approved charter, reviewed within the last year | Charter reviewed annually, updated when scope or context changes |
| Executive sponsorship. A named senior leader is accountable for AI governance | No sponsor | Informal support from a senior leader | Named sponsor, regular briefings | Sponsor actively champions governance, reports to the board |
| Principles. The council has adopted a set of AI principles | No principles | Principles drafted but not operationalized | Principles adopted, referenced in reviews and decisions | Principles embedded in intake forms, cited in decision records, published externally |
| Membership. The council has cross-functional representation with defined roles | No defined membership | Informal group, inconsistent attendance | Defined roles, regular attendance, staggered terms | Active recruitment, onboarding process, diversity of perspective |
| Meeting cadence. The council meets regularly with structured agendas | No regular meetings | Meetings happen but inconsistently | Regular cadence, structured agendas, minutes within 48 hours | Cadence adapted to workload, meetings consistently productive |
Intake and Triage
| Capability | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| AI inventory. A register of all AI systems exists and is maintained | No inventory | Partial inventory, updated sporadically | Inventory covers known systems, updated at intake and quarterly | Inventory validated annually against procurement and IT records, gaps actively closed |
| Intake process. New AI use cases are registered through a standard process | No intake process | Some cases are registered, no standard form | Standard registration form, all new cases go through intake | Intake process measured (volume, turnaround), feedback from submitters incorporated |
| Risk tiering. Use cases are classified by risk level | No tiering | Informal risk judgement by individuals | Defined tiers with criteria, consistently applied | Tiering criteria reviewed every 6 months, calibrated against actual outcomes |
| Routing logic. Cases are routed to the right level of review | No defined routing | Chair routes cases informally | Defined routing rules, turnaround times tracked | Pre-approved patterns in use, routing adapted based on volume and experience |
| Vendor governance. Procured AI is assessed alongside in-house systems | Vendor AI not assessed | Some vendor cases reviewed, no standard checklist | Vendor checklist used consistently, vendor cases tiered appropriately | Vendor assessments integrated with procurement process, ongoing vendor monitoring |
Review and Assurance
| Capability | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Impact assessments. High-risk cases receive structured assessment | No assessments | Assessments done informally or inconsistently | Standard template used for all Tier 3 cases | Assessment quality reviewed, templates updated based on experience |
| Security review. AI-specific security risks are assessed | Security not part of AI review | Security consulted informally on some cases | Security review checklist used for Tier 2+ cases | Security review integrated with red-teaming, updated for emerging threats |
| Decision records. Council decisions are documented with rationale | No decision records | Some decisions recorded, inconsistent format | All decisions logged with rationale, conditions, and review dates | Decision log analyzed for patterns, informs policy updates |
| Human oversight. Oversight levels are defined proportionate to risk | No oversight framework | Oversight discussed informally during review | Oversight level required in impact assessment, matched to tier | Oversight effectiveness monitored (override rates, automation bias indicators) |
Operations
| Capability | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Post-deployment monitoring. Deployed AI systems are monitored for performance and drift | No monitoring after approval | Some systems monitored, no standard cadence | Monitoring calendar in place, periodic reviews by tier | Automated monitoring with alerting, drift detection, fairness tracking |
| Incident management. AI incidents and near-misses are reported and managed | No AI incident process | Incidents handled ad-hoc through general IT process | Dedicated AI incident process, incidents logged and reviewed | Post-incident reviews drive policy updates, near-misses actively tracked |
| Policy refresh. Governance artifacts are reviewed and updated | Policies not reviewed after creation | Occasional updates when problems arise | Scheduled review cadence for all artifacts | Refresh triggered by regulation, incidents, and feedback, with change logs |
| Training and literacy. Staff involved in AI have sufficient AI literacy | No AI-specific training | Some training available, not systematic | Tiered training program (all staff, practitioners, champions, council) | Training effectiveness measured, content updated annually |
| Reporting. The council reports on program health to leadership | No reporting | Informal updates to sponsor | Quarterly reports to sponsor, annual report to board | Reports include maturity trends, benchmarking, strategic recommendations |
Champion Network
| Capability | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Coverage. Champions are embedded in teams that build or use AI | No champions | 1-2 champions, limited coverage | Champions in most high-AI-activity teams | Full coverage, champions in every team with active AI use |
| Engagement. Champions are active and effective | No champion activity | Champions identified but rarely engaged | Champions handle Tier 1-2 cases, attend monthly briefings | Champions contribute to policy, surface insights, peer-support community active |
| Support. Champions have training, time, and tools | No support structure | Informal guidance only | Dedicated time, training program, communication channel | Champions recognized in performance reviews, development pathway defined |
Scoring
After completing the assessment, calculate your profile:
- Count your scores across all 20 capabilities
- Identify gaps: any capability at level 0 is a critical gap that needs immediate attention
- Find your floor: the lowest-scoring section indicates where the program is most vulnerable
Benchmark Targets
| Program Age | Target Profile |
|---|---|
| 0-6 months | Foundations at level 2, everything else at level 1+ |
| 6-12 months | Foundations and Intake at level 2, Review and Operations at level 1-2 |
| 12-24 months | Most capabilities at level 2, some at level 3 |
| 24+ months | Most capabilities at level 2-3, no capabilities at level 0 |
What to Do with the Results
The health check is only valuable if it leads to action.
- Pick 2-3 capabilities to focus on for the next quarter. Do not try to improve everything at once.
- Set specific targets. "Move incident management from level 1 to level 2 by Q3" is actionable. "Improve operations" is not.
- Assign owners. Each improvement target should have a named owner and a concrete next step.
- Report progress. Include health check results and improvement progress in your Quarterly Report.
- Re-run the assessment every 6-12 months to track progress and surface new gaps.