07 · Building an AI Center of Excellence¶
As adoption scales (Module 4) and maturity grows (Module 6), many organizations formalize a dedicated function — an AI Center of Excellence (CoE) — to own enablement, best practices, and cross-team knowledge sharing. This module covers when one is warranted and how to structure it.
1. When a CoE is (and isn't) warranted¶
| Signal a CoE is warranted | Signal it's premature |
|---|---|
| Multiple teams independently solving the same integration/training problems | Only one or two teams use AI tools at all |
| Governance committee (Module 2) is overwhelmed with repeat questions | Adoption is still in the "prove" phase (Module 4) |
| Leadership wants a single source of truth on AI capability and best practice | No executive sponsorship or budget exists yet |
| Cross-team knowledge sharing isn't happening organically | The organization is small enough that informal sharing already works |
A CoE stood up before there's real cross-team demand becomes overhead with no constituency; one stood up too late means duplicated effort persists for longer than necessary.
2. CoE operating models¶
| Model | Structure | Best fit |
|---|---|---|
| Central team, central execution | CoE builds and runs AI initiatives directly for other teams | Early-stage, when in-house AI skill is scarce and concentrated |
| Central team, distributed execution | CoE sets standards/playbooks; teams execute with CoE support | Growing maturity, want to preserve team autonomy |
| Hub-and-spoke | Central CoE plus embedded champions in each team | Large orgs where a fully central team can't scale to demand |
| Community of practice (no formal team) | Voluntary cross-team group, no dedicated headcount | Very early stage, or as a precursor before formal investment |
3. Core CoE responsibilities¶
| Responsibility | Deliverable |
|---|---|
| Playbooks and best practices | Reusable rollout kits (Module 4), prompt/evaluation patterns, do's-and-don'ts |
| Cross-team knowledge sharing | Regular forums, an internal knowledge base, office hours |
| Tooling curation | Maintains the vetted tool catalog referenced in governance (Module 2) |
| Training and enablement | Role-based training curricula, onboarding materials |
| Metrics and reporting | Aggregates ROI and maturity data (Modules 5-6) for leadership |
| Liaison to governance | Feeds real-world friction back into governance policy refinement |
4. Common CoE failure modes¶
| Failure mode | Fix |
|---|---|
| CoE becomes a bottleneck by trying to execute everything centrally | Move to a hub-and-spoke or distributed-execution model as demand grows |
| No clear charter or success metrics for the CoE itself | Define what the CoE is accountable for and measure it like any other function |
| CoE disconnected from governance committee | Establish a direct liaison role or shared membership |
| Seen as "the AI police" rather than an enabler | Balance playbook/enablement work visibly alongside any gatekeeping role |
| Funded as a one-time project rather than an ongoing function | Secure recurring budget and headcount, tied to the "sustain" phase of scaling (Module 4) |
Worked example¶
An organization's governance committee notices the same three integration questions recurring from different teams every month, alongside duplicated training materials being built independently by three departments. This pattern — not tool count or executive enthusiasm — is used as the evidence base to propose a CoE. It launches as a small central team with one champion embedded per major department (hub-and-spoke), owning a shared playbook repository and a monthly cross-team forum. Within two quarters, duplicate training material production stops, and the governance committee's repeat-question volume drops as the CoE's playbooks answer them proactively.
How It Actually Works¶
A CoE is warranted specifically when the prompting-technique and templating problem described in Module 4 of this level ("scaling adoption") has outgrown what any one champion or team can absorb alone: multiple teams independently rediscovering the same effective prompt structures, the same retrieval-integration patterns, or the same verification workflows is a sign that valuable, reusable technical knowledge is being duplicated rather than shared — a coordination failure a CoE exists to fix by centralizing that knowledge capture once instead of letting it happen redundantly, team by team. This is a genuinely different problem from "not enough people have access to tools," which a CoE alone doesn't solve — access is a licensing and governance question (Modules 2 and 3 of this level), while a CoE's core value is turning tacit, individually-held technique into shared, documented organizational capability.
A CoE's ongoing technical-currency function also matters mechanically: as Module 9 of Level 3 covered, underlying model capabilities shift on a different, often faster cadence than product branding suggests (better context windows, better retrieval integration, more reliable agentic tool-use), and a CoE positioned to track that shift can proactively identify when a previously-unreliable use case (say, one that depended on unaided recall) becomes viable via a newer capability (retrieval grounding, better verification loops) — translating a capability shift into an updated internal best practice faster than it would otherwise propagate through an organization via ad hoc individual discovery.
Exercise¶
Using the section 1 signals, assess whether a CoE is currently warranted for your organization (or a plausible one). If yes, choose an operating model from section 2 and draft a one-page charter covering the section 3 responsibilities it would own first.