AI coding assistants and internal AI tooling are only as effective as the infrastructure supporting them. Cortex gives you the visibility, automation, and measurement you need to drive real adoption, not just tool access.AI maturity in Cortex means:
Every team knows which AI tools they’re expected to use and whether they’re meeting that standard
Access requests and onboarding are automated, not manual
Leaders can see whether AI adoption is actually improving engineering performance
Work through the five steps below to configure your workspace.
AI maturity tracking only works if your services, infrastructure, and other entities are in Cortex and have clear owners. Without this foundation, Scorecards have nothing to evaluate, Workflows have no targets, and Eng Intelligence has no baselines.
Every entity in scope has an assigned owner, at least one version control integration is active, and your AI-specific entities (models, pipelines, etc.) are visible in the Cortex catalog.
Ownership gaps - Entities get imported but ownership isn’t set, so Scorecards can’t route notifications and Eng Intelligence can’t group data by team. Bulk-setting ownership via the API or a CSV import is faster than doing it entity by entity.
Integrations connected, but not fully scoped - GitHub might be connected at the org level but missing specific repos, so version control rules silently fail. Double-check that the integration covers all the repos your entities reference.
Scorecards automatically check whether your services and teams are meeting AI adoption standards. The AI maturity template gives you a starting point—a tiered set of rules (Bronze, Silver, Gold) based on common industry practices—that you can customize to fit your organization.
Create a Scorecard using the AI maturity template. The template includes rules that verify AI tooling is properly configured across your repositories. Examples include:
Rule
What it checks
AI instructions file present
A file like AGENTS.md, .cursorrules, or CLAUDE.md exists in the repository
Copilot enabled
GitHub Copilot is enabled for the team owning this service
AI-assisted PR reviews configured
An AI code review tool (e.g. CodeRabbit, GitHub Copilot code review) is active on the repo
AI anomaly detection integrated
The service is connected to an AI-based monitoring or alerting tool
Delete rules that don’t apply, re-weight rules to reflect your priorities, and add custom rules using CQL to check for tooling specific to your organization. Each rule can be assigned more points to signal its relative importance.
Scorecard is scoped to the wrong entities - If the entity filter is too broad, you’ll get noise from entities that shouldn’t be measured against AI standards. If it’s too narrow, key services get missed. Review the filter before publishing.
Template rules reference integrations you haven’t configured yet - For example, a Copilot adoption rule will fail for every entity if the GitHub integration isn’t set up. Cross-check the template rules against your active integrations in Step 1 before going live.
Everything is failing on day one - This is normal. A Scorecard full of red on launch isn’t a problem, it’s the baseline. Communicate that to teams upfront so it doesn’t cause alarm.
Step 3: Automating AI tooling requests with Workflows
A Workflow is an automated, multi-step process that runs entirely within Cortex. It lets teams define tasks, trigger actions, collect input, and route approvals in one place, turning complex, multi-tool operations into a single repeatable experience. For AI maturity, this means you can enforce standards automatically rather than relying on manual checklists or Slack threads.
Identifying good candidates for Workflow automation
Before building, look at where friction exists in your AI maturity program. Scorecards and Eng Intelligence are useful here: if a team consistently fails the same rules, or cycle time data shows work getting stuck at a particular handoff, that’s a strong signal that the underlying process should be a Workflow.
Automating access requestsBuild a Workflow that handles GitHub Copilot (or other AI tool) access requests end-to-end: a developer submits a request, the Workflow routes it for approval, and access is provisioned automatically on approval. See the example Workflow for Copilot access automation.Enforcing standards at service creationUse Scaffolder templates to ensure every new service is created with baseline AI standards already in place: AI instructions files, anomaly detection integrations, and links to AI diagnostic runbooks.
Your AI tool access request Workflow (e.g. GitHub Copilot) is active and has completed at least one successful end-to-end run—request, approval, provisioning—and your Scaffolder template includes AI-specific baseline standards so new services start with them in place. If new services still need manual setup after creation, or engineers are requesting AI tool access informally over Slack, this step isn’t done.
Undefined approval chain - If the Workflow routes a request but nobody is designated to approve it, requests stall and engineers abandon the process and find workarounds.
Scaffolder templates that reference AI tools or integrations not yet fully rolled out - A template that includes AI-based anomaly detection configuration for a tool your org hasn’t standardized on yet will confuse service owners immediately.
Bypassing the Scaffolder by cloning repos or creating services directly - The Scorecard catches the gaps eventually, but you’ll be cleaning up after the fact rather than preventing them.
Cortex MCP lets engineers query your Cortex workspace in natural language from their MCP client (Claude, Cursor, VS Code, and others). Instead of switching to a browser to look up service owners, on-call rotations, or Scorecard status, engineers get answers inline while they work.Relevant for AI Maturity specifically:
Find ownership fast - Ask Who owns the embeddings pipeline? to immediately identify which team is accountable for an AI service, without digging through wikis or Slack history.
Check Scorecard status - Ask What are the failing rules for the inference service AI Maturity Scorecard? to get a prioritized list of gaps to fix.
Get remediation suggestions - Ask What are quick wins for my AI Maturity Scorecard? and MCP will return actionable next steps based on your current scores.
Configure the Cortex MCP. It can be hosted locally or remotely. Refer to Using the Cortex MCP for information on starting a new chat, crafting effective prompts, and using Eng Intelligence metrics in chat.
MCP not connected to the right workspace - If the MCP is pointed at the wrong Cortex workspace or authenticated with credentials that have limited permissions, queries will return incomplete or no results. Verify the connection returns accurate data for a service you know well before rolling it out.
User labels haven’t been configured yet (Step 5 prerequisite) - MCP queries about “which teams are using AI tools” won’t return useful results since Cortex doesn’t know who those teams are yet. See user labels
Questions that are too vague - MCP works best with specific, scoped questions. How are my AI services doing? returns less useful results than What are quick wins for my AI Governance Scorecard? Train teams on the kinds of questions that get actionable answers.
Teams not aware it exists - MCP adoption tends to be low when it’s announced once and forgotten. Engineers default to existing habits (Slack, wikis, asking a colleague) unless MCP is introduced in the context of a workflow they already use, like a Scorecard remediation or an incident investigation.
Before you roll out AI tooling broadly, capture baselines in DORA metrics (deployment frequency, MTTR, change failure rate, lead time) and velocity metrics. These numbers are your before state and you’ll need them to show impact.
Apply user labels to identify teams. User labels let you group engineers into cohorts and compare metrics side by side, e.g. deployment frequency for Copilot users vs. non-users over the same period.This makes it straightforward to answer the question leadership will ask: Is the AI tooling investment actually working?
No baseline captured before rollout - Once AI tools are adopted and metrics shift, you lose the before/after comparison if you didn’t pull numbers first.
User labels not configured - The comparison between AI-adopting teams and non-adopting teams is one of the most valuable things Eng Intelligence can show you, but it requires labels to be set up and assigned before the measurement period starts. Setting them up after the fact means your cohorts aren’t clean.
Tracking the wrong metrics for an AI maturity use case - Cycle time and MTTR are useful, but the more telling signals are PR size, review time, and merge frequency from your version control integration. These show whether AI coding tools are actually changing how engineers work. If those integrations weren’t connected in Step 1, they won’t be available here.
No regular reviewer - Eng Intelligence surfaces trends, but someone needs to own the cadence of reviewing them and acting on regressions. Without that ownership established now, the dashboards go unread.
