Skip to main content
Scorecard evaluations show how your entities measure up against the standards you’ve defined. Reviewing the results helps you spot patterns across teams and entities, and decide where to focus improvement work. When an entity fails a rule, Cortex shows you why, so teams can find and fix the problem quickly. As teams resolve failures, entities become healthier and more mature, and your engineering organization becomes more reliable and compliant overall. This page covers:

​
How Scorecards are evaluated

After you save a Scorecard, Cortex automatically evaluates every entity the Scorecard applies to. Cortex then re-evaluates the Scorecard on a schedule set by its evaluation window, which defaults to every 4 hours. To change the evaluation window, see Create a Scorecard. Cortex also starts a new evaluation whenever a change to the Scorecard definition is processed, whether the change comes through GitOps, the Cortex UI, or the API. Cortex doesn’t evaluate every entity at once. It spreads entity evaluations across the evaluation window, and each evaluation finishes before the next one is scheduled to start. Scorecards that use CQL filters can take longer, because Cortex evaluates each filter separately. To see which rules and filters add the most work, check the Scorecard’s Evaluation cost tab. To see past evaluations and any entities that failed to evaluate, check the Scorecard’s Recent runs tab.

​
Manually triggering an evaluation

Users with the Run re-evaluation of Scorecards permission can trigger an evaluation. To evaluate a Scorecard outside its schedule, for example after you fix a failing rule:
  1. From the main sidebar, expand Scorecards, then select Scorecards.
  2. Select the Scorecard you want to evaluate. The Scorecard page opens.
  3. In the right side panel, click Evaluate now.
    The Evaluate now button in the side panel of a Scorecard page.
Evaluate now isn’t available while an evaluation is already in progress. To re-evaluate a single entity instead of the whole Scorecard, use the Scorecards API.

​
Canceling an evaluation

While a Scorecard is being evaluated, a progress bar appears at the top of its page. To stop the evaluation, click Cancel next to the progress bar.
The Cancel button at the top of a Scorecard page.
The canceled run appears on the Recent runs tab with the Cancelled status.

​
Reviewing a Scorecard in the Cortex UI

To open a Scorecard, select Scorecards in the main sidebar, then select the Scorecard you want to review.
A Scorecard page in Cortex.
The top of the page summarizes the Scorecard’s results:
  • Levels - How many entities are at each level, including entities that haven’t reached a level yet.
  • % at highest level - The percentage of entities that have reached the Scorecard’s highest level.
  • % without level - The percentage of entities that haven’t reached any level.
The side panel on the right shows the Scorecard’s details, including its description, the total number of entities it evaluates, which entities it applies to, scheduled rules, and evaluation status. If you can edit the Scorecard, the side panel also shows its overall evaluation cost.

​
Sharing a Scorecard

To share a Scorecard (for example, to show progress to leadership), click Share in the upper-right corner. Cortex copies a link to your clipboard, which you can send to anyone who has access to view the Scorecard in your workspace.
The Share button in the upper right corner of a Scorecard page.

​
Scorecard tabs

Each tab on a Scorecard page shows a different view of the Scorecard and its results.
The tabs on a Scorecard page.
The Exemptions and Initiatives tabs only appear when the Scorecard has at least one exemption or Initiative.

​
Scores

The Scores tab lists each entity the Scorecard evaluates, the entity’s current level, and its progress in each level. Point-based Scorecards show progress as points, and level-based Scorecards show it as the number of passing rules.
The Scores tab shows scores and levels per entity.
For example, in the screenshot above, the first two entities earned 50 out of 50 possible points in the Bronze level, and 3 out of 3 possible points in the Silver level. Filter and sort the list
  • Click Filter in the upper-right corner of the list to narrow it by domain, failing rules, groups, levels, owners, passing rules, or teams.
  • Click Level to choose what to sort by, and whether to sort ascending or descending.
View an entity’s details
The side panel for an entity on the Scores tab.
Click an entity to open a side panel with more information:
  • Entity name - Links to the entity’s details page.
  • Summary - At the top of the panel, the entity’s current level, score, rank, percentile, and rule progress.
  • Rules - The entity’s passing and failing rules. Exemption requests awaiting review appear under Pending exemptions.
  • Exemptions - Rules the entity is exempt from.
  • Progress - A graph of the entity’s progress over time.
Review the Rules tab closely to find failing rules you can take action on.

​
Overview

The Overview tab highlights trends across the Scorecard, including the highest-ranking entity, the most at-risk entity and rule, the most improved entity, the entity with the biggest drop, and the most-moved rule. Below these highlights, a graph shows the progress of all rules. Click All rules above the graph to filter it by specific rules.
The Overview tab on a Scorecard page.
By default, the Trends and changes section shows data from the last month. Click Last month to choose a different timeframe.

​
Rules

The Rules tab lists the Scorecard’s rules by level. Expand a rule to see its CQL expression. If a rule doesn’t have a name, it appears as its CQL expression.
An expanded rule showing its CQL expression.

​
Reports

The Reports tab links to the Scorecard’s Bird’s Eye report, Progress report, and Report card.
The Reports tab on a Scorecard page.

​
Exemptions

The Exemptions tab lists rule exemptions for the Scorecard, in two views:
  • Approved - Exemptions that are currently in force, plus requests awaiting review.
    The Approved view of the Exemptions tab.
  • All - The Scorecard’s full exemption history, including exemptions that were rejected, revoked, or expired.
    The All view of the Exemptions tab.
Each exemption shows its current status, the entity and rule, who requested it, the reason for the request, who resolved it, and an expiration date. Click Filter to narrow either view by status, entity, rule, requester, or approver. For what each status means, see Scorecard rule exemptions. To review requests, you must have the Configure Scorecard Exemptions permission. In the Approved view:
  • Click Accept to approve a pending request, or Reject to deny it.
  • Click Revoke exemption to end an exemption that’s already approved. A revoked exemption isn’t deleted. It moves to the All view with the REVOKED status and keeps its original approval details.

​
Initiatives

The Initiatives tab lists the Initiatives associated with the Scorecard.
The Initiatives tab on a Scorecard page.

​
Recent runs

The Recent runs tab shows the Scorecard’s evaluation history.
The Recent runs tab on a Scorecard page.
Each run includes:
  • Evaluation start time - The date, time, and timezone the evaluation started.
  • Status - Whether the run is Queued, In progress, Done, Failed, Timed out, or Cancelled.
  • Entities - The number of entities the run evaluated.
Review evaluation failures If any entities failed to evaluate, click Failures in last evaluation in the upper right corner of the list. Failures are grouped by error message, with the number of entities each error affects. The list reflects the Scorecard’s current state, not a specific run. Use the error messages to find the cause, such as a missing integration or an integration that doesn’t have permission to read the data a rule needs. For more help, see Scorecard troubleshooting and FAQ.

​
Evaluation cost

The Evaluation cost tab shows how much work each rule and filter adds when the Scorecard is evaluated. Rules that pull data in real time from third-party integrations generally cost more than rules that use data already stored in Cortex.
The Evaluation cost tab on a Scorecard page.
Each row includes:
  • Rule / filter - The rule or filter name and its CQL expression. If a rule doesn’t have a name, it appears as its CQL expression.
  • Cost - The cost level: Low, Medium, High, or Very high. If Cortex can’t classify a rule, its cost is Unknown.
  • Integrations - The integrations the rule reads data from. Internal means the rule only uses data stored in Cortex.
Rows are sorted from most to least expensive. The Scorecard’s overall cost matches its most expensive rule, and appears under Evaluation cost in the side panel. Hover over the info icon next to it to see how cost is determined. If the overall cost is Unknown, it doesn’t appear in the side panel. To reduce evaluation time, review the most expensive rules first. Where possible, rewrite them to use data that’s already in Cortex. The Evaluation cost tab only appears for users with permission to edit Scorecards, and only when evaluation cost is enabled for your workspace.

​
Reviewing a Scorecard with Cortex MCP

Use Cortex MCP to get quick insights into a Scorecard’s performance. Ask questions in natural language, such as What is going on with my DORA Metrics Scorecard? or What steps can I take to improve my Eng Intelligence Scorecard scores?
First, ask a question about a Scorecard. The MCP client shows which Cortex APIs it’s using to find an answer:An MCP client calling Cortex APIs to answer a question about a Scorecard.Next, it gives you context to help you improve your scores:An MCP response listing priorities for improving a Scorecard.In this example, after listing priorities, the response also suggests quick wins and a systematic approach:An MCP response suggesting quick wins and a systematic approach.
From there, take action to fix the issues and start meeting your standards.

​
Taking action on a Scorecard

Failing rules give teams concrete goals for incremental improvement. Teams can address the highest-impact failures first, track progress over time, and celebrate measurable wins, which reinforces a culture of improvement.

​
Fixing failing rules

1

Identify the failing rules

On the Scorecard’s Scores tab, view the failing rules for each entity. Group by level to see which entities have reached each level.In this example, the entities have reached the Bronze level, but they’re failing rules for Silver and Gold:Entities failing rules in the Silver and Gold levels.
2

Understand the cause

Click an entity to open a side panel that shows why its rules are failing. If the rule has a failure message, it explains how to fix the rule. To add failure messages to your own rules, see Create a Scorecard.A failing rule with its failure message.If the failure message links to a Workflow that fixes the rule, Cortex automatically passes the entity to the Workflow’s entity selector (if it has one), so the entity is selected by default. To set this up manually, append the entity tag or ID to the Workflow URL, for example ?entity={{context.entity.tag}}.When linking to a Workflow from a failed Scorecard rule, Cortex automatically selects the entity to use when running the Workflow.If a rule doesn’t apply to an entity, request an exemption.
3

Track and prioritize issues

Use the Engineering homepage to see a prioritized to-do list of failing rules across the entities and Scorecards you own, so you can focus on the most impactful issues first.A prioritized list of failing rules on the Engineering homepage.To set deadlines for higher-priority work, create an Initiative. Cortex notifies owners and team members as deadlines approach.
4

Fix the issues

Address the underlying issue for each failing rule. This could mean adding missing documentation, configuring an integration, or any other change the rule requires.For example, if a rule fails because PagerDuty isn’t configured for the entity, follow the PagerDuty instructions to add a PagerDuty service, schedule, or escalation policy to the entity’s descriptor YAML.PagerDuty configuration in an entity descriptor YAML file.
5

Re-evaluate the Scorecard

The Scorecard re-evaluates on its schedule, or you can trigger an evaluation manually. After the evaluation finishes, review your progress and any rules that are still failing.A Scorecard showing improved scores after remediation.

​
Improve standards across your organization

As you review evaluations, you might notice trends, such as teams that struggle in specific areas or entities that often miss standards. Beyond fixing individual rules, look for ways to improve the processes behind them.

​
Automate with Workflows

Look for challenges that come up across teams. For example, is a team consistently failing to add runbooks to new services? Are there failing rules related to onboarding? When a standard is often missed, it’s an opportunity to automate that process with a Workflow. Workflows streamline repeatable steps, reduce human error, and help your teams work faster.
A Workflow that scaffolds a new service.For example, you could create a Workflow to standardize how new services are created, with a User Input step that asks users to confirm a runbook has been documented.
When a group of entities is failing rules, look for organizational ways to support your teams or adjust processes. Team patterns
  • Are most failing entities owned by a newer team?
    • Make sure new team members aren’t overwhelmed and have completed any necessary training.
    • Make sure the team understands its scope and priorities.
    • Use Eng Intelligence dashboards and metrics to find which parts of the software development lifecycle each team could improve.
  • Does one team own more entities than others?
    • Make sure work is spread fairly across teams.
    • If one team owns more because of tribal knowledge, encourage team members to document what they know. This can also point to areas where training would help.
    • Once knowledge is documented, link it on related entities.
  • Are some teams or services successful across several areas?
    • Recognize these achievements publicly. Public recognition reinforces good practices and motivates your teams.
Service lifecycle patterns
  • Do older, legacy services have low scores while newer ones meet standards?
    • This points to modernization opportunities.
    • To learn how to use Cortex for modernization, take the Migrations and Modernizations course in Cortex Academy, available to all customers.
Adoption over time
  • Are scores trending upward over the quarter, in line with company initiatives?
    • Scorecard trends over time show whether internal efforts, like a push for better test coverage, are working.
    • To promote specific standards, create an Initiative from a Scorecard to encourage teams to fix failing rules by a deadline.
Last modified on October 12, 2026