Scorecards as code

Once Scorecards are incorporated into your regular engineering workflows, they become production-grade tools. Scorecards set the benchmarks for how your organization defines “good,” which impacts everything from security standards to on-call rotations. Because Scorecards are crucial to achieving and maintaining standards, you may want to employ controlled access and version controlling. Turn on GitOps editing for Scorecards to manage your standards the same way you manage the entities in your catalogs. Each Scorecard gets its own YAML file, and every change shows up in your GitOps logs.

Setting up Scorecards as code

Step 1: Configuring GitOps editing for Scorecards

From the main sidebar, click your avatar in the bottom-left corner.
Select Settings.
Locate the Workspace section, then select GitOps.
On the GitOps page, select the Scorecards tab.
Toggle on Enable GitOps for Scorecard editing.

When GitOps for Scorecard editing is enabled:

You can’t edit Scorecards in the Cortex UI, including ones you originally created there. Edits happen in the Scorecard’s YAML file in your Git repository.
You can still create and delete Scorecards in the UI.

You can also configure an allowlist to control which repositories Cortex imports Scorecards from.

Step 2: Adding Scorecards to your Git repository

You can build a Scorecard in the Cortex UI and convert it to a YAML, or write the YAML from scratch. See the example below. Keep Scorecards in their own repository, separate from your catalog entities, under .cortex/scorecards at the repository root. Avoid putting Scorecard definitions in a service repository. A Scorecard measures many entities, so it doesn’t belong to any single one of them. For example, a simple repository might have the following structure:

.
├── .cortex
│    └── scorecards
│        ├── dora.yml
│        └── performance.yml
└── src
    └── index.js
    └── ...

Any file found within the .cortex/scorecards directory, including its subdirectories, is automatically picked up and parsed as a Scorecard. When GitOps editing is turned on for Scorecards, you can’t edit any Scorecard in the Cortex UI, including Scorecards you originally created there. You can still create and delete Scorecards in the UI. For more information, refer to Using both the UI and GitOps.

Converting an existing Scorecard to code

You can convert Scorecards created in the Cortex UI to code via GitOps. Keep in mind, though, that once GitOps editing is turned on and your Scorecard lives in a YAML file, you can only edit it via Git. To convert an existing Scorecard into code:

From the main sidebar, expand Scorecards.
Select Scorecards.
On the Scorecards page, select the Scorecard you want to convert.
At the top of the page, click the overflow menu icon, then click Export Scorecard YAML.
Add the Scorecard’s YAML file to your Git repository.

Example Scorecard YAML file

The dora-metrics.yaml descriptor file might look something like this:

name: DORA Metrics
tag: dora-metrics
description: >-
  [DORA metrics](https://www.cortex.io/post/understanding-dora-metrics) are used
  by DevOps teams to measure their performance.

  This Scorecard covers deployment frequency, change failure rate, and time to
  restore service.
draft: false
notifications:
  enabled: true
  scoreDropNotificationsEnabled: false
exemptions:
  enabled: true
  autoApprove: false
  userSpecificNotifications: false
ladder:
  levels:
    - name: Bronze
      rank: 1
      description: Baseline delivery practices are in place
      color: "#c38b5f"
    - name: Silver
      rank: 2
      description: Delivery is consistent and incidents are handled promptly
      color: "#8c9298"
    - name: Gold
      rank: 3
      description: Delivery performance meets DORA elite benchmarks
      color: "#cda400"
rules:
  - title: Deploys at least twice a month
    expression: deploys(lookback=duration("P30D"), types=["DEPLOY"]).length >= 2
    identifier: deployment-frequency-baseline
    description: Deployment frequency
    weight: 25
    failureMessage: >-
      This entity deployed fewer than 2 times in the last 30 days. Frequent,
      small deploys reduce the risk of each individual change.
    level: Bronze
  - title: Change failure rate is 30% or less
    expression: >-
      deploys(lookback=duration("P30D"), types=["ROLLBACK"]).length <=
      deploys(lookback=duration("P30D"), types=["DEPLOY"]).length * 0.3
    identifier: change-failure-rate-baseline
    description: Change failure rate
    weight: 25
    failureMessage: >-
      More than 30% of deploys in the last 30 days were rolled back. Check
      test coverage and pre-production validation for this entity.
    level: Bronze
  - title: Time to restore service is under 4 hours
    expression: oncall.analysis(lookback=duration("P30D")).meanSecondsToResolve <= 14400
    identifier: mttr-high
    description: Time to restore service
    weight: 25
    failureMessage: >-
      Incidents took longer than 4 hours to resolve on average. Confirm this
      entity has a current runbook and a working escalation policy.
    level: Silver
  - title: Deploys at least 30 times a month
    expression: deploys(lookback=duration("P30D"), types=["DEPLOY"]).length >= 30
    identifier: deployment-frequency-elite
    description: Deployment frequency
    weight: 25
    failureMessage: >-
      Elite performers deploy on demand, roughly daily or more. Look for manual
      approval steps or batched releases holding this entity back.
    level: Gold
  - title: Time to restore service is under 1 hour
    expression: oncall.analysis(lookback=duration("P30D")).meanSecondsToResolve <= 3600
    identifier: mttr-elite
    description: Time to restore service
    weight: 25
    failureMessage: >-
      Incidents took longer than 1 hour to resolve on average. Automated
      rollback is usually the fastest way to close this gap.
    level: Gold
  # Rules without a level are scored on points alone and do not gate progress
  # through the ladder. Use this for measures you want to track and reward
  # without blocking a level on them.
  - title: Incidents are acknowledged within 5 minutes
    expression: oncall.analysis(lookback=duration("P30D")).meanSecondsToFirstAck <= 300
    identifier: mtta-bonus
    description: Time to acknowledge, tracked alongside the DORA metrics
    weight: 10
    failureMessage: >-
      Incidents took longer than 5 minutes to acknowledge on average. Check
      that this entity's escalation policy has a second level.
    # Rule-level filters scope a rule to a subset of the Scorecard's entities.
    filter:
      kind: GENERIC
      groups:
        include:
          - customer-facing
    # Scheduled rules are visible to teams before they start counting.
    effectiveFrom: "2026-10-01T00:00:00Z"
filter:
  kind: GENERIC
  types:
    include:
      - service
  groups:
    include:
      - production
evaluation:
  window: 4

Scorecard YAML reference

Scorecard objects

Name	Description
`name`	The human-readable name of the Scorecard
`tag`	A unique slug for the Scorecard consisting of only alphanumeric characters and dashes
`description`	A human-readable description of the Scorecard
`draft`	Whether or not the Scorecard is a draft
`notifications`	Notifications settings for the Scorecard
`ladder`	The ladder to apply to the rules
`rules`	A list of rules that are evaluated each time the Scorecard is evaluated
`filter`	Enables the ability to exclude entities from being evaluated by this Scorecard
`evaluation`	Enables the ability to change the evaluation window for this Scorecard

Notifications

Name	Description
`enabled`	Whether or not to include the Scorecard in notifications
`scoreDropNotificationsEnabled`	Whether to notify about entities’ scores that dropped after each evaluation of this Scorecard

Exemptions

Name	Description
`enabled`	Whether or not rule exemptions are enabled for Scorecard
`autoApprove`	Whether or not rule exemptions are auto approved for Scorecard
`userSpecificNotifications`	Whether user-specific rule exemption notifications are enabled for Scorecard

Ladder

Name	Description
`levels`	The levels of the ladder

Level

Name	Description
`name`	The human-readable name of the level
`rank`	The rank of the level within the ladder. Higher rank is better.
`description`	A human-readable description of the level
`color`	The hex color of the badge that is displayed with the level

Rules

Name	Description
`title`	The human-readable name of the rule
`expression`	The CQL expression to evaluate; must evaluate to a boolean
`identifier`	Identifier of the rule, unique within Scorecard scope. Can be configured to a custom value of your choice. If omitted, we will automatically generate a value.
`description`	A human-readable description of the rule
`weight`	The number of points this rule provides when successful
`failureMessage`	A human-readable message that will be presented when the rule is failing
`level`	The name of the level this rule is associated with; can be null even when a ladder is present
`filter`	Enables the ability to exclude entities from being evaluated for this rule
`effectiveFrom`	Date when the rule starts being evaluated, e.g. 2024-01-01T00:00:00Z

Filter

Note: One of types, groups, or query must be present for it to be considered a valid filter.

Name	Description
`kind`	The kind of filter to create. Currently only supports “GENERIC”.
`types`	Types filter (to include / exclude specific types)
`groups`	Groups filter (to include / exclude specific groups)
`query`	A CQL query; only entities matching this query will be evaluated by the Scorecard

TypesFilter

Note: Only one of include or exclude can be specified at a time.

Name	Description
`include`	List of types to include in set of entities
`exclude`	List of types to exclude in the set of entities

GroupsFilter

Name	Description
`include`	List of groups to include in set of entities
`exclude`	List of groups to exclude in the set of entities

Evaluation

Name	Description
`window`	By default, Scorecards are evaluated every four (4) hours. This can be changed under Settings > Scorecards > Default evaluation window. To evaluate Scorecards less frequently, you can override the evaluation window. This helps if you’re encountering problems with rate limits. Note that Scorecards cannot be evaluated more than once per minimum set in Settings > Scorecards > Minimum evaluation window.

Cortex

Get Started

Connect data

Standardize

Improve

Streamline

Configure

Resources

Scorecards as code

Setting up Scorecards as code

Step 1: Configuring GitOps editing for Scorecards

Step 2: Adding Scorecards to your Git repository

Converting an existing Scorecard to code

Example Scorecard YAML file

Scorecard YAML reference

Scorecard objects

Notifications

Exemptions

Ladder

Level

Rules

Filter

TypesFilter

GroupsFilter

Evaluation

​Setting up Scorecards as code

​Step 1: Configuring GitOps editing for Scorecards

​Step 2: Adding Scorecards to your Git repository

​Converting an existing Scorecard to code

​Example Scorecard YAML file

​Scorecard YAML reference

​Scorecard objects

​Notifications

​Exemptions

​Ladder

​Level

​Rules

​Filter

​TypesFilter

​GroupsFilter

​Evaluation

Setting up Scorecards as code

Step 1: Configuring GitOps editing for Scorecards

Step 2: Adding Scorecards to your Git repository

Converting an existing Scorecard to code

Example Scorecard YAML file

Scorecard YAML reference

Scorecard objects

Notifications

Exemptions

Ladder

Level

Rules

Filter

TypesFilter

GroupsFilter

Evaluation