Skip to main content

Scorecards

Scorecards allow your team to define standards like production readiness and development quality, and enforce them without building scripts and maintaining spreadsheets.

UI

Scorecards can be managed through the UI by default, see Help Desk | Creating and Editing Scorecards for a detailed walkthrough.

GitOps

If you would like to manage Scorecards through your GitOps workflow you can disable UI editing through Settings | App Preferences | Enable Scorecard UI Editor.

enable-scorecard-editor

In general, the best place to put Scorecards is in their own repository, separate from catalog entities, at the repository's root directory within .cortex/scorecards. Note that it is not recommended to put Scorecard definitions in a service repository as Scorecards are not meant to be 1:1 with catalog entitites. For example, a simple repository might have the structure:

.
├── .cortex
│    └── scorecards
│        ├── dora.yml
│        └── performance.yml
└── src
    └── index.js
    └── ...

Any file found within the .cortex/scorecards directory will be automatically picked up and parsed as a Scorecard.

Descriptor

The dora.yml descriptor file might look something like this:

---
name: DORA Metrics
tag: dora-metrics
description: >-
  [DORA metrics](https://www.cortex.io/post/understanding-dora-metrics) are used by DevOps teams to measure their performance.

  The 4 key metrics are Lead Time for Changes, Deployment Frequency, Mean Time to Recovery, and Change Failure Rate.
draft: false
ladder:
  levels:
    - name: Gold
      rank: 1
      description: Excellent
      color: "#cda400"
    - name: Silver
      rank: 2
      description: Very good
      color: "#8c9298"
    - name: Bronze
      rank: 3
      description: Pretty good
      color: "#c38b5f"
rules:
  - title: Ratio of rollbacks to deploys in the last 7 days
    expression: >+
      (deploys(lookback=duration("P7D"),types=["ROLLBACK"]).count /
      deploys(lookback=duration("P7D"),types=["DEPLOY"]).count) > 0.05
    description: Change Failure Rate
    weight: 25
    failureMessage: Less than 95% of deployments in the last 7 days were successful
    level: Gold
  - title: Incident was ack'ed within 5 minutes
    expression: oncall.analysis(lookback = duration("P7D")).meanSecondsToFirstAck <= 300
    description: MTTA (Mean time to acknowledge)
    weight: 25
    failureMessage: Incidents in the last 7 days were not ack'ed
    level: Silver
  - title: Last commit was within 24 hours
    expression: git.lastCommit.freshness <= duration("P1D")
    description: Lead Time for Changes
    weight: 25
    failureMessage: No commits in the last 24 hours
    level: Bronze
  - title: Averaging at least one deploy a day in the last 7 days
    expression: deploys(lookback=duration("P7D"),types=["DEPLOY"]).count >= 7
    description: Deployment Frequency
    weight: 25
    failureMessage: No deployments in the last 7 days
filter:
  category: SERVICE
  query: has_group("production")

Objects

Scorecard: {
  name: String
  tag: String,
  description: String?,
  draft: Boolean?,
  ladder: Ladder?,
  rules: List<Rule>,
  filter: Filter?
}
namedescription
nameThe human-readable name of the Scorecard
tagA unique slug for the Scorecard consisting of only alphanumeric characters and dashes
descriptionA human-readable description of the Scorecard
draftWhether or not the Scorecard is a draft
ladderThe Ladder to apply to the Rules
rulesA list of Rules that are evaulated each time the Scorecard is evaluated
filterEnables the ability to exclude entities from being evaluated by this Scorecard
Ladder: {
  levels: List<Level>
}
namedescription
levelsThe levels of the Ladder
Level: {
  name: String,
  rank: Int,
  description: String?,
  color: String
}
namedescription
nameThe human-readable name of the Level
rankThe rank of the Level within the Ladder
descriptionA human-readable description of the Level
colorThe hex color (without the pound sign) of the badge that is displayed with the Level
Rule: {
  title: String?,
  expression: String,
  weight: Int,
  failureMessage: String?
  level: String?
}
namedescription
titleThe human-readable name of the Rule
expressionThe CQL expression to evauluate; must evalutate to a boolean
weightThe number of points this Rule provides when successful
failureMessageA human-readable message that will be presented when the Rule is failing
levelThe name of the Level this Rule is associated with; can be null even when a Ladder is present
Filter: {
  category: String?,
  query: String?
}
namedescription
categoryBy default, Scorecards are evaluated against all Services. To evaluate a Scorecard against Resources you can specify the category as RESOURCE
queryA CQL query that is run against the category; only entities matching this query will be evaluated by the Scorecard