Scorecards
Scorecards allow your team to define standards like production readiness and development quality, and enforce them without building scripts and maintaining spreadsheets.
UI
Scorecards can be managed through the UI by default, see Help Desk | Creating and Editing Scorecards for a detailed walkthrough.
GitOps
If you would like to manage Scorecards through your GitOps workflow you can disable UI editing through Settings | Workspace | Scorecards | UI Editor.
In general, the best place to put Scorecards is in their own repository, separate from catalog entities, at the repository's root directory within .cortex/scorecards
. Note that it is not recommended to put Scorecard definitions in a service repository as Scorecards are not meant to be 1:1 with catalog entitites. For example, a simple repository might have the structure:
.
├── .cortex
│ └── scorecards
│ ├── dora.yml
│ └── performance.yml
└── src
└── index.js
└── ...
Any file found within the .cortex/scorecards
directory will be automatically picked up and parsed as a Scorecard.
Descriptor
The dora-metrics-scorecard.yaml
descriptor file might look something like this:
name: DORA Metrics
tag: dora-metrics
description: >-
[DORA metrics](https://www.cortex.io/post/understanding-dora-metrics) are used by DevOps teams to measure their performance.
The 4 key metrics are Lead Time for Changes, Deployment Frequency, Mean Time to Recovery, and Change Failure Rate.
draft: false
notifications:
enabled: true
exemptions:
enabled: true
autoApprove: false
ladder:
levels:
- name: Bronze
rank: 1
description: Pretty good
color: "#c38b5f"
- name: Silver
rank: 2
description: Very good
color: "#8c9298"
- name: Gold
rank: 3
description: Excellent
color: "#cda400"
rules:
- title: Ratio of rollbacks to deploys in the last 7 days
expression: >+
(deploys(lookback=duration("P7D"),types=["ROLLBACK"]).length /
deploys(lookback=duration("P7D"),types=["DEPLOY"]).length) > 0.05
identifier: 3c42fa96-b422-30a4-b75a-f8b1cc233408
description: Change Failure Rate
weight: 25
failureMessage: Less than 95% of deployments in the last 7 days were successful
level: Gold
- title: Incident was ack'ed within 5 minutes
expression: oncall.analysis(lookback = duration("P7D")).meanSecondsToFirstAck <= 300
identifier: 8713f2c0-f161-3688-9f99-bcfaab476b63
description: MTTA (Mean time to acknowledge)
weight: 25
failureMessage: Incidents in the last 7 days were not ack'ed
level: Silver
- title: Last commit was within 24 hours
expression: git.lastCommit().freshness <= duration("P1D")
identifier: efbd8c51-7643-33cc-8fe7-1b46b2765dc9
description: Lead Time for Changes
weight: 25
failureMessage: No commits in the last 24 hours
level: Bronze
- title: Averaging at least one deploy a day in the last 7 days
expression: deploys(lookback=duration("P7D"),types=["DEPLOY"]).length >= 7
identifier: a16b7eeb-545b-359e-81a7-3946baacdd4b
description: Deployment Frequency
weight: 25
failureMessage: No deployments in the last 7 days
filter:
kind: GENERIC
types:
include:
- service
groups:
include:
- production
evaluation:
window: 4
Objects
Scorecard: {
name: String
tag: String,
description: String?,
draft: Boolean?,
notifications: Notifications?,
exemptions: Exemptions?,
ladder: Ladder?,
rules: List<Rule>,
filter: Filter?
evaluation: Evaluation?
}
name | description |
---|---|
name | The human-readable name of the Scorecard |
tag | A unique slug for the Scorecard consisting of only alphanumeric characters and dashes |
description | A human-readable description of the Scorecard |
draft | Whether or not the Scorecard is a draft |
notifications | Notifications settings for the Scorecard |
ladder | The ladder to apply to the rules |
rules | A list of rules that are evaluated each time the Scorecard is evaluated |
filter | Enables the ability to exclude entities from being evaluated by this Scorecard |
evaluation | Enables the ability to change the evaluation window for this Scorecard |
Notifications: {
enabled: Boolean
}
name | description |
---|---|
enabled | Whether or not to include the Scorecard in notifications |
Exemptions: {
enabled: Boolean?,
autoApprove: Boolean?
}
name | description |
---|---|
enabled | Whether or not rule exemptions are enabled for Scorecard |
autoApprove | Whether or not rule exemptions are auto approved for Scorecard |
Ladder: {
levels: List<Level>
}
name | description |
---|---|
levels | The levels of the ladder |
Level: {
name: String,
rank: Int,
description: String?,
color: String
}
name | description |
---|---|
name | The human-readable name of the level |
rank | The rank of the Level within the ladder. Higher rank is better. |
description | A human-readable description of the level |
color | The hex color of the badge that is displayed with the level |
Rule: {
title: String?,
expression: String,
identifier: String?,
description, String?,
weight: Int,
failureMessage: String?
level: String?,
filter: Filter?
effectiveFrom: Date?
}
name | description |
---|---|
title | The human-readable name of the Rule |
expression | The CQL expression to evaluate; must evaluate to a boolean |
identifier | Identifier of the rule, unique within Scorecard scope. |
description | A human-readable description of the Rule |
weight | The number of points this Rule provides when successful |
failureMessage | A human-readable message that will be presented when the Rule is failing |
level | The name of the level this rule is associated with; can be null even when a ladder is present |
filter | Enables the ability to exclude entities from being evaluated for this rule |
effectiveFrom | Date when the rule starts being evaluated (e.g. 2024-01-01T00:00:00Z) |
Filter: {
kind: String
types: TypesFilter?,
groups: GroupsFilter?,
query: String?
}
Note: One of types, groups, query must be present for it to be considered a valid filter
name | description |
---|---|
kind | The kind of filter to create. Currently only supports "GENERIC" |
types | Types filter (to include / exclude specific types) |
groups | Groups filter (to include / exclude specific groups) |
query | A CQL query; only entities matching this query will be evaluated by the Scorecard |
TypesFilter: {
include: List<String>?,
exclude: List<String>?
}
Note: Only one of include/exclude can be specified at a time
name | description |
---|---|
include | List of types to include in set of entities |
exclude | List of types to exclude in the set of entities |
GroupsFilter: {
include: List<String>?,
exclude: List<String>?
}
name | description |
---|---|
include | List of groups to include in set of entities |
exclude | List of groups to exclude in the set of entities |
Evaluation: {
window: Int?
}
name | description |
---|---|
window | By default, Scorecards are evaluated every 4 hours. That default can be changed under Settings > Scorecards > Default evaluation window. If you would like to evaluate Scorecards less frequently, you can override the evaluation window. This can help if you're encountering problems with rate limits. Note that Scorecards cannot be evaluated more than once per minimum set in Settings > Scorecards > Minimum evaluation window. |