> For the complete documentation index, see [llms.txt](https://docs.cortex.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cortex.io/ingesting-data-into-cortex/integrations/pagerduty/using-the-integration-for-pagerduty.md).

# Using the integration for PagerDuty

How to use the integration for PagerDuty in Cortex

This article explains how to use the integration for PagerDuty. For configuration instructions, see [Configuring the integration for PagerDuty](/ingesting-data-into-cortex/integrations/pagerduty.md#configuring-pagerduty). For instructions on connecting PagerDuty to entities, see [Connecting entities to PagerDuty](/ingesting-data-into-cortex/integrations/pagerduty/connecting-entities-to-pagerduty.md).

After you configure the PagerDuty integration, Cortex surfaces on-call and incident data across the app on:

### Entity pages

View current on-call information in the **On-call** block on an entity's metadata sidebar, and on the entity's details page under **Connections > On-call**.

<div align="left" data-with-frame="true"><figure><img src="https://826863033-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJW7pYRxS4dHS3Hv6wxve%2Fuploads%2FCLPuacBh4amB474qJHiY%2Fpagerduty-entity.png?alt=media&amp;token=c3a45097-162e-46c4-9b25-6848d3cd47e6" alt="On-call information on an entity&#x27;s details page." width="375"><figcaption></figcaption></figure></div>

Select the **Events** tab in an entity's left side panel to view recent events pulled in from PagerDuty.

<div align="left" data-with-frame="true"><figure><img src="https://826863033-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJW7pYRxS4dHS3Hv6wxve%2Fuploads%2FJoVAnpwktv9eNpLQOfmm%2Fpagerduty-events.png?alt=media&amp;token=c8fe9f98-18ed-4cd0-b6b7-d7009f821fd5" alt="The Events page." width="375"><figcaption></figcaption></figure></div>

{% hint style="info" %}
Escalation policy and service details are hyperlinked to the corresponding pages in your PagerDuty instance.
{% endhint %}

### Engineering homepage

The PagerDuty integration allows Cortex to pull on-call information into the **My on-calls** block on the Engineering homepage. On-call data from PagerDuty is refreshed every 60 minutes.

<div align="left" data-with-frame="true"><figure><img src="https://826863033-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJW7pYRxS4dHS3Hv6wxve%2Fuploads%2FC1odz3EoR0827Amk4Emv%2Fpagerduty-homepage.png?alt=media&amp;token=1bf4e05d-0316-4446-a22e-10d1e410e541" alt="The &#x27;My on-calls&#x27; section of the Engineering homepage." width="375"><figcaption></figcaption></figure></div>

### Eng Intelligence

Cortex also pulls in metrics from PagerDuty for Eng Intelligence, which displays MTTR, incidents opened, and incidents opened per week.

## Other ways to use the integration for PagerDuty

### Retrieving on-call information in Slack

If you have the Slack integration set up, you can ask the AI Assistant for current on-call information. Mention `@Cortex` in a channel or direct message and ask a question like "Who's on call for payments-api?" This works for both services and teams with registered PagerDuty schedules or escalation policies.

For setup and usage details, refer to [Using the AI assistant](/ingesting-data-into-cortex/integrations/slack/using-the-integration-for-slack-ai-assistant.md#using-the-ai-assistant).

### Creating Scorecard rules and writing CQL queries with the PagerDuty integration

With the PagerDuty integration, you can create Scorecard rules and write CQL queries based on incidents, escalations, and on-call metadata.

See more examples in the [CQL Explorer](https://app.getcortexapp.com/admin/cql-explorer) in Cortex.

<details>

<summary>Check if on-call is set</summary>

Check if entity has a registered service, schedule, or escalation policy. If the service does not have any registrations in its entity descriptor, Cortex searches for PagerDuty services matching the tag defined in the entity's `x-cortex-tag` field.

**Definition** - `oncall (==/!=) null`

**Example**

For a Scorecard focused an production readiness, you can use this expression to make sure on-call is defined for entities:

```
oncall != null
```

This rule passes if an entity has a service, schedule, or escalation policy set.

</details>

<details>

<summary>Forbidden contact methods</summary>

Number of users in each entity's escalation policy with missing or forbidden contact methods.

Allowed contact methods:

* "SMS"
* "PHONE"
* "EMAIL"
* "PUSH\_NOTIFICATION"
* "SLACK"

**Definition** - `oncall.usersWithoutContactMethods(allowed=<allowed>, onlyCurrentOncall=<boolean>).length`

**Example**

For a Scorecard focused on ownership, you can use this expression to make sure users have required contact methods enabled:

```
oncall.usersWithoutContactMethods(allowed=["SMS", "PHONE"]).length == 0
```

This rule passes if every user in an associated escalation policy has either SMS or phone calls enabled as their contact method.

You can also use this expression in the query builder to find users that lack the required contact method:

```
oncall.usersWithoutContactMethods(allowed=["EMAIL"]) > 0
```

This query surfaces users without email addresses.

If you want to check only current on-call users, you can use the `onlyCurrentOncall` parameter:

```
oncall.usersWithoutContactMethods(allowed=["EMAIL"], onlyCurrentOncall=true) > 0
```

When this parameter is set to `false` or omitted, the expression checks all users in the associated escalation policy for the next 3 months.

</details>

<details>

<summary>Incident response analysis</summary>

Get detailed [on-call analysis stats](https://developer.pagerduty.com/api-reference/694e92fe4f943-get-aggregated-service-data) for each entity:

* Mean assignment count
* Mean engaged seconds
* Mean engaged user count
* Mean seconds to engage
* Mean seconds to first ack
* Mean seconds to mobilize
* Mean seconds to resolve
* Total business-hour erruptions
* Total engaged seconds
* Total escalation count
* Total off-hour erruptions
* Total sleep-hour erruptions
* Total snoozed seconds
* Total incident count
* Up time percent

PagerDuty updates its analytics data once per day, and it can take up to 24 hours before new incidents appear in the analytics API.

**Note that this only works if the entity has a registered PagerDuty service ID or if the PagerDuty service name matches the entity tag.**

**Definition** - `oncall.analysis(lookback = <duration>, priority = <List<String>>)`

**Examples**

PagerDuty analytics can easily be used to craft rules for a DORA metrics Scorecard.

For **mean time to acknowledge**, you can use the `meanSecondsToFirstAck` schema definition:

```
oncall.analysis(lookback = duration("P7D"), priority = ["P1", "P2"]).meanSecondsToFirstAck <= 300
```

Entities pass this rule if incidents in the last week were acknowledged within 5 minutes.

For **mean time to resolve**, you can use `meanSecondsToResolve` to make sure that incidents were handled within an hour:

```
oncall.analysis(lookback = duration("P7D"), priority = ["P1"]).meanSecondsToResolve < 3600
```

You can also use this expression to write a rule that checks an entity's change failure rate:

```
oncall.analysis(lookback = duration("P7D")).totalIncidentCount == 0
```

This rule passes if there weren't any incidents in the last week.

</details>

<details>

<summary>Incidents</summary>

Get incident data for each entity:

* Assignee ID
* Created at
* Incident ID
* Last updated
* Resolved at
* Service ID
* Status

**Note that this only works if the entity has a registered PagerDuty service ID or if the PagerDuty service name matches the entity tag.**

**Definition** - `oncall.incidents(lookback = <duration>)`

**Examples**

For a Scorecard focused on service maturity or quality, you can use this expression to check the number of incidents opened in the last month:

```
oncall.incidents(lookback = duration("P1M")).length < 15
```

Entities pass this rule if they have fewer than 15 incidents opened in the last month.

You can also use this expression to make sure there aren't incidents that remained open over the last month:

```
oncall.incidents(lookback=duration("P1M")).filter((incident) => incident.status.matches("TRIGGERED|ACKNOWLEDGED")).length < 1
```

Or you can check for incidents that took a certain amount of time to resolve:

```
oncall.incidents(lookback=duration("P1M")).filter((incident) => incident.createdAt.until(incident.resolvedAt) > duration("P-2D")).length < 2
```

Entities pass this rule if there were 0 or 1 incidents in the last month that took more than 2 days to resolve.

</details>

<details>

<summary>Number of escalations</summary>

Number of escalation tiers in escalation policy.

**Definition** - `oncall.numOfEscalations()`

**Example**

This expression could be used in a Scorecard focused on production readiness or service maturity:

```
oncall.numOfEscalations() >= 2
```

This rule checks that there are at least two tiers in an escalation policy for a given entity, so that if the first on-call does not ack, there is a backup.

While making sure an on-call policy set is a rule that would be defined in a Scorecard's first level, a rule focused on escalation tiers would make more sense in a higher level.

</details>

<details>

<summary>On-call metadata</summary>

On-call metadata, including type, id, and name.

**Definition** - `oncall.details()`

**Examples**

To find all entities with a schedule-type on-call registration, you can use this expression in the Query builder:

```
oncall.details().type == "schedule"
```

If you're migrating on-call policies, you could use this rule to check for outdated policies. For example, assume that all outdated PagerDuty policies start with "Legacy" in their titles.

```
oncall.details().id.matches("Legacy*") == false
```

Entities with on-call policies that start with "Legacy" fail, while those with other policy names pass.

</details>

## Triggering an incident

A given entity can have a PagerDuty service, schedule, or escalation policy defined, as described in [Connecting via an entity descriptor](/ingesting-data-into-cortex/integrations/pagerduty/connecting-entities-to-pagerduty.md#connecting-via-an-entity-descriptor). Only entities with a PagerDuty service defined include the option to trigger an incident directly from Cortex.

Your PagerDuty API key must include the `Write` permission to trigger incidents from an entity.

**To trigger an incident in PagerDuty**:

1. In Cortex, navigate to an entity.&#x20;
2. From the entity's left sidebar, select **Incidents**.<br>

   <div align="left" data-with-frame="true"><figure><img src="https://826863033-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJW7pYRxS4dHS3Hv6wxve%2Fuploads%2F2iN19Zns7Djw8pYrnW03%2Fpagerduty-incidents.png?alt=media&amp;token=fbdfe090-0651-4c2f-80c7-ce14a8c99a83" alt="The &#x27;Incidents&#x27; tab on the entity&#x27;s left sidebar." width="375"><figcaption></figcaption></figure></div>
3. In the upper-right corner of the **Incidents** page, click **Trigger incident**. The **Trigger incident** side panel opens.
4. Do the following:
   1. Under **Title**, enter a name for the incident (required).&#x20;
   2. Under **Description**, enter a description of the incident.
   3. From the **Urgency** dropdown, select a severity level.
5. Click **Trigger incident**.&#x20;
6. Click **Click here** to view the incident in PagerDuty.
7. Click **Close** to collapse the **Trigger incident** side panel.

## Viewing PagerDuty integration logs

{% hint style="info" %}
This feature is available in Cortex cloud.
{% endhint %}

While viewing an integration's settings page, select the **Logs** tab to view error logs from the last 7 days. You can filter the logs list by configuration and by operation (for example, you could filter to view errors surfaced only via Scorecards).

<div align="left" data-with-frame="true"><figure><img src="https://826863033-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJW7pYRxS4dHS3Hv6wxve%2Fuploads%2Fgit-blob-9fe1dcaae2c411740363a23c6996ee921ecab075%2Fintegrations-logs-tab-generic.png?alt=media" alt="The &#x27;Logs&#x27; tab on an integration&#x27;s settings page shows error information over the past 7 days." width="563"><figcaption></figcaption></figure></div>

Click into a row to get more information, including time stamp, status code, full error, and request path.
