LogoLogo
Login to CortexBook a DemoCortex Academycortex.io
  • Cortex Docs
  • Cortex Quick Start
  • Ingesting data into Cortex
    • Managing Entities
      • Adding entities
        • Add services
        • Add domains
        • Add teams
        • Add custom entity types
        • Defining dependencies
      • Entity details page
      • Defining ownership
      • Defining relationship types
      • Grouping entities
      • Adding external documentation
      • Adding Deploy data
      • Adding custom data
      • Viewing discovered entities
      • Archiving entities
      • Relationship graph
      • Using On-call Assistant for incidents
      • Managing Terraform infra in Cortex
    • Managing Catalogs
    • Integrations
      • Internally hosted integrations
      • ArgoCD
      • AWS
      • Azure DevOps
      • Azure Resources
      • BambooHR
      • Bitbucket
      • BugSnag
      • Buildkite
      • Checkmarx
      • CircleCI
      • ClickUp
      • Codecov
      • Coralogix
      • Custom webhook integrations
      • Datadog
      • Dynatrace
      • Entra ID (Azure AD)
      • FireHydrant
      • GitHub
      • GitLab
      • Google
      • Grafana
      • incident.io
      • Instana
      • Jenkins
      • Jira
      • Kubernetes
      • LaunchDarkly
      • Lightstep
      • Mend
      • Microsoft Teams
      • New Relic
      • Okta
      • Opsgenie
      • PagerDuty
      • Prometheus
      • Rollbar
      • Rootly
      • Sentry
      • ServiceNow
      • Slack
      • Snyk
      • SonarQube
      • Splunk Observability Cloud (SignalFx)
      • Splunk On-Call (VictorOps)
      • Sumo Logic
      • Veracode
      • Wiz
      • Workday
      • xMatters
  • Scorecards
    • Initiatives and Action items
      • Creating issues based on Initiatives
    • Scorecard rule exemptions
    • Scorecard rule filters
    • Scorecard examples
    • Scorecards as code
  • Reports
    • Executive report
    • All Scorecards report
    • Bird's eye report
    • Progress report
    • Report card
  • Eng Intelligence
    • Custom Metrics
    • Jira Metrics
    • Metrics Explorer (Beta)
  • Cortex Query Language (CQL)
    • Using CQL reports
    • Using JQ in Cortex
  • Workflows
    • Creating a Workflow
      • Workflows as code
    • Blocks
    • Running a Workflow
    • Registering a Scaffolder template
      • Scaffolder advanced usage
    • Using a Workflow to sync in ArgoCD
    • Kicking off a Jenkins pipeline in a Workflow
    • Calling internal service endpoints in a Workflow
  • Plugins
    • Creating a plugin
      • Creating a plugin proxy
    • Migrating Backstage plugins to Cortex
  • Engineering homepage
  • Workspace Settings
    • Using GitOps for Cortex
      • GitOps logs
    • Managing users
      • Roles and permissions
        • Custom roles
        • Team ownership entity editing
      • Configuring SSO
        • Microsoft Entra ID
        • Google
        • Other OIDC providers
        • Okta
          • Okta SCIM
      • Configuring identity mappings
      • Onboarding management
    • API keys, secrets, and tokens
      • Secrets
      • Personal tokens
    • Audit logs
    • Entity settings
      • Data verification
      • Auto archiving entities
    • IP allowlist
    • Notifications
      • Notification logs
    • Customizing your workspace
    • Using search in Cortex
  • Cortex API
    • REST API operations
      • API Keys
      • Audit Logs
      • Catalog Entities
      • Custom Data
        • Custom Data (Advanced)
      • Custom Events
      • Custom Metrics
      • Dependencies
      • Deploys
      • Discovery Audit
      • Docs
      • Eng Intel: User Labels
      • Entity Relationship Types (Beta)
      • Entity Relationships (Beta)
      • Entity Types
      • GitOps Logs
      • Groups
      • Initiatives
      • Integrations APIs
        • Azure Active Directory (Entra ID) API
        • Azure Resources API
        • AWS API
        • Azure DevOps API
        • CircleCI API
        • Coralogix API
        • Datadog API
        • GitHub API
        • GitLab API
        • incident.io API
        • LaunchDarkly API
        • New Relic API
        • PagerDuty API
        • Prometheus API
        • SonarQube API
      • IP Allowlist
      • Notification Logs
      • On call
      • Packages
      • Plugins
      • Queries
      • SCIM
      • Scorecards
      • Secrets
      • Team Hierarchies
      • Teams
      • Workflows
Powered by GitBook
On this page
  • How to configure Splunk On-Call with Cortex
  • Prerequisites
  • Configure the integration in Cortex
  • How to connect Cortex entities to Splunk On-Call
  • Editing the entity descriptor
  • Expected results
  • Scorecards and CQL
  • Still need help?​

Was this helpful?

Export as PDF
  1. Ingesting data into Cortex
  2. Integrations

Splunk On-Call (VictorOps)

Last updated 1 month ago

Was this helpful?

Splunk On-Call (formerly known as ) is an alert and on-call management platform.

Integrating Cortex with Splunk On-Call allows you to:

  • Pull in on-call rotation data and escalation policies

    • The on-call user or team will appear in the Current On-call block on an entity's details page.

    • You can also view on-call information on an entity page in its side panel under Integrations > On-call.

  • Create that track progress and drive alignment on projects involving your on-call schedule

How to configure Splunk On-Call with Cortex

Prerequisites

Before getting started:

  • Create a .

    • Note: If the key is granted Read-only permissions, Cortex will only perform GET requests.

  • Obtain your Splunk API ID.

    • In your Splunk On-Call portal, navigate to the Integrations page then click the API tab. Your API ID is displayed above your API keys.

Configure the integration in Cortex

  1. In Cortex, navigate to the :

    1. In Cortex, click your avatar in the lower left corner, then click Settings.

    2. Under "Integrations," click VictorOps.

  2. Click Add configuration.

  3. Configure the VictorOps integration form:

    • API ID: Enter your API ID from Splunk On-Call.

    • API key: Enter your API key from Splunk On-Call.

    • Organization slug: Enter your Splunk On-Call organization slug.

      • This can be found at the end of the URL for your Splunk On-Call portal (e.g., https://portal.victorops.com/dash/)

    • Client ID and Client secret: Enter the client ID and secret associated with the application link you created in the previous steps.

  4. Click Save.

If you’ve set everything up correctly, you’ll see the option to Remove Integration in settings.

You can also use the Test configuration button to confirm that the configuration was successful. If your configuration is valid, you’ll see a banner that says “Configuration is valid. If you see issues, please see documentation or reach out to Cortex support.”

How to connect Cortex entities to Splunk On-Call

Editing the entity descriptor

With the Splunk On-Call integration, you can tie on-call rotations to entities by defining the x-cortex-oncall block with your schedule metadata:

x-cortex-oncall:
  victorops:
    type: SCHEDULE
    id: team-abcd12345
Field
Description
Required

type

Type of on-call data (in this case, SCHEDULE)

✓

id

ID for the team assigned to the given schedule

✓

You can find the team ID in the Splunk On-Call portal on the teams page (e.g., https://portal.victorops.com/dash/cortex-app#/team//users).

Expected results

Entity pages

You can also find on-call information for a given entity on the On-call & incidents page in the entity's sidebar.

Scorecards and CQL

With the Splunk On-Call integration, you can create Scorecard rules and write CQL queries based on Splunk On-Call schedules.

Check if on-call is set

Check if entity has a registered team.

Definition: oncall (==/!=) null

Example

For a Scorecard focused an production readiness, you can use this expression to make sure on-call is defined for entities:

oncall != null

This rule will pass if an entity has a service, schedule, or escalation policy set.

Number of escalations

Number of escalation tiers in escalation policy.

Definition: oncall.numOfEscalations()

Example

This expression could be used in a Scorecard focused on production readiness or service maturity:

oncall.numOfEscalations() >= 2

This rule checks that there are at least two tiers in an escalation policy for a given entity, so that if the first on-call does not ack, there is a backup.

On-call metadata

On-call metadata, including type, ID, and name.

Definition: oncall.details()

Example

You can use this expression in the Query builder to find all entities with an on-call rotation that includes a specific team. Let's say we want to find all entities that the "Sample Team" team is on-call for and the team's ID in Splunk On-Call is sample-team1234. Our query would then be:

oncall.details().id == "sample-team1234"

The following options are available to get assistance from the Cortex Customer Engineering team:

  • Chat: Available in the Resource Center

  • Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your Customer Success Manager.

Once a Splunk On-Call schedule is defined in an entity descriptor, the user or team who is on call will appear in the Current On-call block on that .

See more examples in the in Cortex.

Still need help?

Email: , or open a support ticket in the in app Resource Center

entity's details page
CQL Explorer
​
help@cortex.io
VictorOps
Splunk On-Call API key
VictorOps settings page
Scorecards