Datadog

CatalogDiscoveryScorecards

Summary

Datadog is an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services. You can use Datadog to drive insights into values such as:

• Service discovery
• Monitors
• SLOs
• Dependencies

Setup

In order to connect Cortex to your Datadog instance, you’ll need to add a Datadog Application Key and create a Datadog API key, and add it under Settings → Datadog.

caution

If you do not see the Settings page you're looking for in the sidebar, you likely don't have the proper permissions and need to contact your admin.

Multi-account support

Datadog has multi account support meaning that you can configure any amount of Datadog instances to retrieve data from. Each Datadog instance will require an alias which is a name that we use to correlate the Datadog instance with the YAML registrations for each entity. There is also a default configuration used by Datadog YAML registrations without an alias listed. You can edit aliases and the default configuration under Settings → Datadog.

Registration

Discovery

By default, Cortex will use the entity tag (e.g. my-service) as the "best guess" for the Datadog tag. For example, if your entity tag is my-service, then the corresponding tag in Datadog should also be my-service.

If your Datadog tags don’t cleanly match the Cortex entity tag, you can override this in the Cortex Catalog Descriptor.

Catalog descriptor

If you need to override the automatic discovery, you can define the following block in your Cortex Catalog Descriptor.

x-cortex-apm:
  datadog:
    serviceTags: # List of tags & values
      - tag: service
        value: brain
        alias: my-default-alias # alias is optional and only relevant if you have opted into multi account support
      - tag: backend
        value: brain
        alias: my-other-alias # alias is optional and only relevant if you have opted into multi account support

tip

These tags are used to "discover" your monitors and SLOs. Cortex will look up monitors and SLOs by querying for tag:value OR tag:value ...

If you want to hardcode/override discovery altogether, use the following sections:

Monitors

Adding monitors let you see information about their current status directly in the Catalog, under the Datadog section. You can find your monitors at https://app.datadoghq.com/monitors/manage.

The ID of a monitor is found in the URL when you click on a monitor in your Datadog dashboard, for example https://app.datadoghq.com/monitors/**<MONITOR_ID>**.

info:
  x-cortex-apm:
    datadog: # List of monitor IDs.
      monitors:
        - id: 12345
          alias: my-default-alias # alias is optional and only relevant if you have opted into multi account support
        - id: 67890
          alias: my-other-alias # alias is optional and only relevant if you have opted into multi account support

SLOs

Datadog SLOs can be listed in the Catalog under the Datadog section. You can find the SLOs at https://app.datadoghq.com/slo.

The ID for the SLO can be found in the URL when you click on an SLO in the datadog dashboard, for example https://app.datadoghq.com/slo?slo_id=**<SLO_ID>**&timeframe=7d&tab=status_and_history.

info:
  x-cortex-slos:
    datadog: # List of SLO ids
      - id: 0b73859a3e2504bf09ad23a161702654
        alias: my-default-alias # alias is optional and only relevant if you have opted into multi account support
      - id: 228499184a9efe34d4e4e9df838c7fa1
        alias: my-other-alias # alias is optional and only relevant if you have opted into multi account support

Cortex will do an OR query on Datadog using service:brain OR backend:brain to find the SLOs & Monitors.

Dependencies

Cortex automatically syncs dependencies from Datadog's tracing tool into Cortex. By default, we map entries found in Datadog's tracing map using the Service Identifier (x-cortex-tag).

For example, if you have two services "service-one" and "service-two" that have a dependency edge in Datadog tracing, we expect both services in Cortex to exist with the same service identifiers.

You can override this by setting service tags where tag = service, and value = the name of the service as found in Datadog's dependency map.