AWS

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs.

Integrating Cortex with AWS allows you to:

Track AWS entities in the catalog
Automatically discover and track ownership of AWS entities and dependencies
- You can also enable auto-import of any discovered entities of known types.
Create Scorecards that track progress and drive alignment on projects involving your AWS resources

If you are on a self-hosted Cortex instance, see the self-managed AWS setup page.

How to configure AWS with Cortex

Step 1: Configure the integration in Cortex

In Cortex, navigate to the AWS settings page:
1. In Cortex, click your avatar in the lower left corner, then click Settings.
2. Under "Integrations", click AWS.
Click Add configuration.
In the modal, the JSON configuration, Cortex AWS account ID, and External ID are displayed. Keep this browser window open, as you will need these in the next steps.

Step 2: Set up an IAM policy in AWS

For each account:

Log in to your AWS Management Console and open the IAM console.
Click Policies, then choose Create policy.
Switch to the JSON editor. Copy the JSON starting policy from Cortex, and paste it into the JSON editor.
- This policy allows Cortex to list all resources, resource types, and resource tags.
- If you are pulling in ECS resources, add the following actions to the JSON policy:
```
"rds:Describe*",
"rds:List*",
"s3:Describe*",
"s3:List*"
```
For each resource type that you want to import into Cortex, add policies for reading that type of AWS resource.
- For example, if you want to import resources of type "AWS::IAM::Role", we'll need to have permission to "iam:ListRoles", "iam:ListAttachedRolePolicies", "iam:GetRole", and "iam:ListRolePolicies". Because this is a dynamic feature, Cortex does not automatically determine this. One option is to start with ReadOnlyAccess permissions and remove sensitive permissions as deemed necessary.
Click Review Policy, enter a name, then click Create Policy.

See the AWS documentation for more information: Create IAM policies.

Step 3: Create a role in AWS

This section is specific to cloud-based Cortex accounts. If you are on a self-hosted Cortex instance, please see the AWS account setup guide for self-hosted Cortex.

In AWS, navigate to Roles > Create Role.
For the trusted entity type, select Another AWS account.
In the Account ID field, enter the Cortex AWS account ID that was displayed in Cortex in the earlier steps.
Click Require External ID, then enter the Cortex external ID that was displayed in Cortex in the earlier steps.
Click Next.
Select your newly created policy, and click Next.
Enter a name for your role. Optionally, configure tags. When you are finished, click Create Role.
Search for your new role in the list and copy its name. You will need this in the next steps.
In the upper right corner of AWS, click your name. In the dropdown that appears, copy your AWS account ID. You will need these in the next steps.

Note that if you use multiple AWS accounts, they will share a common rotatable externalId.

Step 4: Finish the configuration in Cortex

Navigate back to the browser window containing your Cortex AWS settings page.
Configure the AWS integration form:
- Account ID: Enter the AWS account ID you obtained in the previous steps.
- IAM role: Enter the role name you obtained in the previous steps.
Click Save.

Step 5: Select AWS resource types

In your AWS integration settings page in Cortex, Cortex will pull all the types you included in the IAM policy into the Cloud Control types dropdown.

If any resource types do not appear in the list, ensure that cloudformation:ListTypes, cloudformation:ListResources, and cloudformation:GetResource are added to the IAM policy so Cortex can pull the list of all available types from AWS.

To select your cloud control types:

In the Cloud control types field, select the types you want Cortex to discover.
- If automatic import is enabled, then these types will automatically be imported into Cortex.
  - If you later need to remove any auto-imported cloud control types, see the FAQ below.
Click Save cloud control types.

If the type you're looking to import is in the list below, please reach out to [email protected] to submit a feature request.

The following Cloud Control types are not currently supported:

AWS::ApiGateway::DocumentationVersion
AWS::ApiGateway::Step
AWS::CloudFormation::ResourceVersion
AWS::CustomerProfiles::Integration
AWS::CustomerProfiles::ObjectType
AWS::EC2::TransitGatewayMulticastGroupMember
AWS::EC2::TransitGatewayMulticastGroupSource
AWS::ECS::TaskSet
AWS::Glue::Attach::SchemaVersion
AWS::Glue::Attach::SchemaVersionMetadata
AWS::IoTSiteWise::AccessPolicy
AWS::IoTSiteWise::Dashboard
AWS::IoTSiteWise::Project
AWS::Kendra::DataSource
AWS::Kendra::Faq
AWS::MediaConnect::FlowEntitlement
AWS::MediaConnect::FlowOutput
AWS::MediaConnect::FlowSource
AWS::MediaConnect::FlowVpcInterface
AWS::MediaPackage::Asset
AWS::MediaPackage::PackagingConfiguration
AWS::NetworkFirewall::LoggingConfiguration
AWS::QuickSight::Analysis
AWS::QuickSight::Dashboard
AWS::QuickSight::DataSet
AWS::QuickSight::DataSource
AWS::QuickSight::Template
AWS::QuickSight::Theme
AWS::RDS::DBProxyTargetGroup
AWS::S3Outposts::AccessPoint
AWS::S3Outposts::Bucket
AWS::SSO::Assignment
AWS::SSO::InstanceAccessControlAttributeConfiguration
AWS::SSO::PermissionSet

How to connect Cortex entities to AWS

Enable automatic import of AWS entities

You can configure automatic import from AWS:

In Cortex, navigate to the Entities Settings page.
Next to Auto import from AWS, Azure, and/or Google Cloud, click the toggle to enable the import.

If you do not have automatic import enabled, you can manually import.

Limit discovery to specific regions

By default, Cortex will search for resources across all AWS regions, but you can limit that to specific regions in the Cortex AWS settings page.

Import entities from AWS

See the Create services documentation for instructions on importing entities.

Discover dependencies for AWS

Cortex automatically discovers dependencies between your services and resources by scanning for resources with specific AWS tags. By default, a service will have dependencies on any Cortex resource that has a corresponding AWS resource with AWS tag key = "service" and tag value = the service's Cortex tag.

Customize AWS tag names

In Cortex under the AWS integration page, you can customize the tag key names. Expand the Dependencies sync from AWS section, then select tags from the dropdown menu. Note that an AND operator is used when you select multiple tags; the resource will need to have all specified tags in order to be recognized by Cortex.

If you do not specify tag names, Cortex will use "service" as the key name.

AWS dependency sync

Cortex syncs AWS tags daily at 8 a.m. UTC. To manually refresh the tags, navigate to the relationship graph in your workspace, click the menu in the upper right corner, then click Sync dependencies.

Use key/value pairs in the entity descriptor for discovery

You can also use explicit tag key/value pairs in the x-cortex-dependency block for AWS dependency discovery. Instead of making a service depend on a resource based on service tags, Cortex will make a service depend on a resource if any of the resource's AWS tags match the explicitly defined key/value pairs in the service's x-cortex-dependency block. For example, the service below will have dependencies on any AWS resource with tag (key = aws:cloudformation:my-key-1, value = arn:aws:cloudformation:my-region:my-value-1) or tag (key = aws:cloudformation:my-key-2, value = arn:aws:cloudformation:my-region:my-value-2).

x-cortex-dependency:
  aws:
    tags:
      - key: my-key-1
        value: my-value-1
      - key: my-key-2
        value: my-value-2
      - key: "aws:cloudformation:my-key-1"
        value: "arn:aws:cloudformation:my-region:my-value-1"
      - key: "aws:cloudformation:my-key-2"
        value: "arn:aws:cloudformation:my-region:my-value-2"

For more information on dependencies, see the Dependencies documentation.

Discover ownership for AWS

Cortex can automatically discover ownership for your AWS resources. To enable this, make sure that your AWS resources have a tag matching the x-cortex-tag of the corresponding Cortex team and enable the “Sync ownership from AWS” toggle in the Settings page. By default, we look for the owner tag. You can also customize the tag key name.

Cortex syncs ownership from AWS every day at 6 am UTC.

Editing the entity descriptor

We recommend automatically importing your AWS entities for the fastest and most efficient experience connecting your data. If you need to manually edit entity descriptors, see the information below.

You can associate a Cortex entity with one or more AWS entities. For certain AWS resource types, Cortex will display those AWS entities' metadata on the Cortex entity page.

x-cortex-infra:
  aws:
    cloudControl:
    - type: AWS::RDS::DBInstance
      region: us-west-2
      accountId: "123456123456"
      identifier: rds-example

Multiple ECS services on a single entity

You can associate a Cortex entity with multiple ECS services.

If you are using the Cloud Control resource types, use the format below:

x-cortex-infra:
  aws:
    cloudControl:
    - type: AWS::ECS::Service
      region: us-west-2
      accountId: "123456123456"
      identifier: ecs-example
    - type: AWS::ECS::Service
      region: us-west-2
      accountID: "34567345673"
      identifier: ecs-example-2

If you are not using Cloud Control types or if you imported your entity prior to Cortex supporting Cloud Control types, you can use the format shown below:

x-cortex-infra:
  aws:
    ecs:
      - clusterArn: abcd
        serviceArn: efgh
      - clusterArn: stuv
        serviceArn: wxyz

The values for clusterArn and serviceArn are defined in ECS.

Discovery audit

Cortex will pull recent changes from your AWS environment into the discovered entities list. Here, you can find new entities in AWS that have not been imported into the catalog - these will have the tag New AWS Resource - as well as entities in the catalog that no longer exist in AWS - these will have the tag AWS Resource Not Detected.

Searching AWS entities in Cortex

The following keys are supported when searching for your AWS entities in Cortex under Catalogs > All Entities:

aws-account-id - Account ID number
aws-account-name - Account alias
aws-region - AWS region of the resource
aws-type - AWS type of the resource
aws-name - AWS name of the resource
aws-identifier - The primary identifier of a resource
aws-secondary-identifier - The secondary identifier of a resouce

Example search queries

aws-type:"AWS::EC2" AND aws-region:"us-west": Search for entities of category EC2 in the any of us-west regions
aws-account-id: "234512324": Search for all entities from the account 234512324
aws-name:"aws-identifier-of-resource" AND aws-account-name:"test-account": Search for entity with identifier aws-identifier-of-resource in the account with alias test-account

Scorecards and CQL

With the AWS integration, you can create Scorecard rules and write CQL queries based on AWS resources.

See more examples in the CQL Explorer in Cortex.

AWS details

Get the AWS details for an entity

Definition: aws.details(): Object

Example

In a Scorecard, you can create a rule to verify that an entity of type lamda has a correct function name:

aws.details().resources.filter((resource) => resource.typeName == "AWS::Lambda::Function").length > 0

You could also create a rule to verify that an entity is not using deprecated runtimes:

aws.details().resources.filter((resource) => resource.typeName == "AWS::Lambda::Function" and resource?.metadata?.get("Runtime")?.matchesIn("(python3\\.6|python2\\.7|dotnetcore2\\.1|ruby2\\.5|nodejs12\\.|nodejs10\\.|nodejs8\\.10|nodejs4\\.3|nodejs6\\.10|dotnetcore1\\.0|dotnetcore2\\.0|nodejs4\\.3-edge|nodejs$)")).length == 0

Background sync

Cortex conducts the following background syncs for AWS:

Ownership sync daily at 6 a.m. UTC
AWS tag sync (dependencies) daily at 8 a.m. UTC
Integration details daily at 10 a.m. UTC

Troubleshooting and FAQ

If I have auto-import enabled, how can I remove cloud control types that I no longer want to be imported?

If you want to remove any of the cloud control types after importing them: Disable the automatic import setting, remove the cloud control types from your AWS integration settings, then enable auto-archival. This will cause the removed cloud control types to be archived during the next sync.

Still need help?

The following options are available to get assistance from the Cortex Customer Engineering team:

Email: [email protected], or open a support ticket in the in app Resource Center
Chat: Available in the Resource Center
Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your Customer Success Manager.

Last updated 8 hours ago

Was this helpful?

How to configure AWS with Cortex

Step 1: Configure the integration in Cortex

Step 2: Set up an IAM policy in AWS

Step 3: Create a role in AWS

Step 4: Finish the configuration in Cortex

Step 5: Select AWS resource types

How to connect Cortex entities to AWS

Enable automatic import of AWS entities

Limit discovery to specific regions

Import entities from AWS

Discover dependencies for AWS

Customize AWS tag names

AWS dependency sync

Discover ownership for AWS

Editing the entity descriptor

Discovery audit

Searching AWS entities in Cortex

Example search queries

Scorecards and CQL

Example

Background sync

Troubleshooting and FAQ

If I have auto-import enabled, how can I remove cloud control types that I no longer want to be imported?

Still need help?​

Still need help?