Skip to main content

AWS

CatalogDiscovery

Overview​

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs. After configuring the integration with Cortex, you will be able to pull in entities, ownership, and dependencies.

If you are on a self-hosted Cortex instance, see the On-premises AWS setup page.

How to configure AWS with Cortex​

caution

If you do not see the settings page you're looking for, you may not have permission to access that page. Please contact your admin for assistance.

Step 1: Configure the integration in Cortex​

  1. In Cortex, navigate to the AWS settings page:
    1. In Cortex, click your avatar in the lower left corner, then click Settings.
    2. Under "Integrations", click AWS.
  2. Click Add AWS configuration.
  3. In the modal, the JSON configuration, Cortex AWS account ID, and External ID are displayed. Keep this browser window open, as you will need these in the next steps.

Step 2: Set up an IAM policy in AWS​

For each account:

  1. Log in to your AWS Management Console and open the IAM console.
  2. Click Policies, then choose Create policy.
  3. Switch to the JSON editor. Copy the JSON starting policy from Cortex, and paste it into the JSON editor.
    • This policy allows Cortex to list all resources, resource types, and resource tags.
    • If you are pulling in ECS resources, add the following actions to the JSON policy:
    "rds:Describe*",
    "rds:List*",
    "s3:Describe*",
    "s3:List*"
  4. For each resource type that you want to import into Cortex, add policies for reading that type of AWS resource.
    • For example, if you want to import resources of type "AWS::IAM::Role", we'll need to have permission to "iam:ListRoles", "iam:ListAttachedRolePolicies", "iam:GetRole", and "iam:ListRolePolicies". Because this is a dynamic feature, Cortex does not automatically determine this. One option is to start with ReadOnlyAccess permissions and remove sensitive permissions as deemed necessary.
  5. Click Review Policy, enter a name, then click Create Policy.

See the AWS documentation for more information: Create IAM policies.

Step 3: Create a role in AWS​

This section is specific to cloud-based Cortex accounts. If you are on a self-hosted Cortex instance, please see the AWS account setup guide for self-hosted Cortex.

  1. In AWS, navigate to Roles > Create Role.
  2. For the trusted entity type, select Another AWS account.
  3. In the Account ID field, enter the Cortex AWS account ID that was displayed in Cortex in the earlier steps.
  4. Click Require External ID, then enter the Cortex external ID that was displayed in Cortex in the earlier steps.
  5. Click Next.
  6. Select your newly created policy, and click Next.
  7. Enter a name for your role. Optionally, configure tags. When you are finished, click Create Role.
  8. Search for your new role in the list and copy its name. You will need this in the next steps.
  9. In the upper right corner of AWS, click your name. In the dropdown that appears, copy your AWS account ID. You will need these in the next steps.

Note that if you use multiple AWS accounts, they will share a common rotatable externalId.

Step 4: Finish the configuration in Cortex​

  1. Navigate back to the browser window containing your Cortex AWS settings page.
  2. Configure the AWS integration form:
    • Account ID: Enter the AWS account ID you obtained in the previous steps.
    • IAM role: Enter the role name you obtained in the previous steps.
  3. Click Save.

Step 5: Resource types setup​

Cortex will pull all the types you included in the IAM policy under the Cloud Control types field in the Settings section. If resource types aren't appearing in the list, there is likely a permission issue, and the role isn’t set up to discover Cloud Control types. Make sure that "cloudformation:ListTypes", "cloudformation:ListResources", and "cloudformation:GetResource" are added to the IAM policy, so Cortex can pull the list of all available types from AWS.

To select your cloud control types:

  1. In the Cloud control types field, select the types you want Cortex to discover/import.
  2. Click Save cloud control types.

Some Cloud Control types are not currently supported. If the type you're looking to import is in the list, please reach out to support@cortex.io to submit a feature request.

AWS::ApiGateway::DocumentationVersion
AWS::ApiGateway::Step
AWS::CloudFormation::ResourceVersion
AWS::CustomerProfiles::Integration
AWS::CustomerProfiles::ObjectType
AWS::EC2::TransitGatewayMulticastGroupMember
AWS::EC2::TransitGatewayMulticastGroupSource
AWS::ECS::TaskSet
AWS::ElasticLoadBalancingV2::Listener
AWS::ElasticLoadBalancingV2::ListenerRule
AWS::Glue::Attach::SchemaVersion
AWS::Glue::Attach::SchemaVersionMetadata
AWS::IoTSiteWise::AccessPolicy
AWS::IoTSiteWise::Dashboard
AWS::IoTSiteWise::Project
AWS::Kendra::DataSource
AWS::Kendra::Faq
AWS::MediaConnect::FlowEntitlement
AWS::MediaConnect::FlowOutput
AWS::MediaConnect::FlowSource
AWS::MediaConnect::FlowVpcInterface
AWS::MediaPackage::Asset
AWS::MediaPackage::PackagingConfiguration
AWS::NetworkFirewall::LoggingConfiguration
AWS::QuickSight::Analysis
AWS::QuickSight::Dashboard
AWS::QuickSight::DataSet
AWS::QuickSight::DataSource
AWS::QuickSight::Template
AWS::QuickSight::Theme
AWS::RDS::DBProxyTargetGroup
AWS::S3Outposts::AccessPoint
AWS::S3Outposts::Bucket
AWS::SSO::Assignment
AWS::SSO::InstanceAccessControlAttributeConfiguration
AWS::SSO::PermissionSet

How to connect Cortex entities to AWS​

Limit discovery to specific regions​

By default, Cortex will search for resources across all AWS regions, but you can limit that to specific regions in the Cortex AWS settings page.

Enable automatic discovery of AWS entities​

You can configure automatic import from AWS:

  1. In Cortex, navigate to the Entities Settings page.
  2. Next to Auto import from AWS, Azure, and/or Google Cloud, click the toggle to enable the import.
    Enable auto discovery of entities

Define dependencies​

Cortex automatically discovers dependencies between your services and resources by scanning for resources with specific AWS tags. By default, a service will have dependencies on any Cortex resource that has a corresponding AWS resource with AWS tag key = "service" and tag value = the service's Cortex tag. In your Cortex settings under the AWS integration section, you can customize the tag key names, or leave them blank to use "service" as the key name.

You can also use explicit tag key/value pairs in the x-cortex-dependency block for AWS dependency discovery. Instead of making a service depend on a resource based on service tags, Cortex will make a service depend on a resource if any of the resource's AWS tags match the explicitly defined key/value pairs in the service's x-cortex-dependency block. For example, the service below will have dependencies on any AWS resource with tag (key = aws:cloudformation:my-key-1, value = arn:aws:cloudformation:my-region:my-value-1) or tag (key = aws:cloudformation:my-key-2, value = arn:aws:cloudformation:my-region:my-value-2).

x-cortex-dependency:
  aws:
    tags:
      - key: my-key-1
        value: my-value-1
      - key: my-key-2
        value: my-value-2
      - key: "aws:cloudformation:my-key-1"
        value: "arn:aws:cloudformation:my-region:my-value-1"
      - key: "aws:cloudformation:my-key-2"
        value: "arn:aws:cloudformation:my-region:my-value-2"

For more information on dependencies, please see the Dependencies documentation and the Dependencies section on this page.

Discover ownership for AWS​

Cortex can automatically discover ownership for your AWS resources. To enable this, make sure that your AWS resources have a tag matching the x-cortex-tag of the corresponding Cortex team and enable the “Sync ownership from AWS” toggle in the Settings page. By default, we look for the owner tag. You can also customize the tag key name.

caution

Cortex syncs ownership from AWS every day at 6 am UTC.

Editing the entity descriptor​

You can associate a Cortex entity with one or more AWS entities. For certain AWS resource types, Cortex will display those AWS entities' metadata on the Cortex entity page.

x-cortex-infra:
  aws:
    arns:
      - arn:aws:rds:us-east-1:229540587644:cluster:alpha
      - arn:aws:rds:us-east-1:229540587644:cluster:bravo

Multiple ECS services on a single entity​

You can associate a Cortex entity with multiple ECS services:

x-cortex-infra:
  aws:
    ecs:
      - clusterArn: abcd
        serviceArn: efgh
      - clusterArn: stuv
        serviceArn: wxyz

The values for clusterArn and serviceArn are defined in ECS.

Discovery audit​

Cortex will pull recent changes from your AWS environment into the discovery audit. Here, you can find new entities in AWS that have not been imported into the catalog - these will have the tag New AWS Resource - as well as entities in the catalog that no longer exist in AWS - these will have the tag AWS Resource Not Detected.

Searching AWS entities in Cortex​

The following keys are supported when searching for your AWS entities in Cortex under Catalogs > All Entities:

  • aws-account-id - Account ID number
  • aws-account-name - Account alias
  • aws-region - AWS region of the resource
  • aws-type - AWS type of the resource
  • aws-name - AWS name of the resource
  • aws-identifier - The primary identifier of a resource
  • aws-secondary-identifier - The secondary identifier of a resouce

Example search queries​

  • aws-type:"AWS::EC2" AND aws-region:"us-west": Search for entities of category EC2 in the any of us-west regions
  • aws-account-id: "234512324": Search for all entities from the account 234512324
  • aws-name:"aws-identifier-of-resource" AND aws-account-name:"test-account": Search for entity with identifier aws-identifier-of-resource in the account with alias test-account
    Example search term in the search bar on the Entities page

Scorecards and CQL​

With the AWS integration, you can create Scorecard rules and write CQL queries based on AWS resources.

See more examples in the CQL Explorer in Cortex.

AWS details

Get the AWS details for an entity

Definition: aws.details(): Object

Example

In a Scorecard, you can create a rule to verify that an entity of type lamda has a correct function name:

aws.details().resources.filter((resource) => resource.typeName == "AWS::Lambda::Function").length > 0

You could also create a rule to verify that an entity is not using deprecated runtimes:

aws.details().resources.filter((resource) => resource.typeName == "AWS::Lambda::Function" and resource?.metadata?.get("Runtime")?.matchesIn("(python3\\.6|python2\\.7|dotnetcore2\\.1|ruby2\\.5|nodejs12\\.|nodejs10\\.|nodejs8\\.10|nodejs4\\.3|nodejs6\\.10|dotnetcore1\\.0|dotnetcore2\\.0|nodejs4\\.3-edge|nodejs$)")).length == 0

Background sync​

Cortex conducts a background sync of AWS integration details every day at 10 a.m. UTC, an ownership sync every day at 6 a.m. UTC. and a dependency sync every day at 11 p.m. UTC.

Still need help?​

The following are all the ways to get assistance from our customer engineering team. Please use the option that is best for your users:

  • Email: help@cortex.io, or open a support ticket in the in app Resource Center
  • Chat: Available in the Resource Center
  • Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your customer success manager.