Skip to main content

AWS

CatalogDiscovery

Summary

Amazon Web Services, Inc. (AWS) that provides on-demand cloud computing platforms and APIs. You can use AWS to drive insights into values such as:

  • Catalog Discovery
  • Dependencies

Setup

In order to connect Cortex to your AWS resources, you’ll need to add your AWS Account ID and IAM Role in Settings → AWS. For Cloud Cortex, we support multiple accounts. Each will share a common externalId that can be rotated. For each account:

caution

If you do not see the Settings page you're looking for in the sidebar, you likely don't have the proper permissions and need to contact your admin.

  1. Log onto the AWS management console and navigate to IAM
  2. Create a new IAM policy by navigating to Policies | Create Policy.
  3. Switch to the JSON editor and insert the following policy:
    {
    "Version":"2012-10-17",
    "Statement": [
    {
    "Action": [
    "ecs:Describe*",
    "ecs:List*",
    "kafka:Describe*",
    "kafka:List*",
    "lambda:Get*",
    "lambda:List*",
    "rds:Describe*",
    "rds:List*",
    "s3:Describe*",
    "s3:List*",
    "s3:GetBucketLocation",
    "s3:GetBucketTagging",
    "tag:GetResources",
    "tag:GetTagKeys",
    "tag:GetTagValues"
    ],
    "Effect":"Allow",
    "Resource":"*"
    }
    ]
    }
  4. Click Review Policy, name it whatever you like, then Create Policy.

Cloud

  1. Next, let's create a new role and attach the new policy to it. Navigate to Roles | Create Role.
  2. Select Another AWS account as the trusted entity type.
  3. For Account ID, fill in the Cortex AWS account ID from Settings | AWS.
  4. Click Require External ID and insert the Cortex external ID from Settings | AWS, then click Next: Permissions.
  5. Select your newly created policy, and click Next: Tags, if you'd like to add any, otherwise skip the next page with Next: Review.
  6. Name your role, and click Create Role.
  7. Finally, copy your new role name and AWS account ID into the Cortex AWS settings page.

On-Prem

For on-prem, you'll need to create access users along with access keys for each AWS account you'd like access to. They should have all the permissions listed above. For each set of access keys, set these environment variables (access key id, access key secret, and account number) in your ConfigMap:

  • AMAZON_ACCESS_0_ID
  • AMAZON_ACCESS_0_SECRET
  • AMAZON_ACCESS_0_ACCOUNT
  • AMAZON_ACCESS_1_ID
  • AMAZON_ACCESS_1_SECRET
  • AMAZON_ACCESS_1_ACCOUNT
  • ...
  • AMAZON_ACCESS_N_ID
  • AMAZON_ACCESS_N_SECRET
  • AMAZON_ACCESS_N_ACCOUNT

Registration

Catalog Descriptor

Cortex uses the resource's ARN to look up catalog entities in your AWS account. You can tie multiple AWS resources to a single entity within Cortex.

x-cortex-infra:
aws:
arns:
- arn:aws:rds:us-east-1:229540587644:cluster:alpha
- arn:aws:rds:us-east-1:229540587644:cluster:bravo

Dependencies

Cortex automatically discovers dependencies between your services and resources by scanning for resources with specific AWS tags. By default, a service will have dependencies on any Cortex resource that has a corresponding AWS resource with AWS tag key = "service" and tag value = the service's Cortex tag. In AWS settings, you can customize the tag key name, or leave it blank to use "service" as the key name.