Google

Overview

Google Workspace is an ownership and cloud resources platform.

Integrating Cortex with Google allows you to:

• Automatically discover and track ownership of Google entities

• Pull in Service Level Objectives (SLOs) from Google Cloud Observability, and view this information on entity pages

• Create Scorecards that track progress and drive alignment on projects involving your Google resources and teams

For information on configuring Google SSO for logging in to Cortex, see the Google SSO documentation.

How to configure Google with Cortex

Prerequisites

Before getting started:

Prerequisite 1: Configure a Google service account and copy its client ID.

Create a Google service account.

• In the Advanced settings, enable Domain-wide Delegation.

• Under the Domain-wide Delegation setting, copy the client ID and store it in a secure location; you will need this in the next steps.

The service account should have the following permissions for each project to enable Google Cloud resources:

Google service account permissions

• AI Platform → AI Platform Viewer, Dataform Viewer, Cloud Storage for Firebase Viewer, Data Catalog Viewer, Vision AI Viewer, Notebooks Viewer, Dataflow Viewer

• Apigee → Cloud Api Hub Viewer

• App Engine → App Engine Viewer

• Artifact Registry → Artifact Registry Reader

• BigQuery → BigQuery Metadata Viewer

• BigQuery Connection → BigQuery Connection User

• Cloud Asset → Cloud Asset Viewer

• Cloud Asset → ListResource

  • Note: This permission is necessary to run services and jobs.

• Cloud Functions → Cloud Functions Viewer

• Cloud Pub/Sub → Pub/Sub Viewer

• Cloud Resource Manager → Browser

• Cloud Run → Cloud Run Viewer

• Cloud SQL → Cloud SQL Viewer

• Cloud Storage → Storage Admin

• Composer → Composer User

• Compute Engine, VM Instances → Compute Viewer

• Kubernetes Engine → Kubernetes Engine Viewer

• Memorystore Memcached → Cloud Memorystore Memcached Viewer

• Memorystore Redis → Cloud Memorystore Redis Viewer

• Monitoring → Monitoring Viewer

• Service Accounts → View Service Accounts

• Spanner → Cloud Spanner Viewer

• VM Instances Vulnerabilities → OS VulnerabilityReport Viewer

• VPC Serverless Connector → Serverless VPC Access Viewer

If you'd like to create a custom role with the minimum permissions required to enable this feature, add the following:

Custom role minimum permissions

aiplatform.datasets.get
aiplatform.datasets.list

aiplatform.endpoints.get
aiplatform.endpoints.list

aiplatform.featurestores.get
aiplatform.featurestores.list

aiplatform.indexEndpoints.get
aiplatform.indexEndpoints.list

aiplatform.batchPredictionJobs.get
aiplatform.batchPredictionJobs.list

aiplatform.modelDeploymentMonitoringJobs.get
aiplatform.modelDeploymentMonitoringJobs.list

aiplatform.trainingPipelines.get
aiplatform.trainingPipelines.list

aiplatform.pipelineJobs.get
aiplatform.pipelineJobs.list

aiplatform.specialistPools.get
aiplatform.specialistPools.list

aiplatform.tensorboardExperiments.get
aiplatform.tensorboardExperiments.list

aiplatform.studies.get
aiplatform.studies.list

aiplatform.apps.get
aiplatform.apps.list

aiplatform.indexes.get
aiplatform.indexes.list

aiplatform.models.get
aiplatform.models.list

aiplatform.tensorboards.get
aiplatform.tensorboards.list

iam.serviceAccounts.get

apihub.apiHubInstances.get

apihub.apis.get
apihub.apis.list

appengine.services.get
appengine.services.list

artifactregistry.repositories.get
artifactregistry.repositories.list

bigquery.connections.get
bigquery.connections.list

bigquery.datasets.get
bigquery.routines.get
bigquery.routines.list

cloudasset.assets.listResource

cloudfunctions.functions.get
cloudfunctions.functions.list

cloudsql.instances.get
cloudsql.instances.list

composer.environments.get
composer.environments.list

compute.urlMaps.list
compute.urlMaps.get
compute.instances.list
compute.instances.get
compute.instanceGroups.list
compute.instanceGroups.get

container.clusters.get
container.clusters.list

container.operations.get
container.operations.list

iam.serviceAccounts.get
iam.serviceAccounts.list

memcache.instances.list
memcache.instances.get

monitoring.services.get
monitoring.services.list
monitoring.slos.get
monitoring.slos.list
monitoring.timeSeries.list

notebooks.instances.get
notebooks.instances.list

osconfig.vulnerabilityReports.get

pubsub.topics.get
pubsub.topics.list

redis.instances.list
redis.instances.get

resourcemanager.projects.get
resourcemanager.projects.list

run.jobs.list
run.jobs.get

run.services.list
run.services.get

spanner.instances.get
spanner.instances.list

spanner.instanceConfigs.get
spanner.instanceConfigs.list

storage.buckets.get
storage.buckets.list

visionai.applications.get
visionai.applications.list

visionai.processors.get
visionai.processors.list

visionai.operators.get
visionai.operators.list

visionai.clusters.get
visionai.clusters.list

vpcaccess.connectors.get
vpcaccess.connectors.list

Prerequisite 2: Configure Google Admin SDK API

Enable the Google Admin SDK API.

Prerequisite 3: Configure Google Cloud resource project permissions

• For Google Cloud resources, in each project, enable the following:

Google Cloud resources project permissions

• App Engine Admin API

• ArtifactRegistry API

• Apigee APIs

• BigQuery API

• BigQuery Connection API

• Cloud Asset API

• Cloud Composer API

• Cloud Functions

• Cloud SQL Admin

• Cloud Storage

• Compute Engine API

• Kubernetes Engine API

• Memorystore for Memcached API

• Memorystore for Redis API

• OS Config API

• Kubernetes Engine API

• Resource Manager API

• Spanner API

• Serverless VPC Access API

• For each project in Vertex AI, enable the following:

  • Cloud Storage API

  • DataCatalog API

  • Dataflow AI API

  • DataForm API

  • Notebooks AI API

  • Vertex AI API

  • Vision AI API

Step 1: Configure the integration in Google

1. In the G Suite admin console, navigate to Security > API Controls > Manage Domain Wide Delegation. Click Add new.

2. Add the client ID you obtained in Prerequisite 1, and include the following scopes:

   • https://www.googleapis.com/auth/admin.directory.group.readonly

   • https://www.googleapis.com/auth/admin.directory.group.member.readonly

3. Navigate to the service account you created for this integration. Click Keys, then generate a key in JSON format.

4. Navigate to Admin Roles > Groups Reader and expand the "Admins" panel.

5. Click Assign service accounts then enter the email of the service account you created for this integration.

Step 2: Configure the integration in Cortex

1. In Cortex, navigate to the Google settings page:

   • Click Integrations from the main nav. Search for and select Google.

2. Click Add configuration.

3. Configure the Google integration form:

   • Domain: Enter your Google domain.

   • Service account email: Enter the email address for the service account.

   • Credentials JSON: Enter the service account JSON key you created in the previous steps.

4. Click Save.

By default, a service will have dependencies on any resource with Google Cloud tag label = "service" and tag value = the service's Cortex tag. After saving your integration, you may customize the tag key name here by entering a new name into the Custom label key field. Leave it blank to use "service" as the key name.

Supported Google entity types

Cortex supports pulling in the following entity types from Google:

Supported Google entity types

• Google Cloud Vertex AI Batch Prediction Job

• Google Cloud Vertex AI Dataset

• Google Cloud Vertex AI Endpoint

• Google Cloud Vertex AI Featurestore

• Google Cloud Vertex AI Index

• Google Cloud Vertex AI Model

• Google Cloud Vertex AI Model Deployment Monitoring Job

• Google Cloud Vertex AI Notebooks Instance

• Google Cloud Vertex AI Pipeline Job

• Google Cloud Vertex AI Platform Index Endpoint

• Google Cloud Vertex AI Specialist Pool

• Google Cloud Vertex AI Study

• Google Cloud Vertex AI Tensorboard

• Google Cloud Vertex AI Training Pipeline

• Google Cloud Vertex AI Vision Application

• Google Cloud Vertex AI Vision Cluster

• Google Cloud Vertex AI Vision Index Point

• Google Cloud Vertex AI Vision Operator

• Google Cloud Vertex AI Vision Processor

• Google Cloud Apigee Api

• Google Cloud Apigee Instance

• Google Cloud App Engine Service

• Google Cloud Artifact Registry Repository

• Google Cloud BigQuery Connection

• Google Cloud BigQuery

• Google Cloud Composer Environment

• Google Cloud Functions

• Google Cloud Kubernetes Engine Clusters

• Google Cloud Kubernetes Engine Operations

• Google Cloud IAM Service Account

• Google Cloud Instance Group

• Google Cloud HTTP(S) Load Balancing

• Google Cloud Memorystore Memcached

• Google Cloud Memorystore Redis

• Google Cloud Project

• Google Cloud Run Job

• Google Cloud Run Service

• Google Cloud Spanner Instance

• Google Cloud Spanner Instance Config

• Google Cloud SQL

• Google Cloud Storage

• Google Cloud Pub/Sub Topics

• Google Cloud VM Instances

• Google Cloud VPC Serverless Connector

How to connect Cortex entities to Google

Enable automatic import of Google entities

You can configure automatic import from Google Cloud. Note that this setting does not include team entities.

1. In Cortex, navigate to Settings > Entities > General.

2. Next to Auto import from AWS, Azure, and/or Google Cloud, click the toggle to enable the import.\

   

Import teams from Google

See the Create teams documentation for instructions on importing entities.

Automatic ownership of Google entities

Cortex can use Google Groups as an ownership provider, automatically syncing memberships from any Google Group mailing list.

Automatic Google dependency discovery

By default, Cortex will try to automatically discover dependencies between your entities and Google Cloud resources with a matching label. By default the label key that will be matched is service, however you can customize this key value in the Google Cloud Settings page.

If you'd like to explicitly define these Google Cloud dependencies, the x-cortex-dependency field should be a map, defined as follows:

x-cortex-dependency:
   gcp:
     labels:
       - key: my-key-1
         value: my-value-1
       - key: my-key-2
         value: my-value-2

Editing the entity descriptor

Groups

x-cortex-owners:
  - type: group
    name: [email protected]
    provider: GOOGLE
    description: This is a description for this owner # optional

The value for name should be the full group email as defined in Google Groups.

Entities

Cortex uses the resource name and project ID to look up catalog entities in your Google Cloud account. Function resource names should be of the format location/function

x-cortex-infra:
  Google Cloud:
    resources:
      - resourceName: location/function
        projectId: project1
        resourceType: function
      - resourceName: example-bucket
        projectId: project1
        resourceType: storage

SLOs

x-cortex-slos:
    gcp:
      - projectId: cortex-gcp-integration
        serviceId: iLE2e4HvR_iVlxAaBbCc12
      - projectId: cortex-gcp-integration
        serviceId: adfdfdafd

The serviceID value is the value of the Unique ID listed on the service page in Google Cloud Observability.

Using the Google integration

View Google Cloud Observability data in entity pages

After integrating with Google, you will see data from Google Cloud Observability on entity details pages:

• On an entity's overview page, see an overview of SLOs for the entity.

• Click Monitoring > Google in an entity's sidebar to see more information about Google SLOs, including the SLO name, its targets, its status, the current value for that entity, and the period of time the SLO is being calculated for. For example, if the time listed is "7 days ago," then the SLO is looking at the time range starting 7 days ago to now.

Scorecards and CQL

With the Google integration, you can create Scorecard rules and write CQL queries based on GCP details, Google Cloud Observability SLOs, and Google teams.

See more examples in the CQL Explorer in Cortex.

GCP details

Get the GCP details for the entity.

Definition: gcp.details()

Examples

A Scorecard might include a rule to verify that an entity has GCP details:

gcp.details() != null

You might include a rule to check whether any labels on the GCP recourse are titled origin:

jq(gcp.details(), ".resources[0].labels | any(\"origin\")")

SLOs

SLOs associated with the entity via ID or tags. You can use this data to check whether an entity has SLOs associated with it, and if those SLOs are passing.

Definition: slos: List<SLO>

Example

In a Scorecard, you can use this expression to make sure an entity is passing its SLOs:

slos().all((slo) => slo.passing) == true

Use this expression to make sure latency Service Level Indicator (SLI) value is above 99.99%:

slos().filter((slo) => slo.name.matchesIn("latency") and slo.sliValue >= 0.9999).length > 0

Ownership CQL

All ownership details

A special built-in type that supports a null check or a count check, used to enforce ownership of entities.

Definition: ownership: Ownership | Null

Example

An initial level in a security Scorecard might include a rule to ensure an entity has at least one team as an owner:

ownership.teams().length > 0

All owner details

List of owners, including team members and individual users, for each entity

Definition: ownership.allOwners()

Example

The Scorecard might include a rule to ensure that entity owners all have an email set:

ownership.allOwners().all((member) => member.email != null)

Team details

List of teams for each entity

Definition: ownership.teams(): List<Team>

Example

The Scorecard might include a rule to ensure that an entity owners all have a description and are not archived:

ownership.teams().all(team => team.description != null and team.isArchived == false)

View integration logs

This feature is available to Cortex cloud customers.

On the integration settings page, click the Logs tab to view logs from the last 7 days. Learn more in Troubleshooting with integration logs.

Background sync

Cortex conducts an ownership sync for Google teams every day at 9 a.m. UTC.

Troubleshooting and FAQ

The GCP integration only supports a single service account. Can I work around this?

By default, GCP service accounts are restricted to the project they were created in. If other projects don’t explicitly allow that service account to access their resources, Cortex can’t collect data from them. To work around this, you can configure a principal service account and associate it with multiple projects in GCP. Once the service account is linked to other projects, Cortex can use that service account to pull data from multiple GCP projects.

After creating a service account that is linked to a project, open your second project in GCP and go to IAM & Admin > IAM > Click +Add. Using the service account ID that you already created, add a principal to the project. Repeat these steps for each project.

Still need help?​

The following options are available to get assistance from the Cortex Customer Engineering team:

• Email: [email protected], or open a support ticket in the in app Resource Center

• Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your Customer Success Manager.