Pub/Sub-Based Google Cloud Integration - Terraform from Google Cloud Shell

tip

To integrate with Google Cloud, Lacework recommends using Guided Configuration, a wizard-like interface that takes your inputs and generates a script that downloads and sets up all necessary Lacework CLI and Terraform components to create the integration non-interactively.

To use guided configuration:

1. In the Lacework Console go to Settings > Integrations > Cloud accounts.
2. Click + Add New.
3. Click Google Cloud Platform and select Guided configuration.

Alternatively, follow the steps in this topic for the following methods:

• Use the Lacework CLI to generate and run Terraform code.
• Create the main.tf file manually and run Terraform from Google Cloud Shell. This may be required for complex integration scenarios.

Overview

Lacework integrates with Google Cloud Platform (GCP) to analyze Cloud Audit Logs and assess cloud resource configurations at the organization level or at the project level. The Lacework platform ingests configuration and audit log information from Google Cloud and reports alerts for anomalous behavior.

• Organization level integrations cover all the existing projects in the organization. After you create the integration, any new projects you create are automatically included in the integration and any projects you delete are automatically excluded.
• Project level integrations cover only a specific project. You must set up a new integration, if required, for any new project you create.

This topic describes how you can set up an organization level or project level configuration and Pub/Sub-based audit log integration by running Lacework Terraform modules from Google Cloud Shell.

Running Terraform from Google Cloud Shell is suitable for one-off integrations where you do not plan to continue using Terraform to manage the integrations. If you plan to continue using Terraform to manage the integrations or store the state of the configuration in a source control management tool such as Git, use the instructions in Pub/Sub-Based Integration From Any Supported Host.

info

If you want to use Terraform to migrate an existing Storage-based audit log integration to a Pub/Sub-based audit log integration, see Migrate From Storage-Based to Pub/Sub-Based Google Cloud Audit Log Integration Using Terraform.

If you are new to the Lacework Terraform Provider or Lacework Terraform Modules, read Terraform for Lacework Overview to learn the basics on how to configure the provider.

Google Cloud Resources

To integrate at the organization or project level, Lacework requires the following resources to be provisioned in Google Cloud. You must create the project and service account in Google Cloud. The Lacework Terraform modules automatically provision the log routing sink, Pub/Sub topic, and Pub/Sub subscription with the required permissions in Google Cloud.

• Google Cloud Project - A project that contains the resources required for the integration with billing enabled.

  • For an organization level integration, Lacework recommends that you create a new project specifically for the Lacework resources.
  • For a project level integration, you can provision the required Lacework resources within the project being integrated.

• Google Pub/Sub Topic - For streaming Cloud Audit Log events to the Lacework platform.

• Google Pub/Sub Subscription - Subscription for Lacework to pull the Audit Log events from the Pub/Sub topic.

• Google Log Routing Sink - To route Cloud Audit Logs to the Pub/Sub topic.

• Service Account for Lacework - To provide Lacework read-only access to Google Cloud with the following roles:

  • Roles for Audit Log integration:

    For organization level integration	For project level integration
    roles/monitoring.viewer	roles/monitoring.viewer
    roles/pubsub.subscriber	roles/pubsub.subscriber
    roles/browser

  • Roles for Configuration integration:

    For organization level integration	For project level integration
    roles/browser	roles/browser
    roles/cloudasset.viewer	roles/cloudasset.viewer
    roles/iam.securityReviewer	roles/iam.securityReviewer
    roles/lwOrgComplianceRole custom IAM role containing the following permissions: bigquery.datasets.get compute.projects.get compute.sslPolicies.get pubsub.topics.get storage.buckets.get	roles/lwComplianceRole custom IAM role containing the following permissions: bigquery.datasets.get compute.projects.get compute.sslPolicies.get pubsub.topics.get storage.buckets.get
    roles/resourcemanager.organizationViewer

For organization level integrations, follow the steps in Integrate Google Cloud at the Organization Level.

For project level integrations, follow the steps in Integrate Google Cloud at the Project level.

note

If you are setting up new Google audit logging (instead of leveraging existing Google audit logging), Lacework recommends that you set a retention policy with a minimum of 7 days.

Prerequisites

• Google Cloud Shell - Google Cloud Shell inherits the permissions of the user running Cloud Shell. Before beginning, determine whether the integration between Google Cloud and Lacework will be at the organization level or the project level, and then ensure the user account running Google Cloud Shell has the following permissions:

  • Organization level integrations
    • roles/owner - For an organization level integration, Lacework recommends that you create a new project specifically for the Lacework resources. The user account used to run Google Cloud Shell must have Owner permissions for that project.
    • roles/resourcemanager.organizationAdmin
    • roles/iam.organizationRoleAdmin
    • roles/logging.configWriter
  • Project level integrations
    • roles/owner - For a project level integration, Lacework recommends that you use the project being integrated to provision the required resources. The user account used to run Google Cloud Shell must have Owner permissions for every project being integrated into Lacework.

• Terraform provider for Lacework - You must configure the Terraform provider for Lacework to authenticate with the Lacework API using a Lacework API key and secret access key. For information on configuring the Terraform provider for Lacework, see Terraform for Lacework Overview.

• Ensure that you are deploying the integration to a supported Google Cloud region.

Terraform Module Dependencies

Lacework Terraform modules for Google Cloud have the following dependencies that will be installed when you run terraform init:

• lacework
• google
• random
• time

For more information on these dependencies, see the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry.

Configure Google Cloud Shell

Google Cloud Shell is an embedded terminal/command-line interface that can you can use within the Google Console. Cloud Shell comes with tools like Google Cloud SDK, gcloud CLI, and Terraform pre-installed to manage and automate your projects and resources.

Lacework provides a robust command-line interface that generates Terraform code, installs the Terraform CLI (if not already installed), and runs Terraform inside Cloud Shell.

Launch Google Cloud Shell

To open Google Cloud Shell, click the Activate Cloud Shell icon on the top right of the Google Cloud Console.

Install the Lacework CLI in Google Cloud Shell

The Terraform provider for Lacework leverages the configuration from the Lacework CLI to authenticate with the Lacework API and configure accounts.

1. Run the following commands to install the Lacework CLI:

   mkdir -p "$HOME"/bin  
   curl https://raw.githubusercontent.com/lacework/go-sdk/main/cli/install.sh | bash -s -- -d "$HOME"/bin 

2. After the Lacework CLI is installed, do the following:

   1. Type exit and press Enter to exit Cloud Shell.
   2. Launch Cloud Shell again.

The Lacework CLI is now ready to use.

note

If you have already installed the Lacework CLI, install it again to obtain the latest version of the CLI.

Create Lacework API Key

The Lacework CLI requires an API key and secret to authenticate with Lacework. Lacework API keys can be created by Lacework account administrators via the Lacework Console. For more information, go to API Access Keys.

1. Log in to the Lacework Console.
2. Click Settings > Configuration > API keys.
3. Click + Add New.
4. Enter a name for the key and an optional description.
5. Click Save.
6. Click the ... icon and then Download to save the API key file locally.

The contents of your API key contain a keyId secret, subAccount, and account:

{
  "keyId": "ACCOUNT_ABCEF01234559B9B07114E834D8570F567C824039756E03",
  "secret": "_abc1234e243a645bcf173ef55b837c19",
  "subAccount": "myaccount",
  "account": "myaccount.lacework.net"
}

Configure the Lacework CLI

To configure the Lacework CLI with the API key downloaded from the previous step, do one of the following:

• Run the lacework configure command in Cloud Shell and provide the following from the API key file:

  • Account: Lacework account URL (for example, YourAccount.lacework.net)
  • Access Key ID: API access key ID
  • Secret Access Key: API access secret

• Upload the API key file to Cloud Shell and run the following command to configure the Lacework CLI. For information about uploading files to Cloud Shell, see Manage Files with Cloud Shell.

  lacework configure -j /path/to/key.json

  For more information, see the Lacework CLI documentation.

Integrate Google Cloud at the Organization Level

This section describes how to integrate Google Cloud and Lacework for analysis of Cloud Audit Logs and configuration assessment at the organization level.

Create a Google Cloud Project

When you create an integration at the Google Cloud organization level, Lacework recommends having a dedicated project to provision the required resources for the integration between Google Cloud and Lacework. Create the project before running Terraform.

1. Log in to the Google Cloud Console.
2. Click the navigation menu in the top left and go to IAM & Admin > Create a Project.
3. Enter the project name, select a billing account, and select the organization you are integrating.
4. Click Create to create the new project.

Add Project Owner Permissions to User Account

Do the following to add role/owner permission on the project for the user running Cloud Shell.

1. In the Google Cloud Console, select the project you created for Lacework resources.
2. Click the navigation menu and go to IAM & Admin > IAM.
3. Filter the list and find the user account that will be used to run Cloud Shell.
4. Click the Edit principal icon and then click + Add Another Role to add the Owner role for the user account.
5. Click Save.

Run the Lacework CLI in Cloud Shell

1. In Cloud Shell, run the lacework generate cloud-account gcp Lacework CLI command to generate and execute the Terraform code for the integration.

   lacework generate cloud-account gcp  \
    --configuration --configuration_integration_name ConfigIntegName \
    --audit_log --use_pub_sub --audit_log_integration_name AuditLogIntegName \
    --organization_integration         \
    --organization_id OrganizationId   \
    --project_id ProjectId             \
    --noninteractive

   Where:

   • ConfigIntegName is the name of the configuration integration.
   • AuditLogIntegName is the name of the audit log integration.
   • OrganizationId is the ID of the Google Cloud organization being integrated.
   • ProjectId is the ID of the project you created for Lacework resources.

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.

Validate the Configuration

To confirm that the integrations are working, use the Lacework CLI or log in to the Lacework Console.

• To validate using the CLI, run the lacework cloud-account list command in Cloud Shell. You should see two integration types: GcpCfg for the Configuration integration, and GcpAlPubSub for the Audit Log integration.
• To validate using the Lacework Console, go to Settings > Integrations > Cloud Accounts. You should see two integration types: Configuration for the Configuration integration, and Audit Log (PubSub) for the Audit Log integration.

info

Google Apps Script Projects are hidden by default. Contact Lacework Support if you want to enable visibility of these projects.

Integrate Google Cloud at the Project Level

This section describes how to integrate Google Cloud and Lacework for analysis of Cloud Audit Logs and configuration assessment at the project level.

In this method Terraform provisions all the required resources in the project being integrated into Lacework.

Configure Project Owner Permissions to User Account

Do the following to configure role/owner for the user running Google Cloud Shell on the project being integrated to Lacework.

1. In the Google Cloud Console, select the project being integrated.
2. Click the navigation menu and go to IAM & Admin > IAM.
3. Filter the list and find the user account that will be used to run Cloud Shell.
4. Click the Edit principal icon and then click + Add Another Role to add the Owner role for the user account.
5. Click Save.

Run the Lacework CLI in Cloud Shell

1. In Cloud Shell, run the lacework generate cloud-account gcp Lacework CLI command to generate and execute the Terraform code for the integration.

   lacework generate cloud-account gcp  \
     --configuration --configuration_integration_name ConfigIntegName \
     --audit_log --use_pub_sub --audit_log_integration_name AuditLogIntegName \
     --project_id ProjectId  \
     --noninteractive 

   Where:

   • ConfigIntegName is the name of the configuration integration.
   • AuditLogIntegName is the name of the audit log integration.
   • ProjectId is the ID of the project being integrated.

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.

Validate the Configuration

To confirm that the integrations are working, use the Lacework CLI or log in to the Lacework Console.

• To validate using the CLI, run the lacework cloud-account list command in Cloud Shell. You should see two integration types: GcpCfg for the Configuration integration, and GcpAlPubSub for the Audit Log integration.
• To validate using the Lacework Console, go to Settings > Integrations > Cloud Accounts. You should see two integration types: Configuration for the Configuration integration, and Audit Log (PubSub) for the Audit Log integration.

info

Google Apps Script Projects are hidden by default. Contact Lacework Support if you want to enable visibility of these projects.

Integrate Multiple Projects with Terraform

To integrate multiple projects with Lacework individually:

1. Run the following lacework generate cloud-account gcp Lacework CLI command:

   lacework generate cloud-account gcp  \
     --configuration \
     --audit_log --use_pub_sub \
     --project_id ProjectId             \
     --projects ProjectId1,ProjectId2   \
     --noninteractive 

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.