Pub/Sub-Based Google Cloud Integration - Terraform from Any Supported Host

tip

To integrate with Google Cloud, Lacework recommends using Guided Configuration, a wizard-like interface that takes your inputs and generates a script that downloads and sets up all necessary Lacework CLI and Terraform components to create the integration non-interactively.

To use guided configuration:

1. In the Lacework Console go to Settings > Integrations > Cloud accounts.
2. Click + Add New.
3. Click Google Cloud Platform and select Guided configuration.

Alternatively, follow the steps in this topic for the following methods:

• Use the Lacework CLI to generate and run Terraform code.
• Create the main.tf file manually and run Terraform from Google Cloud Shell. This may be required for complex integration scenarios.

Overview

Lacework integrates with Google Cloud Platform (GCP) to analyze Cloud Audit Logs and assess cloud resource configurations at the organization level or at the project level. The Lacework platform ingests configuration and audit log information from Google Cloud and reports alerts for anomalous behavior.

• Organization level integrations cover all the existing projects in the organization. After you create the integration, any new projects you create are automatically included in the integration and any projects you delete are automatically excluded.
• Project level integrations cover only a specific project. You must set up a new integration, if required, for any new project you create.

This topic describes how you can set up an organization level or project level configuration and Pub/Sub-based audit log integration by running Lacework Terraform modules on any host supported by Terraform. Lacework recommends this method if you plan to continue using Terraform to manage the integrations or store the state of the configuration in a source control management tool such as Git.

Lacework also supports running Terraform from Google Cloud Shell, which is more suitable for one off integrations where you do not plan to continue using Terraform to manage the integrations. For more information see Pub/Sub-based Integration From Google Cloud Shell.

info

If you want to use Terraform to migrate an existing Storage-based audit log integration to a Pub/Sub-based audit log integration, see Migrate From Storage-Based to Pub/Sub-Based Google Cloud Audit Log Integration Using Terraform.

If you are new to the Lacework Terraform Provider or Lacework Terraform Modules, read Terraform for Lacework Overview to learn the basics on how to configure the provider.

Google Cloud Resources

To integrate at the organization or project level, Lacework requires the following resources to be provisioned in Google Cloud. You must create the project and service account in Google Cloud. The Lacework Terraform modules automatically provision the log routing sink, Pub/Sub topic, and Pub/Sub subscription with the required permissions in Google Cloud.

• Google Cloud Project - A project that contains the resources required for the integration with billing enabled.

  • For an organization level integration, Lacework recommends that you create a new project specifically for the Lacework resources.
  • For a project level integration, you can provision the required Lacework resources within the project being integrated.

• Google Pub/Sub Topic - For streaming Cloud Audit Log events to the Lacework platform.

• Google Pub/Sub Subscription - Subscription for Lacework to pull the Audit Log events from the Pub/Sub topic.

• Google Log Routing Sink - To route Cloud Audit Logs to the Pub/Sub topic.

• Service Account for Lacework - To provide Lacework read-only access to Google Cloud with the following roles:

  • Roles for Audit Log integration:

    For organization level integration	For project level integration
    roles/monitoring.viewer	roles/monitoring.viewer
    roles/pubsub.subscriber	roles/pubsub.subscriber
    roles/browser

  • Roles for Configuration integration:

    For organization level integration	For project level integration
    roles/browser	roles/browser
    roles/cloudasset.viewer	roles/cloudasset.viewer
    roles/iam.securityReviewer	roles/iam.securityReviewer
    roles/lwOrgComplianceRole custom IAM role containing the following permissions: bigquery.datasets.get compute.projects.get compute.sslPolicies.get pubsub.topics.get storage.buckets.get	roles/lwComplianceRole custom IAM role containing the following permissions: bigquery.datasets.get compute.projects.get compute.sslPolicies.get pubsub.topics.get storage.buckets.get
    roles/resourcemanager.organizationViewer

For organization level integrations, follow the steps in Integrate Google Cloud at the Organization Level.

For project level integrations, follow the steps in Integrate Google Cloud at the Project level.

note

If you are setting up new Google audit logging (instead of leveraging existing Google audit logging), Lacework recommends that you set a retention policy with a minimum of 7 days.

Prerequisites

• Google Cloud Console - Administrator access to Google Cloud Console to create projects, service accounts, grant access, and provision resources.
• gcloud CLI - To configure resources in Google Cloud.
• Lacework Console - Org admin or Account admin access to the Lacework Console is required to create API keys and configure integrations with Lacework.
• Lacework CLI - To generate and execute the Terraform code for the integration.
• Terraform - Lacework Terraform projects support Terraform versions ~> 0.14, ~> 0.15, ~> 1.0, and ~> 1.1.
• Ensure that you are deploying the integration to a supported Google Cloud region.

Terraform Module Dependencies

Lacework Terraform modules for Google Cloud have the following dependencies that will be installed when you run terraform init:

• lacework
• google
• random
• time

For more information on these dependencies, see the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry.

Integrate Google Cloud at the Organization Level

This section describes how to integrate Google Cloud and Lacework for analysis of Cloud Audit Logs and configuration assessment at the organization level.

Create a Google Cloud Project

When you create an integration at the Google Cloud organization level, Lacework recommends having a dedicated project to provision the required resources for the integration between Google Cloud and Lacework. Create the project before running Terraform.

1. Log in to Google Cloud Console.
2. Click the navigation menu and go to IAM & Admin > Create a Project.
3. Enter the project name, select a billing account, and select the organization you are integrating.
4. Click Create to create the new project.

Create a Service Account for Terraform

To integrate Google Cloud and Lacework at the organization level, Terraform needs a service account with the following permissions. If you already have a service account configured with these permissions, along with a configured key, skip to the next section.

• roles/owner
• roles/resourcemanager.organizationAdmin
• roles/iam.organizationRoleAdmin
• roles/logging.configWriter

Do the following to create a service account within the project created for Lacework and give the account Owner permissions to the project.

1. In the Google Cloud Console, select the project you created for Lacework resources.
2. Click the navigation menu and go to IAM & Admin > Service Accounts.
3. Click Create Service Account.
4. Enter the service account name (for example, terraform-provisioning), add a description, and click Create and Continue.
5. In the Grant this service account access to project section, give the service account Owner permissions to the project, and click Continue.
6. Click Done to create the service account.

Add Service Account to Google Cloud Organization

You must add the service account to the Google Cloud organization being integrated with Lacework, and add the required organization level permissions to the service account for Terraform to be able to configure the organization being integrated.

1. In the Google Cloud Console, select the organization you are integrating with Lacework.
2. Click the navigation menu and go to IAM & Admin > IAM.
3. Click the + Grant Access button.
4. In the Add principals section, search for the service account and select it.
5. In the Assign roles section, add the following roles for the service account:
   • Organization Administrator
   • Organization Role Administrator
   • Logs Configuration Writer
6. Click Save.

Create Service Account Key

To run Terraform locally, you must create and download a key for the service account created in the previous step. This section describes how to create a service account key and download it to your local system as a JSON file.

1. In the Google Cloud Console, select the project you created for Lacework resources.
2. Click the navigation menu and go to IAM & Admin > Service Accounts.
3. Click the Actions menu next to the service account and select Manage keys.
4. Click Add Key > Create new key.
5. Select JSON as the key type.
6. Click Create to download the key.

Configure the Lacework CLI

Ensure you have the Lacework CLI installed and configured to the Lacework account you plan to integrate.

1. Install the Lacework CLI.
2. Create your API key.
3. Configure your Lacework CLI.

note

If you have already installed the Lacework CLI, install it again to obtain the latest version of the CLI.

Run the Lacework CLI

1. Run the lacework generate cloud-account gcp Lacework CLI command to generate and execute the Terraform code for the integration.

   lacework generate cloud-account gcp  \
     --configuration --configuration_integration_name ConfigIntegName \
     --audit_log --use_pub_sub --audit_log_integration_name AuditLogIntegName \
     --organization_integration         \
     --organization_id OrganizationId   \
     --project_id ProjectId             \
     --service_account_credentials PathToServiceAccountKeyFile \
     --noninteractive 

   Where:

   • ConfigIntegName is the name of the configuration integration.
   • AuditLogIntegName is the name of the audit log integration.
   • OrganizationId is the ID of the Google Cloud organization being integrated.
   • ProjectId is the ID of the project you created for Lacework resources.
   • PathToServiceAccountKeyFile is the path to the service account key JSON file.

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.

Validate the Configuration

To confirm that the integrations are working, use the Lacework CLI or log in to the Lacework Console.

• To validate using the Lacework CLI, run the lacework cloud-account list command. You should see two integration types: GcpCfg for the Configuration integration, and GcpAlPubSub for the Audit Log integration.
• To validate using the Lacework Console, go to Settings > Integrations > Cloud Accounts. You should see two integration types: Configuration for the Configuration integration, and Audit Log (PubSub) for the Audit Log integration.

info

Google Apps Script Projects are hidden by default. Contact Lacework Support if you want to enable visibility of these projects.

Integrate Google Cloud at the Project Level

This section describes how to integrate Google Cloud and Lacework for analysis of Cloud Audit Logs and configuration assessment at the project level.

In this method, Terraform provisions all the required resources in the project being integrated into Lacework.

Create a Service Account for Terraform

To integrate Google Cloud and Lacework at the project level, Terraform needs a service account with the following permission for the project being integrated. If you already have a service account configured with this permission, along with a configured key, skip to the next section.

• roles/owner

Do the following to create a service account within the project and give the account Owner permissions to the project.

1. Log in to the Google Cloud Console.
2. Select the project being integrated with Lacework.
3. Click the navigation menu and go to IAM & Admin > Service Accounts.
4. Click Create Service Account.
5. Enter the service account name (for example, terraform-provisioning), add a description, and click Create and Continue.
6. In the Grant this service account access to project section, give the service account Owner permissions to the project, and click Continue.
7. Click Done to create the service account.

Create Service Account Key

To run Terraform locally, you must create and download a key for the service account created in the previous step. This section describes how to create a service account key and download it to your local system as a JSON file.

1. In the Google Cloud Console, select the project created for Lacework resources.
2. Click the navigation menu and go to IAM & Admin > Service Accounts.
3. Click the Actions menu next to the service account and select Manage keys.
4. Click Add Key > Create new key.
5. Select JSON as the key type.
6. Click Create to download the key.

Configure the Lacework CLI

Ensure you have the Lacework CLI installed and configured to the Lacework account you plan to integrate.

1. Install the Lacework CLI.
2. Create your API key.
3. Configure your Lacework CLI.

note

If you have already installed the Lacework CLI, install it again to obtain the latest version of the CLI.

Run the Lacework CLI

1. Run the lacework generate cloud-account gcp Lacework CLI command to generate and execute the Terraform code for the integration.

   lacework generate cloud-account gcp  \
     --configuration --configuration_integration_name ConfigIntegName \
     --audit_log --use_pub_sub --audit_log_integration_name AuditLogIntegName \
     --project_id ProjectId   \
     --service_account_credentials PathToServiceAccountKeyFile \
     --noninteractive

   Where:

   • ConfigIntegName is the name of the configuration integration.
   • AuditLogIntegName is the name of the audit log integration.
   • ProjectId is the ID of the project being integrated.
   • PathToServiceAccountKeyFile is the path to the service account key JSON file.

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.

Validate the Configuration

To confirm that the integrations are working, use the Lacework CLI or log in to the Lacework Console.

• To validate using the Lacework CLI, run the lacework cloud-account list command. You should see two integration types: GcpCfg for the Configuration integration, and GcpAlPubSub for the Audit Log integration.
• To validate using the Lacework Console, go to Settings > Integrations > Cloud Accounts. You should see two integration types: Configuration for the Configuration integration, and Audit Log (PubSub) for the Audit Log integration.

info

Google Apps Script Projects are hidden by default. Contact Lacework Support if you want to enable visibility of these projects.

Integrate Multiple Google Cloud Projects Using Terraform

To integrate multiple projects with Lacework individually:

1. Run the following lacework generate cloud-account gcp Lacework CLI command:

   lacework generate cloud-account gcp  \
     --configuration \
     --audit_log --use_pub_sub \
     --project_id ProjectId             \
     --projects ProjectId1,ProjectId2   \
     --noninteractive

   The Terraform files are created in the ~/lacework/gcp directory.

2. Navigate to the ~/lacework/gcp directory.

3. Run terraform plan and review the changes that will be applied.

4. Once satisfied with the changes that will be applied, run terraform apply to execute Terraform.

note

If you are creating the main.tf file manually, you can use Terraform inputs to customize Lacework Terraform modules. See the documentation for the terraform-gcp-config and terraform-gcp-pub-sub-audit-log modules on the Lacework Terraform Registry for the complete list of inputs for each module.

Troubleshooting

How do I fix this issue: Error: no project value set?

If you see the following error:
Error: no project value set. project_id must be set at the resource level, or a default project value must be specified on the provider

Add a project to the Google provider. See Google provider documentation for details.