lacework-global-297
Encrypt Dataproc Cluster using Customer-Managed Encryption Key (CMEK) (Automated)
Profile Applicability
• Level 2
Description
When you use Dataproc, Persistent Disks (PDs) associated with the Compute Engine VMs in your cluster and a Cloud Storage staging bucket store cluster and job data. A Google-generated Data Encryption Key (DEK) and Key Encryption Key (KEK) encrypts the PD and bucket data. The CMEK feature allows you to create, use, and revoke the key encryption key (KEK). Google still controls the data encryption key (DEK).
Rationale
"Cloud services offer the ability to protect data related to those services using encryption keys managed by the customer within Cloud KMS. These encryption keys are called customer-managed encryption keys (CMEK). When you protect data in Google Cloud services with CMEK, the CMEK key is within your control.
Audit
From Console
- Login to the GCP Console and navigate to the Dataproc Cluster page by visiting https://console.cloud.google.com/dataproc/clusters.
- Select the project from the project dropdown list.
- On the
Dataproc Clusters
page, select the cluster and click on the Name attribute value that you want to examine. - On the
details
page, select theConfigurations
tab. - On the
Configurations
tab, check theEncryption type
configuration attribute value. If the value is set toGoogle-managed key
, then Dataproc Cluster is not encrypted with Customer managed encryption keys.
Repeat step no. 3 - 5 for other Dataproc Clusters available in the selected project.
- Change the project from the project dropdown list and repeat the audit procedure for other projects.
From Command Line
- Run clusters list command to list all the Dataproc Clusters available in the region:
gcloud dataproc clusters list --region='us-central1'
- Run clusters describe command to get the key details of the selected cluster:
gcloud dataproc clusters describe <cluster_name> --region=us-central1 --flatten=config.encryptionConfig.gcePdKmsKeyName
- If the above command output return "null", then the selected cluster is not encrypted with Customer managed encryption keys.
- Repeat step no. 2 and 3 for other Dataproc Clusters available in the selected region. Change the region by updating --region and repeat step no. 2 for other clusters available in the project. Change the project by running the below command and repeat the audit procedure for other Dataproc clusters available in other projects:
gcloud config set project <project_ID>"
Remediation
From Console
- Login to the GCP Console and navigate to the Dataproc Cluster page by visiting: https://console.cloud.google.com/dataproc/clusters.
- Select the project from the projects dropdown list.
- On the
Dataproc Cluster
page, click theCreate Cluster
to create a new cluster with Customer managed encryption keys. - On
Create a cluster
page, perform below steps:
- Inside the
Set up cluster
section, perform below steps:- In the
Name
text box, provide a name for your cluster. - From
Location
select the location in which you want to deploy a cluster. - Configure other configurations as per your requirements.
- In the
- Inside the
Configure Nodes
andCustomize cluster
section, configure the settings as per your requirements. - Inside the
Manage security
section, perform below steps:- From
Encryption
, selectCustomer-managed key
. - Select a customer-managed key from dropdown list.
- Ensure that the selected Key Management Service (KMS) Key have Cloud KMS CryptoKey Encrypter/Decrypter role assign to Dataproc Cluster service account ("serviceAccount:service-<project_number>@compute-system.iam.gserviceaccount.com").
- Click
Create
to create a cluster. - After cluster creation, migrate all your workloads from the older cluster to the new cluster and delete the old cluster by performing the below steps:
- On the
Clusters
page, select the old cluster and clickDelete cluster
. - On the
Confirm deletion
window, clickConfirm
to delete the cluster. - Repeat the preceding step for other Dataproc clusters available in the selected project.
- Change the project from the project dropdown list and repeat the remediation procedure for other Dataproc clusters available in other projects.
- From
From Command Line
Before creating cluster ensure that the selected KMS Key have Cloud KMS CryptoKey Encrypter/Decrypter role assign to Dataproc Cluster service account ("serviceAccount:service-<project_number>@compute-system.iam.gserviceaccount.com"). Run clusters create command to create new cluster with customer-managed key:
gcloud dataproc clusters create <cluster_name> --region=<region> --gce-pd-kms-key=<key_resource_name>
The preceding command creates a new cluster in the selected region.
After cluster creation, migrate all your workloads from the older cluster to the new cluster and Run clusters delete command to delete cluster:
gcloud dataproc clusters delete <cluster_name> --region=<region>
Repeat step no. 1 to create a new Dataproc cluster. Change the project by running the below command and repeat the remediation procedure for other projects:
gcloud config set project <project_ID>
References
https://cloud.google.com/docs/security/encryption/default-encryption