Accessing Secret Manager from Terraform

Terraform is a popular tool for managing infrastructure configurations as code, but what if your infrastructure needs secrets like API keys or credentials? Google Secret Manager is a Google Cloud service that stores API keys, passwords, certificates, and other sensitive data. It provides convenience while improving security. This post explores how to access Secret Manager secrets from Terraform.

The full sample code for this post is available on GitHub.

Setup

To get started, create a Google Cloud account, install the Cloud SDK, and install Terraform for your device. If you have not already done so, authenticate to the Cloud SDK:

$ gcloud auth application-default login

This will open a browser and prompt a sign in for your Google account. This only needs to be done once per device.

Create a new folder in which to create your Terraform configurations. All future commands will be run from this folder and files will be created in this folder.

$ mkdir terraform-access-secret-manager

$ cd terraform-access-secret-manager

Finally, create a secret - we will access this secret from Terraform:

$ echo -n "my super secret data" | gcloud beta secrets create "my-secret" \
    --data-file - \
    --replication-policy "automatic"

Configurations

Create a file named versions.tf that define the version constraints.

terraform {
  required_version = ">= 0.12"

  required_providers {
    google-beta = ">= 3.8"
  }
}

This file creates two constraints:

• terraform - This is the actual Terraform binary version. We require version 12.0+.

• google-beta - This is the Google beta provider for Terraform. We require 3.8+ because Secret Manager support was added in version 3.8.0. Note that we are using the beta provider. At the time of this writing, Secret Manager is in beta, therefore we use the beta provider.

Create a file named main.tf and configure the Google provider stanza:

provider "google-beta" {
  project = "YOUR_PROJECT_ID" # replace with your project ID
}

To access a secret from Google Secret Manager, use the google_secret_manager_secret_version data source:

data "google_secret_manager_secret_version" "my-secret" {
  provider = google-beta

  secret  = "my-secret"
  version = "1"
}

This example accesses version "1" of the Secret Manager secret named "my-secret". If you omit the version argument, Terraform will fetch the latest version. However, most scenarios including Infrastructure as Code, we strongly recommend pinning to a specific version.

Finally, for demonstration purposes, create an output that prints the value of the secret:

output "secret" {
  value = data.google_secret_manager_secret_version.my-secret.secret_data
}

Execute Terraform

Now that the configurations are written, initialize Terraform.

$ terraform init

Next, plan the changes.

$ terraform plan

The output will show the actions Terraform is going to take.

Plan: 1 to add, 0 to change, 0 to destroy.

Now apply the changes. Terraform will automatically handle the order of operations.

$ terraform apply

The output will show the progress and eventually the secret value.

# ...

Outputs:

secret = my super secret data

Security note

When you access the secret, the plaintext value will be stored in the Terraform State. This means anyone with access to the state file will be able to see the secret. This is a known issue in Terraform, and you can read more about it on this Terraform GitHub issue.

To help mitigate this, we strongly recommend using Remote State and carefully auditing access to secrets.

Next steps

This post only begins to scratch the surface of the functionality offered by the google_cloud_secret_manager_secret_version data source in Terraform. Check out the Secret Manager Terraform resource documentation for a full list of all arguments and attributes available. You can also learn more about Secret Manager and Secret Manager pricing on the Google Cloud website.