Managing Secret Manager with Terraform
Terraform is a popular tool for managing infrastructure configurations as code, but what if your infrastructure needs to create or delete secrets like API keys or credentials? Google Secret Manager is a Google Cloud service that stores API keys, passwords, certificates, and other sensitive data. It provides convenience while improving security. This post explores how to manage Secret Manager secrets with Terraform.
The full sample code for this post is available on GitHub.
Setup
To get started, create a Google Cloud account, install the Cloud SDK, and install Terraform for your device. If you have not already done so, authenticate to the Cloud SDK:
$ gcloud auth application-default login
This will open a browser and prompt a sign in for your Google account. This only needs to be done once per device.
Create a new folder in which to create your Terraform configurations. All future commands will be run from this folder and files will be created in this folder.
$ mkdir terraform-manage-secret-manager
$ cd terraform-manage-secret-manager
Configurations
Create a file named versions.tf
that define the version constraints.
terraform {
required_version = ">= 0.12"
required_providers {
google-beta = ">= 3.8"
}
}
This file creates two constraints:
terraform
- This is the actual Terraform binary version. We require version 12.0+.google-beta
- This is the Google beta provider for Terraform. We require 3.8+ because Secret Manager support was added in version 3.8.0. Note that we are using the beta provider. At the time of this writing, Secret Manager is in beta, therefore we use the beta provider.
Create a file named main.tf
and configure the Google provider stanza:
provider "google-beta" {
project = "YOUR_PROJECT_ID" # replace with your project ID
}
Enable the Secret Manager API. This only needs to be done once per project, but it is an idempotent operation.
resource "google_project_service" "secretmanager" {
provider = google-beta
service = "secretmanager.googleapis.com"
}
Create a Secret Manager secret named "my-secret"
with an automatic replication policy:
resource "google_secret_manager_secret" "my-secret" {
provider = google-beta
secret_id = "my-secret"
replication {
automatic = true
}
depends_on = [google_project_service.secretmanager]
}
Create a new version of the secret:
resource "google_secret_manager_secret_version" "my-secret-1" {
provider = google-beta
secret = google_secret_manager_secret.my-secret.id
secret_data = "my super secret data"
}
Grant a user or service account IAM permissions to access the secret:
resource "google_secret_manager_secret_iam_member" "my-app" {
provider = google-beta
secret_id = google_secret_manager_secret.my-secret.id
role = "roles/secretmanager.secretAccessor"
member = "user:foo@bar.com" # or serviceAccount:my-app@...
}
Execute Terraform
Now that the configurations are written, initialize Terraform.
$ terraform init
Next, plan the changes.
$ terraform plan
The output will show the actions Terraform is going to take.
Plan: 4 to add, 0 to change, 0 to destroy.
Now apply the changes. Terraform will automatically handle the order of operations.
$ terraform apply
The output will show the progress and eventually the URL.
google_project_service.secretmanager: Creating...
google_project_service.secretmanager: Still creating... [10s elapsed]
google_project_service.secretmanager: Creation complete after 20s [id=project/secretmanager.googleapis.com]
google_secret_manager_secret.my-secret: Creating...
google_secret_manager_secret.my-secret: Creation complete after 3s [id=projects/project/secrets/my-secret]
google_secret_manager_secret_iam_member.my-app: Creating...
google_secret_manager_secret_version.my-secret-1: Creating...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Security note
When you access the secret, the plaintext value will be stored in the Terraform State. This means anyone with access to the state file will be able to see the secret. This is a known issue in Terraform, and you can read more about it on this Terraform GitHub issue.
To help mitigate this, we strongly recommend using Remote State and carefully auditing access to secrets.
Tearing it down
One of the beauties of Terraform is its self-contained nature. Even though Secret Manager pricing is relative inexpensive, some folks want to cleanup things when they are done.
When you are done experimenting, tear everything down with a single Terraform command!
$ terraform destroy
Confirm "yes" at the prompt.
Next steps
This post only begins to scratch the surface of the functionality offered by Secret Manager with Terraform. Check out the Secret Manager Terraform resource documentation for a full list of all arguments and attributes available. You can also learn more about Secret Manager and Secret Manager pricing on the Google Cloud website.
About Seth
Seth Vargo is a Distinguished Software Engineer at Google. Previously he worked at HashiCorp, Chef Software, CustomInk, and some Pittsburgh-based startups. He is the author of Learning Chef and is passionate about reducing inequality in technology. When he is not writing, working on open source, teaching, or speaking at conferences, Seth advises non-profits.