Playing around with Workload Identity on GKE

October 30, 2019 3 minutes read

TechStack

Google recently beta-released Workload Identity, a solution to use Google Service Accounts (GSA) from Workloads in Kubernetes using Kubernetes Service Accounts (KSA).

Setting this up requires some steps and using it with other technologies requires some wiring. My current setup involves Terraform for cloud infrastructure setup, Helm charts to bundle applications and FluxCD to manage GKE clusters in a GitOps way.

Setup of GSA and KSA

So you have a GKE cluster with Workload Identity enabled on the cluster and Node Pool. Now how to create a KSA which can use a GSA? Of course, write a Terraform module!

provider "google" {}
provider "kubernetes" {}

variable "name" {
  description = "Name of the Google and Kubernetes Account that is created"
  type        = string
}

variable "namespace" {
  description = "Kubernetes namespace where the SA is created"
  type        = string
}

data "google_project" "this" {}

resource "google_service_account" "this" {
  account_id = var.name
}

resource "kubernetes_service_account" "this" {
  metadata {
    annotations = {
      managed_by_terraform = true

      "iam.gke.io/gcp-service-account" = google_service_account.this.email
    }

    name      = var.name
    namespace = var.namespace
  }

  automount_service_account_token = true
}


resource "google_service_account_iam_member" "workload_identity_user" {
  service_account_id = google_service_account.this.name
  role               = "roles/iam.workloadIdentityUser"
  member             = "serviceAccount:${data.google_project.this.project_id}.svc.id.goog[${var.namespace}/${kubernetes_service_account.this.metadata.0.name}]"
}

output "kubernetes_service_account" {
  description = "The created kubernetes service account"
  value       = kubernetes_service_account.this
}

output "google_service_account" {
  description = "The created google service account"
  value       = google_service_account.this
}

Now you can use this module wherever you need to create a KSA for a workload that has to access some Google API:

module "workload_identity_service_account" {
  source = "../workload_identity_service_account" # or something else

  providers = {
    google     = google
    kubernetes = kubernetes
  }

  name      = "workload-name"
  namespace = "default"
}

And now to the really fun part: automagically making a Helm chart using this KSA when managed via Flux!

So whats the problem? Terraform knows how our KSA is named, our Helm chart needs this info. Hopefully our Helm chart allows us to pass something akin to serviceAccount.name: whatever (if not, make it so!).

Thankfully FluxCD allows us, to get values from one (or multiple) configMap and pass them to a helmRelease:

apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
  name: workload
  namespace: default
spec:
  releaseName: workload
  chart:
    git: SOME_GIT_REPO
    path: charts/workload
  valuesFrom:
    - configMapKeyRef:
        name: workload-values
    - configMapKeyRef:
        name: other-values
  values:
    otherStuff: true

So let’s also create this configMap from Terraform:


resource "kubernetes_config_map" "workload_values" {
  metadata {
    name      = "workload-values"
    namespace = "default"
  }

  data = {
    "values.yaml" = <<-YAML
    app:
      serviceAccount:
        create: false
        name: ${module.workload_identity_service_account.kubernetes_service_account.metadata.0.name}
    YAML
  }
}

Conclusion

With this setup the following happens:

Terraform creates a KSA and a GSA, the KSA is allowed to impersonate the GSA.
Terraform will also create a configMap which holds values for the Helm chart.
FluxCD picks up these values, merges them with others and deploys the helmRelease.
The workload will now identify as the GSA when calling Google APIs

Therefore you can now also use Terraform to grant IAM permissions to the GSA, e.g.:

resource "google_project_iam_member" "storage_object_viewer" {
  project = data.google_project.this.project_id
  role    = "roles/storage.objectViewer"
  member  = "serviceAccount:${module.workload_identity_service_account.google_service_account.email}"
}

TechStack

Setup of GSA and KSA

Sharing Terraform knowledge with Flux

Conclusion