Where it began: GCP and Kubernetes

Published: Jul 3, 2019 by Isaac Johnson

It’s probably about time we cover the other leading cloud compute provider - Google. Google gave birth to Kubernetes as an internal project called “Borg” in 2003 and eventually it grew to the point of Open Sourcing it 10 years later as the k8s we know and love today.

So as one might expect, GCP has a pretty solid Kubernetes offering.

Creating from the UI:

Here we can create a cluster:

Here we can pick “Standard cluster” or “Your first cluster”. For the purpose of this demo, we’ll use the cheaper “Your first cluster”.

What is particularly helpful is the REST and commandline equivalency is there at the bottom:

gcloud beta container --project "api-project-1234512345" clusters create "your-first-cluster-1" --zone "us-central1-a" --no-enable-basic-auth --cluster-version "1.13.6-gke.13" --machine-type "g1-small" --image-type "COS" --disk-type "pd-standard" --disk-size "30" --scopes "https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" --num-nodes "1" --no-enable-cloud-logging --no-enable-cloud-monitoring --no-enable-ip-alias --network "projects/api-project-1234512345/global/networks/default" --subnetwork "projects/api-project-1234512345/regions/us-central1/subnetworks/default" --addons HorizontalPodAutoscaling,HttpLoadBalancing --enable-autoupgrade --enable-autorepair

Clicking create will start the cluster creation process:

Once created, click connect

You will then get links to the dashboard and the gcloud CLI to get your kubeconfig.

gcloud container clusters get-credentials your-first-cluster-1 --zone us-central1-a --project api-project-1234512345 gcloud container clusters get-credentials your-first-cluster-1 --zone us-central1-a --project api-project-1234512345

e.g.

$ export PATH=/Users/isaac.johnson/Downloads/google-cloud-sdk/bin:$PATH
AHD-MBP13-048:bin isaac.johnson$ gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&prompt=select_account&response_type=code&client_id=1234512345.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&access_type=offline


WARNING: `gcloud auth login` no longer writes application default credentials.
If you need to use ADC, see:
  gcloud auth application-default --help

You are now logged in as [isaac.johnson@cyberdyne.com].
Your current project is [None]. You can change this setting by running:
  $ gcloud config set project PROJECT_ID


To take a quick anonymous survey, run:
  $ gcloud alpha survey

$ gcloud container clusters get-credentials your-first-cluster-1 --zone us-central1-a --project api-project-123451234
Fetching cluster endpoint and auth data.
kubeconfig entry generated for your-first-cluster-1.

and now we can access our cluster

$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system heapster-v1.6.0-beta.1-6b748f594f-pgptw 2/2 Running 0 8m20s
kube-system kube-dns-78d5dcc986-gq6f2 4/4 Running 0 8m36s
kube-system kube-dns-autoscaler-bb58c6784-twtld 1/1 Running 0 8m14s
kube-system kube-proxy-gke-your-first-cluster-1-pool-1-de77438b-1g7p 1/1 Running 0 8m27s
kube-system l7-default-backend-fd59995cd-2bjr9 1/1 Running 0 8m37s
kube-system metrics-server-v0.3.1-57c75779f-dvf7d 2/2 Running 0 8m20s

The GCP dashboard will show us basic details of our cluster:

Metrics with StackDriver

You can enable stack driver metrics as well:

You can view metrics like CPU:

Lastly, we can also delete the GKE cluster from the UI:

You can use the CLI as well:

$ gcloud container clusters delete your-first-cluster-1 --zone us-central1-a --project api-project-1234512345
The following clusters will be deleted.
 - [your-first-cluster-1] in [us-central1-a]

Do you want to continue (Y/n)? Y

Deleting cluster your-first-cluster-1...⠼     

Terraform

You can download your API key from GKE as a JSON:

Then before we do any terraform commands, set the location to the key json:

export GOOGLE_APPLICATION_CREDENTIALS="/Users/isaac.johnson/Downloads/API Project-207859f62628.json"

Terraform main.tf:

resource "google_container_cluster" "primary" {
  name = "idj-gke-2"
  location = "us-central1-a"
  project = "api-project-1234512345"
  initial_node_count = 3

  master_auth {
    username = "myuname"
    password = "MyPassword2Strong2Use"

    client_certificate_config {
      issue_client_certificate = false
    }
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]

    metadata = {
      disable-legacy-endpoints = "true"
    }

    labels = {
      foo = "bar"
    }

    tags = ["foo", "bar"]
  }

  timeouts {
    create = "30m"
    update = "40m"
  }
}

Terraform Plan:

Use plan to create the plan file and validate we have the right required vars set

$ terraform plan -out tfplan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + google_container_cluster.primary
      id: <computed>
      additional_zones.#: <computed>
      addons_config.#: <computed>
      cluster_autoscaling.#: <computed>
      cluster_ipv4_cidr: <computed>
      enable_binary_authorization: <computed>
      enable_kubernetes_alpha: "false"
      enable_legacy_abac: "false"
      enable_tpu: <computed>
      endpoint: <computed>
      initial_node_count: "3"
      instance_group_urls.#: <computed>
      ip_allocation_policy.#: <computed>
      location: "us-central1-a"
      logging_service: <computed>
      master_auth.#: "1"
      master_auth.0.client_certificate: <computed>
      master_auth.0.client_certificate_config.#: "1"
      master_auth.0.client_certificate_config.0.issue_client_certificate: "false"
      master_auth.0.client_key: <computed>
      master_auth.0.cluster_ca_certificate: <computed>
      master_auth.0.password: <sensitive>
      master_auth.0.username: "myuname"
      master_version: <computed>
      monitoring_service: <computed>
      name: "idj-gke-2"
      network: "default"
      network_policy.#: <computed>
      node_config.#: "1"
      node_config.0.disk_size_gb: <computed>
      node_config.0.disk_type: <computed>
      node_config.0.guest_accelerator.#: <computed>
      node_config.0.image_type: <computed>
      node_config.0.labels.%: "1"
      node_config.0.labels.foo: "bar"
      node_config.0.local_ssd_count: <computed>
      node_config.0.machine_type: <computed>
      node_config.0.metadata.%: "1"
      node_config.0.metadata.disable-legacy-endpoints: "true"
      node_config.0.oauth_scopes.#: "2"
      node_config.0.oauth_scopes.1277378754: "https://www.googleapis.com/auth/monitoring"
      node_config.0.oauth_scopes.172152165: "https://www.googleapis.com/auth/logging.write"
      node_config.0.preemptible: "false"
      node_config.0.service_account: <computed>
      node_config.0.tags.#: "2"
      node_config.0.tags.0: "foo"
      node_config.0.tags.1: "bar"
      node_locations.#: <computed>
      node_pool.#: <computed>
      node_version: <computed>
      project: "api-project-1234512345"
      region: <computed>
      services_ipv4_cidr: <computed>
      zone: <computed>


Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

This plan was saved to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

Terraform Apply:

$ terraform apply tfplan
google_container_cluster.primary: Creating...
  additional_zones.#: "" => "<computed>"
  addons_config.#: "" => "<computed>"
  cluster_autoscaling.#: "" => "<computed>"
  cluster_ipv4_cidr: "" => "<computed>"
  enable_binary_authorization: "" => "<computed>"
  enable_kubernetes_alpha: "" => "false"
  enable_legacy_abac: "" => "false"
  enable_tpu: "" => "<computed>"
  endpoint: "" => "<computed>"
  initial_node_count: "" => "3"
  instance_group_urls.#: "" => "<computed>"
  ip_allocation_policy.#: "" => "<computed>"
  location: "" => "us-central1-a"
  logging_service: "" => "<computed>"
  master_auth.#: "" => "1"
  master_auth.0.client_certificate: "" => "<computed>"
  master_auth.0.client_certificate_config.#: "" => "1"
  master_auth.0.client_certificate_config.0.issue_client_certificate: "" => "false"
  master_auth.0.client_key: "<sensitive>" => "<sensitive>"
  master_auth.0.cluster_ca_certificate: "" => "<computed>"
  master_auth.0.password: "<sensitive>" => "<sensitive>"
  master_auth.0.username: "" => "myuname"
  master_version: "" => "<computed>"
  monitoring_service: "" => "<computed>"
  name: "" => "idj-gke-2"
  network: "" => "default"
  network_policy.#: "" => "<computed>"
  node_config.#: "" => "1"
  node_config.0.disk_size_gb: "" => "<computed>"
  node_config.0.disk_type: "" => "<computed>"
  node_config.0.guest_accelerator.#: "" => "<computed>"
  node_config.0.image_type: "" => "<computed>"
  node_config.0.labels.%: "" => "1"
  node_config.0.labels.foo: "" => "bar"
  node_config.0.local_ssd_count: "" => "<computed>"
  node_config.0.machine_type: "" => "<computed>"
  node_config.0.metadata.%: "" => "1"
  node_config.0.metadata.disable-legacy-endpoints: "" => "true"
  node_config.0.oauth_scopes.#: "" => "2"
  node_config.0.oauth_scopes.1277378754: "" => "https://www.googleapis.com/auth/monitoring"
  node_config.0.oauth_scopes.172152165: "" => "https://www.googleapis.com/auth/logging.write"
  node_config.0.preemptible: "" => "false"
  node_config.0.service_account: "" => "<computed>"
  node_config.0.tags.#: "" => "2"
  node_config.0.tags.0: "" => "foo"
  node_config.0.tags.1: "" => "bar"
  node_locations.#: "" => "<computed>"
  node_pool.#: "" => "<computed>"
  node_version: "" => "<computed>"
  project: "" => "api-project-1234512345"
  region: "" => "<computed>"
  services_ipv4_cidr: "" => "<computed>"
  zone: "" => "<computed>"
google_container_cluster.primary: Still creating... (10s elapsed)
google_container_cluster.primary: Still creating... (20s elapsed)
google_container_cluster.primary: Still creating... (30s elapsed)
google_container_cluster.primary: Still creating... (40s elapsed)
google_container_cluster.primary: Still creating... (50s elapsed)
google_container_cluster.primary: Still creating... (1m0s elapsed)
google_container_cluster.primary: Still creating... (1m10s elapsed)
google_container_cluster.primary: Still creating... (1m20s elapsed)
google_container_cluster.primary: Still creating... (1m30s elapsed)
google_container_cluster.primary: Still creating... (1m40s elapsed)
google_container_cluster.primary: Still creating... (1m50s elapsed)
google_container_cluster.primary: Still creating... (2m0s elapsed)
google_container_cluster.primary: Still creating... (2m10s elapsed)
google_container_cluster.primary: Still creating... (2m20s elapsed)
google_container_cluster.primary: Still creating... (2m30s elapsed)
google_container_cluster.primary: Still creating... (2m40s elapsed)
google_container_cluster.primary: Still creating... (2m50s elapsed)
google_container_cluster.primary: Creation complete after 3m0s (ID: idj-gke-2)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

We can now see it:

Then we can also get the credentials and see a different set of pods from our Terraform based install:

$ gcloud container clusters get-credentials idj-gke-2 --zone us-central1-a --project api-project-1234512345
Fetching cluster endpoint and auth data.
kubeconfig entry generated for idj-gke-2.
AHD-MBP13-048:bin isaac.johnson$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system event-exporter-v0.2.4-5f7d5d7dd4-z64jg 2/2 Running 0 7m6s
kube-system fluentd-gcp-scaler-7b895cbc89-4c74g 1/1 Running 0 6m51s
kube-system fluentd-gcp-v3.2.0-69ksm 2/2 Running 0 6m11s
kube-system fluentd-gcp-v3.2.0-88xfk 2/2 Running 0 6m22s
kube-system fluentd-gcp-v3.2.0-zwmnd 2/2 Running 0 6m6s
kube-system heapster-v1.6.0-beta.1-5454f94f99-m5wlx 3/3 Running 0 6m28s
kube-system kube-dns-autoscaler-76fcd5f658-cnmdq 1/1 Running 0 6m45s
kube-system kube-dns-b46cc9485-82s95 4/4 Running 0 7m7s
kube-system kube-dns-b46cc9485-jxz42 4/4 Running 0 6m27s
kube-system kube-proxy-gke-idj-gke-2-default-pool-f9511ddf-443f 1/1 Running 0 6m51s
kube-system kube-proxy-gke-idj-gke-2-default-pool-f9511ddf-82x9 1/1 Running 0 6m51s
kube-system kube-proxy-gke-idj-gke-2-default-pool-f9511ddf-cl87 1/1 Running 0 6m53s
kube-system l7-default-backend-6f8697844f-9sgdr 1/1 Running 0 7m8s
kube-system metrics-server-v0.3.1-5b4d6d8d98-8crc7 2/2 Running 0 6m27s
kube-system prometheus-to-sd-9ghlb 1/1 Running 0 6m51s
kube-system prometheus-to-sd-gk75j 1/1 Running 0 6m53s
kube-system prometheus-to-sd-gmppn 1/1 Running 0 6m51s

We can then delete with Terraform and see it reflected in the UI:

$ terraform destroy
google_container_cluster.primary: Refreshing state... (ID: idj-gke-2)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  - google_container_cluster.primary


Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

google_container_cluster.primary: Destroying... (ID: idj-gke-2)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 10s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 20s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 30s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 40s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 50s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m0s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m10s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m20s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m30s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m40s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 1m50s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m0s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m10s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m20s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m30s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m40s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 2m50s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m0s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m10s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m20s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m30s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m40s elapsed)
google_container_cluster.primary: Still destroying... (ID: idj-gke-2, 3m50s elapsed)
google_container_cluster.primary: Destruction complete after 3m58s

Destroy complete! Resources: 1 destroyed.

mouse over the cluster name in the UI during deleting to see the indicator

Summary:

We explored GCP’s offering for Kubernetes and found it’s a great featured option. We did not explore costs (mostly because they aren’t front and center). We did find Terraform was easy to use to create a new GCP Cluster.