Last time we experimented with GCP App Engine Standard which provides an easy way to launch a containerized app provided we stay within some narrowly defined guardrails.
But what if we wish a bit more flexibility? That is where App Engine Flex comes into play. GCP App Engine Flex allows us to run more languages and adds scaling and load balancing into the mix.
Creating a NodeJS Express App:
Let’s whip up a quick express app.
$ mkdir myapp_express && cd myapp_express
$ cat app.js
const express = require('express')
const app = express()
const port = 3000
app.get('/', (req, res) => res.send('Hello World from Express!'))
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
And it’s package.json
$ cat package.json
{
"name": "myapp-express",
"description": "Simple Hello World Node.js sample for GCP App Engine Flexible Environment.",
"version": "0.0.1",
"private": true,
"license": "MIT",
"author": "Isaac Johnson",
"repository": {
"type": "git",
"url": "https://github.com/TBD.git"
},
"engines": {
"node": ">=8.0.0"
},
"scripts": {
"deploy": "gcloud app deploy",
"start": "node app.js",
"system-test": "repo-tools test app",
"test": "npm run system-test",
"e2e-test": "repo-tools test deploy"
},
"dependencies": {
"express": "^4.16.3"
},
"devDependencies": {
"@google-cloud/nodejs-repo-tools": "^3.3.0"
},
"cloud-repo-tools": {
"test": {
"app": {
"msg": "Hello, world from Express!"
}
},
"requiresKeyFile": true,
"requiresProjectId": true
}
}
We can start it locally and see it works
$ npm start
> myapp-express@0.0.1 start /Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express
> node app.js
Example app listening on port 3000!
Creating our project in GCP
$ gcloud projects create myexpressapp
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/myexpressapp].
Waiting for [operations/cp.7217125519072015406] to finish...done.
Enabling service [cloudapis.googleapis.com] on project [myexpressapp]...
Operation "operations/acf.f70bb3dc-1b3b-4dc1-8ec9-318abc03c437" finished successfully.
$ gcloud app create --project=myexpressapp
You are creating an app for project [myexpressapp].
WARNING: Creating an App Engine application for a project is irreversible and the region
cannot be changed. More information about regions is at
<https://cloud.google.com/appengine/docs/locations>.
Please choose the region where you want your App Engine application
located:
[1] asia-east2 (supports standard and flexible)
[2] asia-northeast1 (supports standard and flexible)
[3] asia-northeast2 (supports standard and flexible)
[4] asia-south1 (supports standard and flexible)
[5] australia-southeast1 (supports standard and flexible)
[6] europe-west (supports standard and flexible)
[7] europe-west2 (supports standard and flexible)
[8] europe-west3 (supports standard and flexible)
[9] europe-west6 (supports standard and flexible)
[10] northamerica-northeast1 (supports standard and flexible)
[11] southamerica-east1 (supports standard and flexible)
[12] us-central (supports standard and flexible)
[13] us-east1 (supports standard and flexible)
[14] us-east4 (supports standard and flexible)
[15] us-west2 (supports standard and flexible)
[16] cancel
Please enter your numeric choice: 12
Creating App Engine application in project [myexpressapp] and region [us-central]....done.
Success! The app is now created. Please use `gcloud app deploy` to deploy your first app.
Before we deploy, a word of caution. If you didn’t “set as default” when you created the project, your gcloud environment might be pointing to the wrong project.
When i first attempted to deploy, i caught that it was about to deploy to the old (standard) project:
$ gcloud app deploy
Services to deploy:
descriptor: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express/app.yaml]
source: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express]
target project: [api-project-799042244963]
target service: [default]
target version: [20190706t200912]
target url: [https://api-project-799042244963.appspot.com]
Do you want to continue (Y/n)? n
ERROR: (gcloud.app.deploy) Aborted by user.
If you want to avoid mistakes, make sure to add --project:
$ gcloud app deploy --project=myexpressapp
Services to deploy:
descriptor: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express/app.yaml]
source: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express]
target project: [myexpressapp]
target service: [default]
target version: [20190706t201121]
target url: [https://myexpressapp.appspot.com]
Do you want to continue (Y/n)? Y
Enabling service [appengineflex.googleapis.com] on project [myexpressapp]...
ERROR: (gcloud.app.deploy) Error [400] Billing must be enabled for activation of service '[appengineflex.googleapis.com, container.googleapis.com, container.googleapis.com, cloudbuild.googleapis.com, containerregistry.googleapis.com, compute.googleapis.com, compute.googleapis.com, compute.googleapis.com]' in project '536574115962' to proceed.
Detailed error information:
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
violations:
- description: "billing-enabled: Project's billing account is not found. https://console.developers.google.com/project/536574115962/settings"
subject: '536574115962'
type: serviceusage/billing-enabled
Going to the link shows that new projects are not automatically added to a billing account:
Once billing is enabled:
We can then deploy:
$ gcloud app deploy --project=myexpressapp
Services to deploy:
descriptor: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express/app.yaml]
source: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express]
target project: [myexpressapp]
target service: [default]
target version: [20190706t201514]
target url: [https://myexpressapp.appspot.com]
Do you want to continue (Y/n)? Y
Enabling service [appengineflex.googleapis.com] on project [myexpressapp]...
Operation "operations/acf.3434d0f5-6d69-4d57-8b1b-dd8ddbe43858" finished successfully.
Beginning deployment of service [default]...
Building and pushing image for service [default]
Started cloud build [f51e39e7-d70e-4463-805d-403a4336a68a].
To see logs in the Cloud Console: https://console.cloud.google.com/gcr/builds/f51e39e7-d70e-4463-805d-403a4336a68a?project=536574115962
-------------------------------------------------------------------- REMOTE BUILD OUTPUT ---------------------------------------------------------------------
starting build "f51e39e7-d70e-4463-805d-403a4336a68a"
FETCHSOURCE
Fetching storage object: gs://staging.myexpressapp.appspot.com/us.gcr.io/myexpressapp/appengine/default.20190706t201514:latest#1562462236244127
Copying gs://staging.myexpressapp.appspot.com/us.gcr.io/myexpressapp/appengine/default.20190706t201514:latest#1562462236244127...
/ [1 files][ 53.6 KiB/ 53.6 KiB]
Operation completed over 1 objects/53.6 KiB.
BUILD
Starting Step #0
Step #0: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570: Pulling from gcp-runtimes/nodejs/gen-dockerfile
Step #0: e297e9f843a4: Already exists
Step #0: f0efa83c9481: Already exists
Step #0: 3c2cba919283: Already exists
Step #0: a2b48c99666a: Already exists
Step #0: 411ccdae4c24: Already exists
Step #0: 79684cbacd3d: Already exists
Step #0: 08781d1cfbfc: Already exists
Step #0: 677b602e52e5: Already exists
Step #0: 5c5aaaf0f730: Already exists
Step #0: 965f09661298: Already exists
Step #0: c44479ecce83: Pulling fs layer
Step #0: 2e6280c67bf0: Pulling fs layer
Step #0: c44479ecce83: Verifying Checksum
Step #0: c44479ecce83: Download complete
Step #0: c44479ecce83: Pull complete
Step #0: 2e6280c67bf0: Verifying Checksum
Step #0: 2e6280c67bf0: Download complete
Step #0: 2e6280c67bf0: Pull complete
Step #0: Digest: sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: Checking for Node.js.
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest): gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870
Step #1: INFO[0000] Removing ignored files from build context: [node_modules .dockerignore Dockerfile npm-debug.log yarn-error.log .git .hg .svn app.yaml]
Step #1: INFO[0000] Downloading base image gcr.io/google-appengine/nodejs@sha256:6f91f5a574865bc0b707f8d5e575d65623ea3af6162cc6c75782151ea8e53ab9
Step #1: INFO[0015] Taking snapshot of full filesystem...
Step #1: INFO[0026] Using files from context: [/workspace]
Step #1: INFO[0026] COPY . /app/
Step #1: INFO[0026] Taking snapshot of files...
Step #1: INFO[0026] RUN /usr/local/bin/install_node '>=8.0.0'
Step #1: INFO[0026] cmd: /bin/sh
Step #1: INFO[0026] args: [-c /usr/local/bin/install_node '>=8.0.0']
Step #1: INFO[0026] Taking snapshot of full filesystem...
Step #1: INFO[0036] No files were changed, appending empty layer to config. No layer added to image.
Step #1: INFO[0036] RUN npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)
Step #1: INFO[0036] cmd: /bin/sh
Step #1: INFO[0036] args: [-c npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)]
Step #1: added 50 packages from 37 contributors and audited 4160 packages in 3.631s
Step #1: found 0 vulnerabilities
Step #1:
Step #1: INFO[0040] Taking snapshot of full filesystem...
Step #1: INFO[0045] CMD npm start
Step #1: 2019/07/07 01:18:21 existing blob: sha256:fdca1807eafb2b085a2b6c5362a32da4cabcfa36a4c461b0b5d443be67dbfeb4
Step #1: 2019/07/07 01:18:21 existing blob: sha256:2d08728a05254e0446429efa1de10c0396faacdaab56cc585dba86b3cdc5729e
Step #1: 2019/07/07 01:18:21 existing blob: sha256:d9733a3b830b66289659e7094e8209f2d6fea8ac83cedfe68bbf8e4a16bdc7a2
Step #1: 2019/07/07 01:18:21 existing blob: sha256:98063bc0bae51ad4a84c7c2670cae6c8430a4563dc6fbf0c1ae60a88d410f3d9
Step #1: 2019/07/07 01:18:21 existing blob: sha256:6132012d11419a4196a643d9ae6595b90f6a787f6cee31cdfd2dd0c40aadc916
Step #1: 2019/07/07 01:18:21 existing blob: sha256:3c2cba919283a210665e480bcbf943eaaf4ed87a83f02e81bb286b8bdead0e75
Step #1: 2019/07/07 01:18:21 existing blob: sha256:75efb7080e80b173e59b3123c78a9a54748fc72b680eece7632724a40436703b
Step #1: 2019/07/07 01:18:21 existing blob: sha256:012efc4864f8b4e3d3f56a4604210db12edc9cdec787c76b4a3553f91c938798
Step #1: 2019/07/07 01:18:21 existing blob: sha256:dc6189c3ce2e8422d5f78765382a26d3a38a781f284ff635f9d0e152ef0c7c81
Step #1: 2019/07/07 01:18:22 existing blob: sha256:c0669b9fd98cd27b7900f3fd1011c3eeb5ba7de37e46321f109eea658921beff
Step #1: 2019/07/07 01:18:23 pushed blob sha256:2f94a03db848f1711bd5a3367b5fcbc8422bc559a56e878bec37159f84ac2150
Step #1: 2019/07/07 01:18:23 pushed blob sha256:2d6f77fbf38d858e4a86e53b68d3baaac7a80d3bcf5151272a89465044427344
Step #1: 2019/07/07 01:18:23 pushed blob sha256:d900417189aadac7b2ec9de5bbec80bb20d009c97a8b29e6287b105442ee585d
Step #1: 2019/07/07 01:18:24 us.gcr.io/myexpressapp/appengine/default.20190706t201514:latest: digest: sha256:403e537fa7a8ac5f030f757935c09de1bd2ccc984cedbedc898316f1e50dee3f size: 2218
Finished Step #1
PUSH
DONE
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Updating service [default] (this may take several minutes)...done.
Setting traffic split for service [default]...done.
Deployed service [default] to [https://myexpressapp.appspot.com]
You can stream logs from the command line by running:
$ gcloud app logs tail -s default
To view your application in the web browser run:
$ gcloud app browse --project=myexpressapp
However, doing the app browse shows it fails:
Ahh, we neglected to allow the port to be passed in. Therefore publishing it would fail (as there is no guarantee you would get 3000):
$ git diff
diff --git a/myapp_express/app.js b/myapp_express/app.js
index 988a490..ad0af93 100644
--- a/myapp_express/app.js
+++ b/myapp_express/app.js
@@ -1,6 +1,6 @@
const express = require('express')
const app = express()
-const port = 3000
+const port = process.env.PORT || 8080;
app.get('/', (req, res) => res.send('Hello World from Express!'))
We can first test it:
$ npm start
> myapp-express@0.0.1 start /Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express
> node app.js
Example app listening on port 8080!
And lastly, deploy it
$ gcloud app deploy --project=myexpressapp
Services to deploy:
descriptor: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express/app.yaml]
source: [/Users/isaac.johnson/Workspaces/gcloud-appengine/myapp_express]
target project: [myexpressapp]
target service: [default]
target version: [20190706t203411]
target url: [https://myexpressapp.appspot.com]
Do you want to continue (Y/n)? Y
Beginning deployment of service [default]...
Building and pushing image for service [default]
Started cloud build [4f1d99d3-c0a6-46fc-af3f-9756fba64119].
To see logs in the Cloud Console: https://console.cloud.google.com/gcr/builds/4f1d99d3-c0a6-46fc-af3f-9756fba64119?project=536574115962
-------------------------------------------------------------------- REMOTE BUILD OUTPUT ---------------------------------------------------------------------
starting build "4f1d99d3-c0a6-46fc-af3f-9756fba64119"
FETCHSOURCE
Fetching storage object: gs://staging.myexpressapp.appspot.com/us.gcr.io/myexpressapp/appengine/default.20190706t203411:latest#1562463254613175
Copying gs://staging.myexpressapp.appspot.com/us.gcr.io/myexpressapp/appengine/default.20190706t203411:latest#1562463254613175...
/ [1 files][ 53.6 KiB/ 53.6 KiB]
Operation completed over 1 objects/53.6 KiB.
BUILD
Starting Step #0
Step #0: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570: Pulling from gcp-runtimes/nodejs/gen-dockerfile
Step #0: e297e9f843a4: Already exists
Step #0: f0efa83c9481: Already exists
Step #0: 3c2cba919283: Already exists
Step #0: a2b48c99666a: Already exists
Step #0: 411ccdae4c24: Already exists
Step #0: 79684cbacd3d: Already exists
Step #0: 08781d1cfbfc: Already exists
Step #0: 677b602e52e5: Already exists
Step #0: 5c5aaaf0f730: Already exists
Step #0: 965f09661298: Already exists
Step #0: c44479ecce83: Pulling fs layer
Step #0: 2e6280c67bf0: Pulling fs layer
Step #0: c44479ecce83: Verifying Checksum
Step #0: c44479ecce83: Download complete
Step #0: c44479ecce83: Pull complete
Step #0: 2e6280c67bf0: Verifying Checksum
Step #0: 2e6280c67bf0: Download complete
Step #0: 2e6280c67bf0: Pull complete
Step #0: Digest: sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82486b24fde393cc1d6938ee371d8439303eca30c7093969a7634ae32dd5d570
Step #0: Checking for Node.js.
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest): gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870
Step #1: INFO[0000] Removing ignored files from build context: [node_modules .dockerignore Dockerfile npm-debug.log yarn-error.log .git .hg .svn app.yaml]
Step #1: INFO[0000] Downloading base image gcr.io/google-appengine/nodejs@sha256:6f91f5a574865bc0b707f8d5e575d65623ea3af6162cc6c75782151ea8e53ab9
Step #1: INFO[0015] Taking snapshot of full filesystem...
Step #1: INFO[0026] Using files from context: [/workspace]
Step #1: INFO[0026] COPY . /app/
Step #1: INFO[0026] Taking snapshot of files...
Step #1: INFO[0026] RUN /usr/local/bin/install_node '>=8.0.0'
Step #1: INFO[0026] cmd: /bin/sh
Step #1: INFO[0026] args: [-c /usr/local/bin/install_node '>=8.0.0']
Step #1: INFO[0026] Taking snapshot of full filesystem...
Step #1: INFO[0035] No files were changed, appending empty layer to config. No layer added to image.
Step #1: INFO[0035] RUN npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)
Step #1: INFO[0035] cmd: /bin/sh
Step #1: INFO[0035] args: [-c npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)]
Step #1: added 50 packages from 37 contributors and audited 4160 packages in 3.522s
Step #1: found 0 vulnerabilities
Step #1:
Step #1: INFO[0040] Taking snapshot of full filesystem...
Step #1: INFO[0044] CMD npm start
Step #1: 2019/07/07 01:35:21 existing blob: sha256:d9733a3b830b66289659e7094e8209f2d6fea8ac83cedfe68bbf8e4a16bdc7a2
Step #1: 2019/07/07 01:35:21 existing blob: sha256:012efc4864f8b4e3d3f56a4604210db12edc9cdec787c76b4a3553f91c938798
Step #1: 2019/07/07 01:35:21 existing blob: sha256:98063bc0bae51ad4a84c7c2670cae6c8430a4563dc6fbf0c1ae60a88d410f3d9
Step #1: 2019/07/07 01:35:21 existing blob: sha256:3c2cba919283a210665e480bcbf943eaaf4ed87a83f02e81bb286b8bdead0e75
Step #1: 2019/07/07 01:35:21 existing blob: sha256:6132012d11419a4196a643d9ae6595b90f6a787f6cee31cdfd2dd0c40aadc916
Step #1: 2019/07/07 01:35:21 existing blob: sha256:c0669b9fd98cd27b7900f3fd1011c3eeb5ba7de37e46321f109eea658921beff
Step #1: 2019/07/07 01:35:21 existing blob: sha256:fdca1807eafb2b085a2b6c5362a32da4cabcfa36a4c461b0b5d443be67dbfeb4
Step #1: 2019/07/07 01:35:21 existing blob: sha256:2d08728a05254e0446429efa1de10c0396faacdaab56cc585dba86b3cdc5729e
Step #1: 2019/07/07 01:35:21 existing blob: sha256:dc6189c3ce2e8422d5f78765382a26d3a38a781f284ff635f9d0e152ef0c7c81
Step #1: 2019/07/07 01:35:21 existing blob: sha256:75efb7080e80b173e59b3123c78a9a54748fc72b680eece7632724a40436703b
Step #1: 2019/07/07 01:35:22 pushed blob sha256:b19cb739a9d8277f6b74847a4bd7387e3e67ff0a22cef36f22610fb79b88b7b2
Step #1: 2019/07/07 01:35:23 pushed blob sha256:bf1ba6b77ff413b3b683c5ffdb5e75a7b42b981e91f466a605ff717f73cf2129
Step #1: 2019/07/07 01:35:23 pushed blob sha256:dce02cc99323bba5b57671d282ea23ca529f32eb384006ee0bef2bced75636b4
Step #1: 2019/07/07 01:35:24 us.gcr.io/myexpressapp/appengine/default.20190706t203411:latest: digest: sha256:020995b99f2435282d0273349120c25f96cff2ebf3de9550db3ac8a429253d93 size: 2218
Finished Step #1
PUSH
DONE
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Updating service [default] (this may take several minutes)...done.
Setting traffic split for service [default]...done.
Stopping version [myexpressapp/default/20190706t201514].
Sent request to stop version [myexpressapp/default/20190706t201514]. This operation may take some time to complete. If you would like to verify that it succeeded, run:
$ gcloud app versions describe -s default 20190706t201514
until it shows that the version has stopped.
Deployed service [default] to [https://myexpressapp.appspot.com]
You can stream logs from the command line by running:
$ gcloud app logs tail -s default
To view your application in the web browser run:
$ gcloud app browse --project=myexpressapp
AHD-MBP13-048:myapp_express isaac.johnson$ gcloud app browse --project=myexpressapp
Opening [https://myexpressapp.appspot.com] in a new tab in your default browser.
And now we see it running:
Setting custom DNS names:
Since we route our Gandi.net domain to AWS Route 53, we head to AWS to create a TXT record:
We can also try CNAME:
At this point we create the above records in Route53 to handle the above settings:
Upon completion, we see our custom domains listed in that tab:
Now applied, we can route traffic via http and https:
Summary
In this tutorial we created a simple NodeJS Express application and tested locally with npm start. We deployed it and found a non-working version (due to fixed ports). We corrected and deployed again. We learned to enable billing and after validating it worked on a provided URL (https://myexpressapp.appspot.com). We then went through the steps to take a domain which was owned via Gandi.net and routed via AWS Route 53 and direct a subdomain to GCP. GCP then created mapping for us and generated a valid SSL certificate for https://www.myflexapp.freshbrewed.science/ and https://myflexapp.freshbrewed.science/.
GCP App Engine Flex is a great way to host a larger or more complex application without needing the overhead of managing a k8s cluster. Unlike App Engine Standard, there are costs associated so creating a budget in GCP and setting alerts is a likely next step.
