helios/heliosian
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite deployment and dev docs for the heliosian project and keyless CI/CD

Browse Source
This commit is contained in:
Ian Gulliver
2026-08-19 22:47:27 -07:00
parent fa86f5913c
commit 15da5ea291
4 changed files with 49 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deploy.md
+42 -26
View File
@@ -1,35 +1,40 @@
# Production deployment
The server runs as Cloud Run service `heliosian` in project `gen-lang-client-0758114984`, region `us-west1`, at https://heliosian-326077318680.us-west1.run.app. Everything below is driven with the gcloud CLI (`brew install --cask gcloud-cli`) authenticated as the deploy identity:
The server runs as Cloud Run service `heliosian` in project `heliosian`, region `us-west1`, at https://heliosian-489539474126.us-west1.run.app, with https://who2.heliosian.com mapped on top. Administration is driven with the gcloud CLI (`brew install --cask gcloud-cli`) authenticated as a project owner (`gcloud auth login`).
gcloud auth activate-service-account --key-file=creds/service-account.json
gcloud config set project gen-lang-client-0758114984
## Build and deploy pipeline
## Image
Every push to `main` on `github.com/heliosian/heliosian` deploys production. The Cloud Build trigger `build-main` (region `us-west1`, attached through the `github` connection) carries its build config inline — nothing in the repo — and runs three steps:
The Dockerfile builds in two stages: a `golang` stage compiles the static binary (`CGO_ENABLED=0`), and `gcr.io/distroless/static-debian12` — CA certificates and tzdata, nothing else — carries the binary plus `web/`, whose templates and static assets are read from disk at runtime. `sampledata/` is deliberately excluded: production always sets `DIRECTORY_SHEET`, and a misconfigured server fails at startup rather than silently serving sample data. `creds/` never enters the image or the source upload (`.dockerignore`, `.gcloudignore`).
1. `docker build` of the repo's Dockerfile, tagged with the commit SHA and `latest`
2. push the SHA tag to Artifact Registry (`us-west1-docker.pkg.dev/heliosian/heliosian/heliosian`)
3. `gcloud run deploy heliosian --image …:<sha>` — image only; every other service setting persists from the configuration below
Cloud Build produces the image into Artifact Registry. Builds ship git HEAD, never the working tree — building the live tree can capture files mid-edit:
The trigger names its build service account (the project's default compute service account) explicitly; trigger creation in this project refuses to infer one. Builds ship the pushed commit, never a working tree. To rebuild and redeploy current `main` without a push:
BUILDDIR=$(mktemp -d)
git archive HEAD | tar -x -C "$BUILDDIR"
gcloud builds submit --tag us-west1-docker.pkg.dev/gen-lang-client-0758114984/heliosian/heliosian "$BUILDDIR"
gcloud builds triggers run build-main --region=us-west1 --branch=main
The Dockerfile builds in two stages: a `golang` stage compiles the static binary (`CGO_ENABLED=0`), and `gcr.io/distroless/static-debian12` — CA certificates and tzdata, nothing else — carries the binary plus `web/`, whose templates and static assets are read from disk at runtime. `sampledata/` is deliberately excluded: production always sets `DIRECTORY_SHEET`, and a misconfigured server fails at startup rather than silently serving sample data. `creds/` never enters the image (`.dockerignore`, `.gcloudignore`), and the image holds no credentials of any kind — the runtime identity below is the only Google identity in play.
## Service
The service was created once with:
gcloud run deploy heliosian \
--image us-west1-docker.pkg.dev/gen-lang-client-0758114984/heliosian/heliosian:latest \
--image us-west1-docker.pkg.dev/heliosian/heliosian/heliosian:latest \
--region us-west1 --allow-unauthenticated \
--min-instances 1 --max-instances 1 --memory 2Gi --no-cpu-throttling \
--service-account directory@heliosian.iam.gserviceaccount.com \
--min-instances 1 --max-instances 1 --memory 4Gi --no-cpu-throttling \
--set-env-vars DIRECTORY_SHEET=<spreadsheet id>,GOOGLE_CLIENT_ID=<oauth client id> \
--set-secrets "/app/creds/service-account.json=heliosian-sa-key:latest,SESSION_KEY=heliosian-session-key:latest,GOOGLE_MAPS_SERVER_KEY=heliosian-geocoding-key:latest,GOOGLE_MAPS_BROWSER_KEY=heliosian-maps-browser-key:latest" \
--set-secrets "SESSION_KEY=heliosian-session-key:latest,GOOGLE_MAPS_SERVER_KEY=heliosian-geocoding-key:latest,GOOGLE_MAPS_BROWSER_KEY=heliosian-maps-browser-key:latest" \
--quiet
Each flag is load-bearing:
- `--min-instances 1` — startup preloads every media file from Drive before listening (about 90 seconds); far too slow for scale-to-zero.
- `--service-account` — the runtime identity. Application-default credentials inside the container resolve to `directory@` through the metadata server; there is no key file anywhere in the system.
- `--min-instances 1` — startup preloads every media file from Drive before listening (about two and a half minutes all told); far too slow for scale-to-zero, and close enough to Cloud Run's four-minute startup probe window to watch as the media set grows.
- `--max-instances 1` — the directory model and blob store live in per-instance memory with no cross-instance coherency; a self-service edit refreshes only the instance that handled it, so a second instance would serve stale data.
- `--memory 2Gi` — the blob store holds all media and thumbnails in RAM. The startup log line `blob store: … MB in memory` reports the footprint; resize when it approaches the limit.
- `--memory 4Gi` — the blob store holds all media and thumbnails in RAM, and the preload peaks well above the steady-state footprint (a roughly 600 MB blob store OOMed a 2 GiB instance during preload). The startup log line `blob store: … MB in memory` reports the steady state; resize when boots start failing or the footprint approaches half the limit.
- `--no-cpu-throttling` — the directory model and blob store refresh on five-minute tickers between requests; default throttling would starve them.
- `--allow-unauthenticated` — the app enforces its own Google sign-in; Cloud Run must let everyone reach the login page.
@@ -39,29 +44,40 @@ Cloud Run injects `PORT`; the server honors it.
Plain environment variables:
- `DIRECTORY_SHEET` — the production spreadsheet id. The sheet is the single spreadsheet shared with the service account.
- `GOOGLE_CLIENT_ID` — the OAuth web client id; not a secret (it is embedded in the login page). The client secret from `creds/oauth-client.json` is never used by the server and lives nowhere in production.
- `DIRECTORY_SHEET` — the production spreadsheet id: the `Directory` sheet living in the community shared drive.
- `GOOGLE_CLIENT_ID` — the OAuth web client id; not a secret (it is embedded in the login page).
Secret Manager secrets, delivered per `--set-secrets` above:
Secret Manager secrets, delivered as environment variables per `--set-secrets` above. Values are used raw, so payloads must not carry trailing newlines:
- `heliosian-sa-key` — `creds/service-account.json`, mounted as a file at `/app/creds/service-account.json` (the exact path the server opens, relative to `/app`)
- `heliosian-session-key` — session-cookie HMAC key (any long random string); losing or rotating it signs everyone out
- `heliosian-geocoding-key` — `creds/geocoding.key`
- `heliosian-maps-browser-key` — `creds/maps.key`
- `heliosian-geocoding-key` — the Geocoding API server key, mirrored locally as `creds/geocoding.key`
- `heliosian-maps-browser-key` — the Maps JavaScript browser key, mirrored locally as `creds/maps.key`
## IAM
`heliosian-test@gen-lang-client-0758114984.iam.gserviceaccount.com` serves two unrelated purposes:
`directory@heliosian.iam.gserviceaccount.com` is both the data identity and the runtime identity:
- Data access: the spreadsheet (as editor — uploads write media cells and the Change Log tab) and the media shared drive (as content manager — uploads create and archive files) are shared with it in Drive/Sheets directly — never through project IAM.
- Deploy identity: project roles Editor, Service Account User, Cloud Run Admin, and Secret Manager Admin. The extra roles exist because Editor cannot set IAM policy on services or secrets.
- Data access: content manager on the community shared drive — which covers editing the `Directory` sheet inside it (self-service edits write cells and append to the Change Log tab) and managing media (uploads create files and archive old versions). Shared in Drive directly, never through project IAM.
- Runtime: the Cloud Run service runs as it, and it holds Secret Manager Secret Accessor on each secret individually.
- Humans: `roles/iam.serviceAccountTokenCreator` on this account enables the local impersonation that real-data development uses (`docs/dev.md`).
The runtime identity is the default compute service account (`326077318680-compute@developer.gserviceaccount.com`) holding Secret Manager Secret Accessor on each secret individually. The basic Editor role deliberately cannot read secret payloads, so these explicit grants are the only thing standing between the service and a startup failure — the console's inherited-role rows on a secret's Permissions tab do not imply payload access.
The build service account (the default compute service account) holds `cloudbuild.builds.builder`, `run.developer`, and Service Account User on `directory@` — the last because deploying a service that runs as an account requires permission to act as it.
The `heliosian.com` organization ships Google's secure-by-default org policies, two of which matter here:
- `iam.managed.disableServiceAccountKeyCreation` stays enforced. The keyless design depends on it staying enforced: no key for `directory@` can exist.
- `iam.allowedPolicyMemberDomains` is overridden to allow-all at the project scope only, because `--allow-unauthenticated` requires granting `run.invoker` to `allUsers`.
## OAuth
The service URL belongs in the OAuth client's authorized JavaScript origins alongside `http://localhost:8080`; sign-in fails on any origin not listed, and edits take a few minutes to propagate. The browser maps key is rendered into every page, so it carries an HTTP-referer restriction for the service URL and localhost (see `docs/dev.md`).
The consent screen lives in this project, audience External and published to production. Internal is not an option: the app restricts sign-in to `heliosschool.org` accounts (`internal/auth`), and those live outside the `heliosian.com` org. Published-External keeps the basic sign-in scopes free of verification friction.
The web client's authorized JavaScript origins are `http://localhost:8080`, the run.app URL, and `https://who2.heliosian.com`; sign-in fails on any origin not listed, and edits take a few minutes to propagate. The browser maps key is rendered into every page, so it carries an HTTP-referer restriction for the same three origins.
## Domain
`who2.heliosian.com` is a Cloud Run domain mapping on the service. DNS carries `who2 CNAME ghs.googlehosted.com.`; Google provisions and renews the certificate once the record resolves. The run.app URL stays live alongside it.
## Verifying a deploy
The startup log (Cloud Run → Logs, or `gcloud logging read`) shows the full boot sequence: geocoding count, directory model load, the blob store footprint line, then `listening`. After any deploy, an existing session should still work — if everyone got signed out, `SESSION_KEY` stopped reaching the server.
The startup log (Cloud Run → Logs, or `gcloud logging read`) shows the full boot sequence: the blob store footprint line, geocoding count, directory model load, then `listening` — about two and a half minutes after the instance starts. After any deploy, an existing session should still work — if everyone got signed out, `SESSION_KEY` stopped reaching the server.