helios/heliosian
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Move media from Drive to a Cloud Storage bucket with stored thumbnails

Browse Source
This commit is contained in:
Ian Gulliver
2026-08-20 11:47:42 -07:00
parent 52668a6a09
commit 75e37bc02c
11 changed files with 390 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev.md
+4 -3
View File
@@ -16,14 +16,14 @@ In sample mode `tools/screenshot` captures pages directly, no session needed (se
DIRECTORY_SHEET=<spreadsheet id> go run .
serves from the production spreadsheet and media drive (see `docs/data.md`) and turns on the full stack. The model loads at startup — the server refuses to start if the load fails — and reloads every five minutes. Real data never leaves the process: nothing is written to disk. Requirements:
serves from the production spreadsheet and media bucket (see `docs/data.md`) and turns on the full stack. The model loads at startup — the server refuses to start if the load fails — and reloads every five minutes. Real data never leaves the process: nothing is written to disk. Requirements:
- **Sign-in** — everything sits behind Google sign-in restricted to the school's Workspace domain (API paths get a 401 instead of the login page). The OAuth web client is read from `creds/oauth-client.json` or `GOOGLE_CLIENT_ID`; its authorized JavaScript origins must include `http://localhost:8080`. The server issues its own HMAC-signed session cookie; set `SESSION_KEY` to keep sessions valid across restarts.
- **Data access** — Google credentials come from application-default credentials impersonating the data service account, set up once per machine:
gcloud auth application-default login --impersonate-service-account=directory@heliosian.iam.gserviceaccount.com
This requires `roles/iam.serviceAccountTokenCreator` on that account. The community shared drive — which holds the `Directory` sheet and all media — is shared with `directory@` as content manager; local processes then act as exactly the identity production runs as, with no key file anywhere (the org forbids creating one).
This requires `roles/iam.serviceAccountTokenCreator` on that account. `directory@` reaches the `Directory` sheet as a content manager of the community shared drive, and the media bucket through project IAM; local processes then act as exactly the identity production runs as, with no key file anywhere (the org forbids creating one).
- **Maps** — a server key for the Geocoding API (`creds/geocoding.key` or `GOOGLE_MAPS_SERVER_KEY`; never rendered into pages, restrict by server IP or leave unrestricted for dev) and a browser key for the Maps JavaScript API (`creds/maps.key` or `GOOGLE_MAPS_BROWSER_KEY`; rendered into pages, restrict by HTTP referer). Geocoding results are cached in memory per address.
To capture authenticated real-data pages, launch the capture browser (`go run ./tools/capturebrowser`), sign in to the local server there once, and use `tools/browse` or `tools/screenshot -remote` — the session cookie lives in the capture profile.
@@ -42,7 +42,7 @@ No Node, no Docker, and no cloud credentials are needed for local development. R
## Tools
Each runs as `go run ./tools/<name>`. The sheet and drive tools authenticate with the same impersonated application-default credentials as the server (see Real data).
Each runs as `go run ./tools/<name>`. The sheet, drive, and bucket tools authenticate with the same impersonated application-default credentials as the server (see Real data).
- `screenshot`, `capturebrowser`, `browse` — page capture and browser driving; see `docs/screenshots.md`
- `startserver` — launch the app detached, wait for it to listen, print the pid, log path, and a minted session cookie (needs `SESSION_KEY` and `DIRECTORY_SHEET`)
@@ -55,4 +55,5 @@ Each runs as `go run ./tools/<name>`. The sheet and drive tools authenticate wit
- `createtabs` — create the directory sheet's local-layer tabs with their header rows
- `setcell` — set one cell in a tab by key column, appending the row if missing
- `probeblob` — time the download of a few drive media files
- `migratemedia` — copy the drive media folders into the media bucket, generating thumbnails; retired once the bucket is the only store
- `splash` — regenerate the iOS splash battery from the captured original page; see `docs/pwa.md`