Rename Section to Crew, stage the load pipeline, dedupe lookups and token minting, drop burned tools
This commit is contained in:
+1
-1
@@ -133,7 +133,7 @@ The loader hard-fails — no fallbacks, server refuses to start — on:
|
||||
- conflicting values for the same adult across import rows
|
||||
- an invalid canonical value from any layer
|
||||
|
||||
The import tool ingests a Veracross CSV export, rewrites the Veracross Import tab, re-runs the full pipeline, and reports the local layer's health beyond the fatal checks: useless overrides (value identical to what the record has anyway), `-` on flagged rows, name mappings or additions that Veracross has since made redundant, and media files whose name matches no current person or family — including family blobs orphaned by a membership change. Running it is the whole import procedure. `tools/findsheet` lists the spreadsheets visible to the service account; `tools/sheets` dumps a sheet's tabs, headers, and rows.
|
||||
The import procedure is manual today: `tools/writetab` writes the Veracross CSV export into the Veracross Import tab (header-checked), and `tools/loadcheck` re-runs the full pipeline against the sheet and prints a model summary. A single import tool that also reports the local layer's health beyond the fatal checks — useless overrides (value identical to what the record has anyway), `-` on flagged rows, name mappings or additions that Veracross has since made redundant, and media files whose name matches no current person or family, including family blobs orphaned by a membership change — is planned (`docs/plan.md`). `tools/findsheet` lists the spreadsheets visible to the service account; `tools/sheets` dumps a sheet's tabs, headers, and rows.
|
||||
|
||||
## Sourcing
|
||||
|
||||
|
||||
+17
@@ -35,3 +35,20 @@ Development happens on macOS. Two Homebrew installs cover everything here and in
|
||||
- **Google Chrome** — launched headless by the screenshot tool from its standard install location; never opened by hand.
|
||||
|
||||
No Node, no Docker, and no cloud credentials are needed for local development. Repository layout is in the README.
|
||||
|
||||
## Tools
|
||||
|
||||
Each runs as `go run ./tools/<name>`. The sheet and drive tools authenticate with `creds/service-account.json`.
|
||||
|
||||
- `screenshot`, `capturebrowser`, `browse` — page capture and browser driving; see `docs/screenshots.md`
|
||||
- `startserver` — launch the app detached, wait for it to listen, print the pid, log path, and a minted session cookie (needs `SESSION_KEY` and `DIRECTORY_SHEET`)
|
||||
- `cookie` — print a signed session cookie for local API testing
|
||||
- `loadcheck` — run the full load pipeline against a sheet and print a model summary
|
||||
- `columns` — print each directory table's column names from the configured source
|
||||
- `findsheet` — list spreadsheets visible to the service account
|
||||
- `sheets` — dump a sheet's tabs, sizes, and header rows
|
||||
- `dumptab` / `writetab` — copy one tab to a local CSV / write a local CSV into a tab, header-checked
|
||||
- `createtabs` — create the directory sheet's local-layer tabs with their header rows
|
||||
- `setcell` — set one cell in a tab by key column, appending the row if missing
|
||||
- `probeblob` — time the download of a few drive media files
|
||||
- `splash` — regenerate the iOS splash battery from the captured original page; see `docs/pwa.md`
|
||||
|
||||
@@ -5,3 +5,7 @@ What remains to build. Current behavior is documented in `docs/dev.md`, `docs/da
|
||||
## Hosting and deployment
|
||||
|
||||
- Move from `gen-lang-client-0758114984` to the school's project: recreate the OAuth client there (Internal consent screen, only available inside the school's Workspace org, removes unverified-app friction), plus the service account, secrets, and service; re-share the spreadsheet and media drive with the new service account.
|
||||
|
||||
## Import tool
|
||||
|
||||
- One command that ingests a Veracross CSV export, rewrites the Veracross Import tab, re-runs the full pipeline, and reports the local layer's health beyond the fatal checks: useless overrides, `-` on flagged rows, name mappings or additions Veracross has made redundant, and media files matching no current person or family (including family blobs orphaned by a membership change). Replaces the manual writetab + loadcheck procedure in `docs/data.md`.
|
||||
|
||||
Reference in New Issue
Block a user