Open websites in the Page view and let agents drive them

This commit is contained in:
Ian Gulliver
2026-10-21 21:34:26 -07:00
parent d6e1895591
commit 57d660c57f
8 changed files with 786 additions and 390 deletions
+3 -2
View File
@@ -4,7 +4,7 @@ Platform constraints, build requirements and SwiftUI behavior observed live.
## Decision
**One web view only**: a task's Page view, which shows an HTML file from the task's worktree or one of its exposed previews in a `WKWebView`, in the attached sheet over the lead pane. Nothing else renders HTML: no `WKWebView` or `WebView` in the window, a dialog, Markdown, diagrams or the terminal. Any other link to a web page opens in the default browser. Mermaid needs a browser DOM, and its automatic layout can't give the banded, nested look of a hand-drawn architecture picture, so byre has its own ```` ```diagram ```` block (rows, nested boxes, a side column, routed arrows, named highlights), laid out on a grid by the app; its syntax is the `diagrams` knowledge-base article. It is byre's only diagram format: a ```` ```mermaid ```` block shows as code, and the workflow stage graphs are drawn by generating its source. Agents see their diagrams through the `render_diagram` tool, which runs the installed app's own binary as `Byre render-diagram` (source on stdin, PNG on stdout), so the picture is exactly what the transcript draws; that mode opens no window and never touches the app's iCloud-synced store, so it uses the default text size. byre's Markdown dialect (choice markers, `<details>` sections, `#/task/` links, worktree image paths, `NO_REPLY`) matches no library, so the app parses it itself.
**One web view only**: a task's Page view, which shows an HTML file from the task's worktree, one of its exposed previews, or a website an agent opened with `show_page`'s `url`, in a `WKWebView`, in the attached sheet over the lead pane. Nothing else renders HTML: no `WKWebView` or `WebView` in the window, a dialog, Markdown, diagrams or the terminal. Any other link to a web page opens in the default browser. Mermaid needs a browser DOM, and its automatic layout can't give the banded, nested look of a hand-drawn architecture picture, so byre has its own ```` ```diagram ```` block (rows, nested boxes, a side column, routed arrows, named highlights), laid out on a grid by the app; its syntax is the `diagrams` knowledge-base article. It is byre's only diagram format: a ```` ```mermaid ```` block shows as code, and the workflow stage graphs are drawn by generating its source. Agents see their diagrams through the `render_diagram` tool, which runs the installed app's own binary as `Byre render-diagram` (source on stdin, PNG on stdout), so the picture is exactly what the transcript draws; that mode opens no window and never touches the app's iCloud-synced store, so it uses the default text size. byre's Markdown dialect (choice markers, `<details>` sections, `#/task/` links, worktree image paths, `NO_REPLY`) matches no library, so the app parses it itself.
The app joins the tailnet as its own node with libtailscale and reaches byre's HTTPS origin on byre's node, so byre knows who is calling from the node itself. It never sends identity headers, and certificate validation is never relaxed.
@@ -35,7 +35,8 @@ Every build is signed for distribution outside the App Store: the team's Develop
- The transcript is a plain `VStack` in a `ScrollView`, not lazy: a `LazyVStack` estimates the height of rows it hasn't shown yet and corrects it when they scroll into view, without moving the scroll offset, so scrolling up through rows not yet shown jumped by each correction; SwiftUI can't move the offset mid-scroll without stopping its momentum.
- The terminal cursor is a steady block: SwiftTerm's blinking cursor restarts its Core Animation fade on every keypress, scheduled 0.7 s out, and on a ProMotion display macOS then holds the window's screen update until that fade starts, so each typed letter appears about 0.7 s late. SwiftTerm redraws the whole screen whenever a terminal color is set, even when its value hasn't changed.
- The Page view loads a worktree page from a plain-HTTP server the app runs on a loopback port (`http://127.0.0.1:<port>/<task id>/<path>`), which fetches each file through the server's `getTaskRaw` endpoint. The page's relative links to its stylesheets, scripts and images resolve to files beside it, and "↗ Browser" opens the same address. Previews load from their loopback forwards at `<name>-<task>.localhost`. App Transport Security blocks plain HTTP to `*.localhost` even with `NSAllowsLocalNetworking`; the `localhost` exception domain with subdomains in `Info.plist` allows it (loopback IP addresses need no exception).
- The Page view loads a worktree page from a plain-HTTP server the app runs on a loopback port (`http://127.0.0.1:<port>/<task id>/<path>`), which fetches each file through the server's `getTaskRaw` endpoint. The page's relative links to its stylesheets, scripts and images resolve to files beside it, and "↗ Browser" opens the same address. Previews load from their loopback forwards at `<name>-<task>.localhost`. App Transport Security blocks plain HTTP to `*.localhost` even with `NSAllowsLocalNetworking`; the `localhost` exception domain with subdomains in `Info.plist` allows it (loopback IP addresses need no exception). `NSAllowsArbitraryLoadsInWebContent` lets the Page view load plain-HTTP websites without lifting ATS for the app's own requests.
- Each Page view tab's `WKWebView` belongs to `Pages`, not to the SwiftUI view, so it keeps its page while the sheet is closed and agents can drive it with `drive_page` (navigate, JavaScript, capture) from the client connection. All tabs share WebKit's default persistent data store, so a site the human signs into stays signed in across launches. A link that opens a new window becomes a new tab of the same task. Capture refuses a tab that isn't on screen (sheet closed, or another tab showing), since its snapshot would show nothing the human sees. Passkey sign-in may fail, because WebKit allows WebAuthn in an embedded view only for apps with Apple's web browser entitlement; a password with a 2FA code works.
## Updates