hands test 2

docs/operations.md

@@ -129,8 +129,8 @@ Implementer sessions get `byre-hands` as a stdio MCP server named `hands` (`hand
 - `screen_list`: running apps, on-screen windows (front to back) and displays.
 - `capture`: one window, or a display, through ScreenCaptureKit. A window is captured as its rectangle of its display with only its own app's windows included, so other apps covering it don't hide it and its menus, popups and sheets show; capturing the window alone fails while a popup is attached to it. The image is scaled to `max_edge` (default 1568, as much as a model sees; 16 to 8192) and the reply gives the pixel-to-point scale.
 - `ax_tree`: a window's or an app's menu bar's accessibility tree, one line per element with text or actions, each with a ref (`e12`) that `act` takes. Refs live in the process and each tree replaces them. They remember the window, or the app whose menu bar, they came from, and `act` refuses a ref that isn't from its window or its app's menu bar. Unlabelled containers are left out with their children shown in their place, and static text already contained in its labelled ancestor's text is dropped.
-- `act`: a list of steps in one window: accessibility actions on refs (press, set value, focus, confirm, any named action), then real mouse and keyboard events posted with `CGEvent` (click, double and right click, drag, scroll, typing, key combinations), Before each of those steps it reads the window's current frame, for `[x, y]`, and makes sure the window is in front: its app frontmost and the app's focused window (through accessibility, so the app's own popups don't count against it). When it isn't, it activates the app and raises the window, and stops with an error, sending nothing, if the window still isn't in front. Every session's `byre-hands` drives the same desktop, so a call that can change what is on screen or focused holds an exclusive `flock` on `byre-hands-input.lock` in the user's temporary directory (`$TMPDIR`): an `act` call with steps and every `browser` call except a `tabs` without capture, each for the whole call, its waiting, tree and capture included, so what it returns shows its own work. Another such call waits up to 30s for it, then fails without doing anything. The lock is released when the steps end or the process exits. Typing posts each ASCII character as its key code, with shift where needed, since Orion ignores key events that carry only a Unicode string. Afterwards it can wait (`wait_ms`, then `wait_for` text in the window's tree) and return the tree and a capture, so one call clicks, waits and looks.
-- `browser`: Orion through AppleScript: list tabs, navigate (then poll `document.readyState`), run JavaScript in a tab (`do JavaScript`), select and close tabs, with the same `wait_for` (page text) and capture. Capturing a call that names a tab selects that tab first, since only the selected tab is on screen, and waits 300ms for it to draw. The Orion window is found by its AppleScript `id`, which is its window server window id. Durations (`wait_ms`, `timeout_ms`, `act`'s `wait` steps) must be 0 to 600000 ms. `do JavaScript` needs Orion's Develop → "Allow JavaScript from Apple Events".
+- `act`: a list of steps in one window: accessibility actions on refs (press, set value, focus, confirm, any named action), then real mouse and keyboard events posted with `CGEvent` (click, double and right click, drag, scroll, typing, key combinations), Before each of those steps it reads the window's current frame, for `[x, y]`, and makes sure the window is in front: its app frontmost and the app's focused window (through accessibility, so the app's own popups don't count against it). When it isn't, it activates the app and raises the window, and stops with an error, sending nothing, if the window still isn't in front. Every session's `byre-hands` drives the same desktop, so a call that can change what is on screen or focused holds an exclusive `flock` on `byre-hands-input.lock` in the user's temporary directory (`$TMPDIR`): an `act` call with steps and every `browser` call except a `tabs` without capture, each for the whole call, its waiting, tree and capture included, so what it returns shows its own work. Another such call waits up to 30s for it, then fails without doing anything. The lock is released when the steps end or the process exits. Typing posts each ASCII character as its key code, with shift where needed, since Orion ignores key events that carry only a Unicode string. Afterwards it can wait (`wait_ms`, then `wait_for`) and return the tree and a capture, so one call clicks, waits and looks. `wait_for` counts the window's elements whose title, description or value contains the text, ignoring case, once before the steps and then every 200ms, and ends when the count rises above the first one (above zero with no steps), so text already on screen, such as another tab's title, doesn't end the wait. Some apps list `confirm` but ignore it, Orion's address bar among them; the tool description says to send Return instead.
+- `browser`: Orion through AppleScript: list tabs, navigate (then poll `document.readyState`), run JavaScript in a tab (`do JavaScript`), select and close tabs, with `wait_for` (the tab's `document.title` and `document.body.innerText`, ignoring case) and capture. Capturing a call that names a tab selects that tab first, since only the selected tab is on screen, and waits 300ms for it to draw. The Orion window is found by its AppleScript `id`, which is its window server window id. Durations (`wait_ms`, `timeout_ms`, `act`'s `wait` steps) must be 0 to 600000 ms. `do JavaScript` needs Orion's Develop → "Allow JavaScript from Apple Events".
 
 `byre-hands` reads stdin on a background thread and starts each request as its own detached task as soon as it arrives, so one call stuck on an app doesn't hold up the others; responses are written whole, one line each, under a lock, in the order they finish. Ref names come from one counter for the life of the process, so no two trees ever share a name, even trees built at the same time; the ref table is swapped in under a lock once a tree is complete, so a ref from any tree but the last published one fails as unknown instead of naming another tree's element. Each `act` call takes the table as it stands when the call arrives and resolves every step's refs from that copy, so a tree finishing during its waits can't point a ref at a different element. AppleScript needs the main thread with a run loop, so every script runs there (`osa` in `Browser.swift`, `DispatchQueue.main.sync`, while the main thread runs `RunLoop.main`), one at a time, wrapped in `with timeout of 10 seconds`; `NSApplication.shared` connects the process to the window server at startup, without which ScreenCaptureKit asserts. An app that is in a menu-tracking loop or otherwise busy answers neither Apple events nor accessibility requests: a script fails with AppleScript's timeout error after 10s, and accessibility requests time out after 3s (the messaging timeout set on the system-wide element at startup), at which point a tree walk or `wait_for` stops with an error saying the app isn't answering instead of timing out on every element.