Wake iso's display for hands calls while it sleeps
This commit is contained in:
@@ -163,6 +163,8 @@ Implementer sessions get `byre-hands` as a stdio MCP server named `hands` (`hand
|
||||
|
||||
`byre-hands` reads stdin on a background thread and starts each request as its own detached task as soon as it arrives, so one call stuck on an app doesn't hold up the others; responses are written whole, one line each, under a lock, in the order they finish. Ref names come from one counter for the life of the process, so no two trees ever share a name, even trees built at the same time; the ref table is swapped in under a lock once a tree is complete, so a ref from any tree but the last published one fails as unknown instead of naming another tree's element. Each `act` call takes the table as it stands when the call arrives and resolves every step's refs from that copy, so a tree finishing during its waits can't point a ref at a different element. AppleScript needs the main thread with a run loop, so every script runs there (`osa` in `Browser.swift`, `DispatchQueue.main.sync`, while the main thread runs `RunLoop.main`), one at a time, wrapped in `with timeout of 10 seconds`; `NSApplication.shared` connects the process to the window server at startup, without which ScreenCaptureKit asserts. An app that is in a menu-tracking loop or otherwise busy answers neither Apple events nor accessibility requests: a script fails with AppleScript's timeout error after 10s, and accessibility requests time out after 3s (the messaging timeout set on the system-wide element at startup), and every accessibility read (`attr` in `AX.swift`, and the action-name and settable checks in the tree walk) throws on the first timeout, so a tree walk, `wait_for`, a ref's position or the front-window check stops with an error saying the app isn't answering instead of waiting 3s on every attribute of every element.
|
||||
|
||||
**Display sleep.** While the display sleeps, ScreenCaptureKit lists no displays and `capture` finds nothing to capture. So every tool call runs inside `withDisplayAwake` (`Display.swift`). It holds a `PreventUserIdleDisplaySleep` assertion for the whole call. When the main display is asleep (`CGDisplayIsAsleep`, or zero bounds), it also declares user activity (`IOPMAssertionDeclareUserActivity`), which turns the display on. It then waits, polling every 100ms for up to 10s, until ScreenCaptureKit lists the display again. If ScreenCaptureKit refuses (no Screen Recording grant), it stops waiting and runs the call, so tools that don't capture still work and `capture` reports the refusal itself. A declared activity doesn't reset the keyboard and mouse idle time, so once the call's assertion is gone, the display sleeps again after powerd's grace period (`com.apple.powermanagement.delayDisplayOff` in `pmset -g assertions`), not after the full display-sleep delay. `pmset -g log` shows each wake as `Display is turned on` beside `byre-hands` assertions.
|
||||
|
||||
**Permissions.** Agents' processes, `byre-hands` included, run as byre's descendants, so macOS privacy controls (TCC) hold byre responsible for them: the Screen Recording, Accessibility and Automation grants in System Settings → Privacy & Security are byre's, made by clicking at the Mac's screen, and each app scripted for the first time asks for its own Automation grant there. `ax_tree` and `act` check `AXIsProcessTrusted` first and fail saying byre lacks Accessibility when it isn't granted, since an untrusted process otherwise just sees no windows and no menu bar. Accessibility can list byre switched off; only its switch being on counts. `codesign` gives byre the identifier `run.fc.byre` and the team's certificate (Apple Development from a checkout, Developer ID in a release), which TCC keys grants on, so they carry over from build to build, but not between a checkout's build and a release's. `codesign -dv <binary>` shows the signature a binary has, and `launchctl print gui/$(id -u)/run.fc.byre` which binary the service runs.
|
||||
|
||||
## Screenshots from implementers
|
||||
|
||||
Reference in New Issue
Block a user