Lead show moved pane despite unsent text
This commit is contained in:
Ian Gulliver
+2
-2
@@ -18,7 +18,7 @@
|
||||
|
||||
## Data
|
||||
|
||||
State lives in the directory given by `byre serve --data`, by default the `byre` directory under Go's `os.UserConfigDir()` (on macOS, `~/Library/Application Support/byre`): `byre.db`, `worktrees/<project-id>/<task-id>`, `worktrees/<project-id>/<task-id>-media` (the task's media folder, see "Screenshots from implementers"), `leads/<login>` (each lead agent's working directory, the login path-escaped), `blobs/<blob-id>` (uploaded images; the `blobs` table maps each id to its content type and its path relative to this directory). Task branches are `byre/<task-id>` in the project's repo.
|
||||
State lives in the directory given by `byre serve --data`, by default the `byre` directory under Go's `os.UserConfigDir()` (on macOS, `~/Library/Application Support/byre`): `byre.db`, `worktrees/<project-id>/<task-id>`, `worktrees/<project-id>/<task-id>-media` (the task's media folder, see "Screenshots from implementers"), `leads/<login>` (each lead agent's working directory, the login path-escaped), `hands/<session-id>/byre-hands` (each running implementer session's copy of `byre-hands`, see "Screen and app control"), `blobs/<blob-id>` (uploaded images; the `blobs` table maps each id to its content type and its path relative to this directory). Task branches are `byre/<task-id>` in the project's repo.
|
||||
|
||||
Schema changes are the `migrations` slice in `internal/store/store.go`: named entries, each with a short kebab-case name saying what it does. On open, byre applies, in slice order, every entry whose name isn't in the `migrations` table, each in its own transaction that also records the name. Each runs with foreign keys off, so a migration can rebuild a table other tables reference, and `PRAGMA foreign_key_check` must come back empty before it commits. byre is deployed from task branches that each append their own migrations, so after a merge a migration can land before others the database has already applied, and it still runs. Give a new migration a name no other branch will use, and make any data migration safe to run twice. `sqlite3 byre.db 'SELECT * FROM migrations'` lists what has been applied. A database from before the table existed is moved over on open: the first `PRAGMA user_version` entries are recorded as applied, and `user_version` is no longer read or written.
|
||||
|
||||
@@ -137,7 +137,7 @@ Clients reach `tailscale serve` over HTTP/2; it proxies to byre over HTTP/1.1 an
|
||||
|
||||
## Screen and app control
|
||||
|
||||
Implementer sessions get `byre-hands` as a stdio MCP server named `hands` (`handsMCP` in `internal/agents/hands.go`), passed the same way as reviewers' Playwright server (see "Reviews"); reviewers and lead agents don't get it. It is a Swift executable built from `hands/` and installed in `~/bin`, which is on the service's `PATH`, and each session runs its own copy, so a hung or crashed one affects only that agent. It speaks MCP's JSON-RPC over stdin and stdout, one message per line, and returns captures as inline PNG image content. Its tools, whose descriptions in `hands/Sources/byre-hands/Tools.swift` are what agents read:
|
||||
Implementer sessions get `byre-hands` as a stdio MCP server named `hands` (`handsMCP` in `internal/agents/hands.go`), passed the same way as reviewers' Playwright server (see "Reviews"); reviewers and lead agents don't get it. It is a Swift executable built from `hands/` and installed in `~/bin`, which is on the service's `PATH`. Each session runs its own process from its own file: when the session starts, byre copies the `byre-hands` it finds on `PATH` to `hands/<session-id>/byre-hands` in the data directory and runs that, and it removes the directory when the session's agent process ends. A hung or crashed one affects only that agent. The separate files matter because replayd, the daemon behind ScreenCaptureKit, keys its clients by executable path: two processes run from one path keep cancelling each other's connection, and every capture from that path hangs. replayd's log shows it as `accepted client connection PID: <a>` and `invalidated because the current process cancelled the connection` alternating between two pids, thousands of times a second. It speaks MCP's JSON-RPC over stdin and stdout, one message per line, and returns captures as inline PNG image content. Its tools, whose descriptions in `hands/Sources/byre-hands/Tools.swift` are what agents read:
|
||||
|
||||
- `screen_list`: running apps, on-screen windows (front to back) and displays.
|
||||
- `capture`: one window, or a display, through ScreenCaptureKit. A window is captured as its rectangle of its display with only its own app's windows included, so other apps covering it don't hide it and its menus, popups and sheets show; capturing the window alone fails while a popup is attached to it. The image is scaled to `max_edge` (default 1568, as much as a model sees; 16 to 8192) and the reply gives the pixel-to-point scale. With `save`, it is also written as `capture-<random>.png` in the directory `byre-hands` was started with as `--media-dir`, and the reply gives the path; a `byre-hands` started without one (the macOS app's, which `client_hands` reaches) refuses `save`.
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
package agents
|
||||
|
||||
import "g.fc.run/misc/byre/internal/mcp"
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
|
||||
"g.fc.run/misc/byre/internal/mcp"
|
||||
)
|
||||
|
||||
const screenshotGuidance = "\n- When a turn changes something you can see, you can show it in your final message: save screenshots with the Playwright tools (pages you serve on this task's ports) or hands' capture with save (apps on this Mac), and embed each as a Markdown image, , with the path the tool returned. They go to the task's media folder, outside the worktree, so they are never committed, and are deleted when the task closes. Leave out a screenshot that adds nothing to the text."
|
||||
|
||||
@@ -8,6 +14,34 @@ func MediaDir(worktreePath string) string {
|
||||
return worktreePath + "-media"
|
||||
}
|
||||
|
||||
func handsMCP(mediaDir string) mcp.Stdio {
|
||||
return mcp.Stdio{Name: "hands", Command: "byre-hands", Args: []string{"--media-dir", mediaDir}}
|
||||
func (m *Manager) handsDir(sessionID string) string {
|
||||
return filepath.Join(m.handsRoot, sessionID)
|
||||
}
|
||||
|
||||
// replayd keys ScreenCaptureKit clients by executable path, so two hands
|
||||
// processes run from one path keep cancelling each other's connection.
|
||||
func (m *Manager) handsMCP(sessionID, mediaDir string) (mcp.Stdio, error) {
|
||||
src, err := exec.LookPath("byre-hands")
|
||||
if err != nil {
|
||||
return mcp.Stdio{}, err
|
||||
}
|
||||
data, err := os.ReadFile(src)
|
||||
if err != nil {
|
||||
return mcp.Stdio{}, err
|
||||
}
|
||||
dir := m.handsDir(sessionID)
|
||||
err = os.RemoveAll(dir)
|
||||
if err != nil {
|
||||
return mcp.Stdio{}, err
|
||||
}
|
||||
err = os.MkdirAll(dir, 0o755)
|
||||
if err != nil {
|
||||
return mcp.Stdio{}, err
|
||||
}
|
||||
bin := filepath.Join(dir, "byre-hands")
|
||||
err = os.WriteFile(bin, data, 0o755)
|
||||
if err != nil {
|
||||
return mcp.Stdio{}, err
|
||||
}
|
||||
return mcp.Stdio{Name: "hands", Command: bin, Args: []string{"--media-dir", mediaDir}}, nil
|
||||
}
|
||||
|
||||
@@ -37,6 +37,7 @@ type Manager struct {
|
||||
store *store.Store
|
||||
worktreeRoot string
|
||||
leadRoot string
|
||||
handsRoot string
|
||||
codexHome string
|
||||
status func(context.Context) (api.Status, error)
|
||||
addProject func(ctx context.Context, name, repoPath, createdBy string) (api.Project, error)
|
||||
@@ -122,11 +123,12 @@ func (l *live) implementer() bool {
|
||||
return l.owner == "" && l.reviewID == ""
|
||||
}
|
||||
|
||||
func New(st *store.Store, worktreeRoot, leadRoot, codexHome string, status func(context.Context) (api.Status, error), addProject func(ctx context.Context, name, repoPath, createdBy string) (api.Project, error), clientRegistry *clients.Registry) *Manager {
|
||||
func New(st *store.Store, worktreeRoot, leadRoot, handsRoot, codexHome string, status func(context.Context) (api.Status, error), addProject func(ctx context.Context, name, repoPath, createdBy string) (api.Project, error), clientRegistry *clients.Registry) *Manager {
|
||||
m := &Manager{
|
||||
store: st,
|
||||
worktreeRoot: worktreeRoot,
|
||||
leadRoot: leadRoot,
|
||||
handsRoot: handsRoot,
|
||||
codexHome: codexHome,
|
||||
status: status,
|
||||
addProject: addProject,
|
||||
@@ -768,7 +770,11 @@ func (m *Manager) startLive(ctx context.Context, sess api.AgentSession, task api
|
||||
return nil, err
|
||||
}
|
||||
g += screenshotGuidance
|
||||
stdio = []mcp.Stdio{handsMCP(media), playwrightMCP(media)}
|
||||
hands, err := m.handsMCP(sess.Id, media)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
stdio = []mcp.Stdio{hands, playwrightMCP(media)}
|
||||
}
|
||||
if sess.Role == api.Reviewer {
|
||||
r, err := m.store.ReviewBySession(ctx, sess.Id)
|
||||
@@ -819,6 +825,11 @@ func (m *Manager) pump(l *live) {
|
||||
|
||||
l.sess.drive(m, l)
|
||||
|
||||
err := os.RemoveAll(m.handsDir(l.id))
|
||||
if err != nil {
|
||||
slog.Error("remove session's byre-hands", "session", l.id, "err", err)
|
||||
}
|
||||
|
||||
m.mu.Lock()
|
||||
delete(m.live, l.id)
|
||||
closing := m.closing
|
||||
|
||||
@@ -68,7 +68,7 @@ func New(dataDir string, portFirst int) (*Server, error) {
|
||||
}
|
||||
}
|
||||
|
||||
s.agents = agents.New(s.store, filepath.Join(dataDir, "worktrees"), filepath.Join(dataDir, "leads"), filepath.Join(dataDir, "codex"), s.status, s.createProject, s.clients)
|
||||
s.agents = agents.New(s.store, filepath.Join(dataDir, "worktrees"), filepath.Join(dataDir, "leads"), filepath.Join(dataDir, "hands"), filepath.Join(dataDir, "codex"), s.status, s.createProject, s.clients)
|
||||
err = s.agents.Recover(context.Background())
|
||||
if err != nil {
|
||||
s.store.Close()
|
||||
|
||||
@@ -24,6 +24,7 @@ final class PaneFocus {
|
||||
if self.window === window {
|
||||
return
|
||||
}
|
||||
log.notice("pane focus: attach window=\(ObjectIdentifier(window).debugDescription, privacy: .public) title=\(window.title, privacy: .public)")
|
||||
detach()
|
||||
self.window = window
|
||||
let center = NotificationCenter.default
|
||||
@@ -60,6 +61,7 @@ final class PaneFocus {
|
||||
}
|
||||
|
||||
func detach() {
|
||||
log.notice("pane focus: detach window=\(self.window.map { ObjectIdentifier($0).debugDescription } ?? "nil", privacy: .public) observers=\(self.observers.count)")
|
||||
for o in observers {
|
||||
NotificationCenter.default.removeObserver(o)
|
||||
}
|
||||
@@ -125,6 +127,7 @@ final class PaneFocus {
|
||||
}
|
||||
if let p = contentPoint(view.bounds, in: view) {
|
||||
inside = paneFrame.contains(p)
|
||||
log.notice("pane focus: responder \(String(describing: type(of: view)), privacy: .public) point=\(p.debugDescription, privacy: .public) frame=\(self.paneFrame.debugDescription, privacy: .public) inside=\(self.inside)")
|
||||
}
|
||||
update()
|
||||
}
|
||||
@@ -135,6 +138,7 @@ final class PaneFocus {
|
||||
}
|
||||
let p = content.convert(location, from: nil)
|
||||
inside = paneFrame.contains(CGPoint(x: p.x, y: content.isFlipped ? p.y : content.bounds.height - p.y))
|
||||
log.notice("pane focus: click point=\(p.debugDescription, privacy: .public) flipped=\(content.isFlipped) frame=\(self.paneFrame.debugDescription, privacy: .public) inside=\(self.inside)")
|
||||
DispatchQueue.main.async { [weak self] in
|
||||
self?.responderChanged()
|
||||
}
|
||||
@@ -143,6 +147,9 @@ final class PaneFocus {
|
||||
private func update() {
|
||||
let now = (window?.isKeyWindow ?? false) && NSApp.isActive && inside
|
||||
let draft = !drafts.isEmpty
|
||||
let attached = window.map { ObjectIdentifier($0).debugDescription } ?? "nil"
|
||||
let keyWindow = NSApp.keyWindow.map { ObjectIdentifier($0).debugDescription } ?? "nil"
|
||||
log.notice("pane focus: update window=\(attached, privacy: .public) keyWindow=\(keyWindow, privacy: .public) key=\(self.window?.isKeyWindow ?? false) active=\(NSApp.isActive) inside=\(self.inside) now=\(now) draft=\(draft) focused=\(self.focused) drafting=\(self.drafting)")
|
||||
if now == focused && draft == drafting {
|
||||
return
|
||||
}
|
||||
@@ -158,6 +165,7 @@ final class PaneFocus {
|
||||
await previous?.value
|
||||
do {
|
||||
_ = try await api.client.setPaneFocus(body: .json(body)).noContent
|
||||
log.notice("pane focus: sent focused=\(body.focused) drafting=\(body.drafting)")
|
||||
} catch {
|
||||
log.error("set pane focus: \(errorText(error), privacy: .public)")
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user