ai/byre
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Paste and drop images into every box that talks to an agent

Browse Source
This commit is contained in:
Ian Gulliver
2026-10-20 17:32:32 -07:00
parent 49f57b73fc
commit 15482473f7
9 changed files with 133 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/macos.md
+1 -1
View File
@@ -179,7 +179,7 @@ Places where SwiftUI alone may fall short, each checked side by side against its
| Need | SwiftUI | Fallback | Spike result |
| --- | --- | --- | --- |
| Enter sends, Shift-Enter newline, nothing during IME composition | `TextEditor` with `onKeyPress` | `NSTextView` in an `NSViewRepresentable`, sending from `doCommand(by: insertNewline:)`, which the input method never reaches while it holds marked text | No fallback: `onKeyPress(.return)` runs before the `TextEditor`'s `NSTextView` and its input context. It sends, and returns `.ignored` for Shift so the text view adds the newline (checked by hand). Because it runs first, it must also return `.ignored` while the focused text view's `hasMarkedText()` is true, or Return sends the draft mid-composition. An in-process test that set marked text with `setMarkedText` and delivered a Return `NSEvent` to the window showed both: an unguarded handler sent the draft with the marked text still pending, and the guarded one sent nothing and passed Return on to the text view. From there the input method handles Return as it does in any `NSTextView`, so the `NSTextView` fallback would behave the same. The guard reads the key window's first responder, which is the composer while the user types |
| Image paste and drop into the composer | `onPasteCommand` and `onDrop` | The same `NSTextView`, overriding `paste(_:)` and drag handling | No fallback, but not through these two modifiers: `TextEditor`'s own text view takes ⌘V and drops before `onPasteCommand` and `onDrop` and inserts the file's path as text. Instead the app replaces Edit's pasteboard group (`CommandGroup(replacing: .pasteboard)` with Cut, Copy, Paste and Select All sent to the first responder): Paste attaches the clipboard's image when the composer has focus (`focusedValue`) and otherwise sends the ordinary `paste:`, so text paste works in every field and the terminal. Drops land on a `Color.clear` overlay on the `TextEditor` with `onDrop` and `allowsHitTesting(false)`: it still receives the drag, and clicks and typing pass through to the editor (with hit testing on, the editor can't be clicked). Both attach thumbnails and upload to `POST /api/blobs` |
| Image paste and drop into every box that talks to an agent (the composer, a question's and a ready call's Other…, line comments, the switcher) | `onPasteCommand` and `onDrop` | The same `NSTextView`, overriding `paste(_:)` and drag handling | No fallback, but not through these two modifiers: `TextEditor`'s own text view takes ⌘V and drops before `onPasteCommand` and `onDrop` and inserts the file's path as text. Instead the app replaces Edit's pasteboard group (`CommandGroup(replacing: .pasteboard)` with Cut, Copy, Paste and Select All sent to the first responder): Paste attaches the clipboard's image when the composer has focus (`focusedValue`) and otherwise sends the ordinary `paste:`, so text paste works in every field and the terminal. Drops land on a `Color.clear` overlay on the `TextEditor` with `onDrop` and `allowsHitTesting(false)`: it still receives the drag, and clicks and typing pass through to the editor (with hit testing on, the editor can't be clicked). Both attach thumbnails and upload to `POST /api/blobs`. Every such box gets both, and the accent border while a drag is over it, from one modifier, `.imageInput(key)` in `Composer.swift`, and each sends with images and no text |
| Transcript of a long task (thousands of events) scrolling smoothly and staying stuck to the bottom | `VStack` in a `ScrollView` | `List` with its own row reuse | No fallback: the stack stays at the bottom while rows arrive and a message streams (`ScrollPosition.scrollTo(edge: .bottom)` on each change while within 80 points of the bottom, tracked with `onScrollGeometryChange`), and stays put when scrolled up. A `LazyVStack` jumped while scrolling up through rows it hadn't shown yet, as it corrected their estimated heights |
| Selecting text across blocks and messages | `textSelection(.enabled)` selects within one `Text` only | A selectable `NSTextView` in an `NSViewRepresentable` | Fallback used within a message: `MarkdownView` draws each run of consecutive paragraphs, headings and lists (bullets as text, hanging indents from tab stops and paragraph styles) as one `NSAttributedString` in one non-editable `FlowText` text view, so a drag crosses blocks and bullets. Code blocks, diffs, tables, quotes, diagrams, `<details>`, paragraphs with images and choice lists stay SwiftUI views and end a run. Messages are still separate views, so a drag selects within one run of text in one message and each message is copied on its own |
| The terminal | No terminal view in SwiftUI | SwiftTerm's AppKit `TerminalView` in an `NSViewRepresentable` | Fallback used, as planned: SwiftTerm over `URLSessionWebSocketTask` gives a working shell; Ctrl-C, Ctrl-L, Ctrl-K and Ctrl-R reach it. A local `keyDown` monitor takes Ctrl-K/P/L/S// as Go shortcuts except while the first responder is the `TerminalView` |