54 lines
7.4 KiB
Markdown
54 lines
7.4 KiB
Markdown
# Codex app-server protocol
|
|
|
|
How `internal/codex` drives `codex app-server`. The server's own schema is the reference: `codex app-server generate-json-schema --experimental --out DIR` writes it, and `codex_app_server_protocol.schemas.json` in DIR is the whole bundle (top-level definitions plus a nested `v2` set).
|
|
|
|
## Binary and version
|
|
|
|
byre runs `codex` from `PATH`. On the owner's machine that is the CLI inside the ChatGPT desktop app (`ls -la ~/.local/bin/codex` shows the link), so the version moves when the app updates itself; `codex --version` prints it. byre passes `-c check_for_update_on_startup=false` so the CLI never checks for or offers updates of its own. `SchemaVersion` in `internal/codex/protocol.gen.go` names the version the types were generated from; when it differs from `codex --version`, regenerate and re-record (below).
|
|
|
|
## Generated types
|
|
|
|
`go generate ./internal/codex` runs `internal/codex/gen`: it asks the CLI for the experimental schema, then writes `protocol.gen.go` with a Go type for every params and response type of the client methods listed in `clientMethods` (`internal/codex/gen/main.go`), every server notification and every server request, and everything they reference. The experimental schema is used because the server sends experimental fields (such as `environments` on a thread) even to clients that don't opt into them.
|
|
|
|
- A `oneOf`/`anyOf` whose alternatives are all objects becomes one struct with every alternative's fields (a field whose type differs between alternatives is `json.RawMessage`), plus `<Type>Type<Value>` constants for its `type` discriminator, as for `ThreadItem`.
|
|
- Alternatives that are all strings become a string type with a constant per value; a mix of strings and objects (such as `AskForApproval`) is `json.RawMessage`.
|
|
- Optional and nullable fields get `omitempty`; optional structs are pointers.
|
|
- `Method<Name>` constants name every method; `clientResults`, `notificationParams`, `serverRequestParams` and `serverRequestResponses` map methods to their types.
|
|
|
|
## Launch and framing
|
|
|
|
```
|
|
codex app-server -c check_for_update_on_startup=false
|
|
```
|
|
|
|
Run in the session's working directory, with `GIT_AUTHOR_*`/`GIT_COMMITTER_*` set to the commit identity and anything in `Options.Env` (byre's agent manager adds the task's ports and `CODEX_HOME`, see `docs/operations.md`); shell commands inherit them. NDJSON both ways on stdio. Messages are JSON-RPC 2.0 without the `"jsonrpc"` member: requests `{"id","method","params"}`, responses `{"id","result"}` or `{"id","error":{"code","message"}}`, notifications `{"method","params"}`. The server adds `emittedAtMs` to its notifications. The server reads `config.toml` from `CODEX_HOME` (default `~/.codex`), so the MCP servers, plugins and `notify` program configured there apply to the session; `-c mcp_servers={}` doesn't remove them, a separate `CODEX_HOME` does.
|
|
|
|
## Session lifecycle
|
|
|
|
1. `initialize` with `clientInfo` and `capabilities.experimentalApi` true (dynamic tools need it: without it `thread/start` with `dynamicTools` fails with `thread/start.dynamicTools requires experimentalApi capability`), then the `initialized` notification.
|
|
2. `thread/start` (`cwd`, `model`, `approvalPolicy` `never`, `sandbox` `danger-full-access`, `developerInstructions` for byre's guidance, `dynamicTools` for `Options.MCPServers`), or `thread/resume` with `threadId` and the same settings but no tools, or `thread/fork` (`Options.ForkFrom` as `threadId`, `Options.ForkAt` as `lastTurnId`, the last turn the fork keeps; the turn must not be in progress) with the same settings. The server picks the thread id, returned as `thread.id`; resuming keeps it, forking makes a new one. `Session.ID` is the thread id. A thread's dynamic tools are stored with it, so resumed and forked threads keep them.
|
|
3. `turn/start` with `threadId` and `input`, a list of `UserInput`: `{"type":"text","text":...}`, and for each image `{"type":"image","url":"data:<type>;base64,..."}` before the text. With `Options.Reminder` set, every `turn/start` carries it as `additionalContext` `{"byre":{"kind":"application","value":...}}`. `model` on it switches the model for that turn and every later one (`Session.SetModel` holds the model for the next `Send`); `thread/read` reports the thread's current `model`. The response carries the new turn; `turn/started` follows.
|
|
4. `turn/steer` with `threadId`, `expectedTurnId` (the active turn) and `input` adds input to the running turn (`Session.Steer`); the model reads it at its next step without the turn ending.
|
|
5. `turn/interrupt` with `threadId` and the active `turnId` ends the turn: `turn/completed` arrives with `status` `interrupted` within milliseconds, a running command is killed, and the thread stays usable. `Session` tracks the active turn from `turn/started` and `turn/completed`.
|
|
6. `thread/compact/start` runs compaction as a turn of its own: `turn/started`, an item of type `contextCompaction`, `turn/completed`. The thread keeps its id and remembers the conversation in summary.
|
|
|
|
## Dynamic tools
|
|
|
|
Each `mcp.Server` in `Options.MCPServers` becomes one `dynamicTools` entry of type `namespace`, named after the server, holding its tools as `function`s with their names, descriptions and input schemas. Names and namespaces starting with `mcp__` are reserved (`dynamic tool name is reserved`), so byre's tools are `ask_human` and so on in the namespace `byre`; the model refers to them as `byre__ask_human`. A call arrives as the server request `item/tool/call` (`namespace`, `tool`, `arguments`, `callId`, `turnId`); `Session` runs that tool's `Handler` and answers `{"success":true,"contentItems":[{"type":"inputText","text":...}]}`, or `success` false with the error text. The call also shows as a `dynamicToolCall` item.
|
|
|
|
Closing stdin ends the process.
|
|
|
|
## Events
|
|
|
|
- `turn/started`, `turn/completed` (`turn.status` `completed`, `interrupted` or `failed` with `turn.error`). A failed request to the model also sends an `error` notification first, with `willRetry`.
|
|
- `item/started` and `item/completed` with a `ThreadItem`: `userMessage`, `agentMessage` (`text`), `reasoning`, `commandExecution` (`command`, `aggregatedOutput`, `exitCode`), `fileChange`, `mcpToolCall`, `contextCompaction` and others.
|
|
- `item/agentMessage/delta` (`itemId`, `delta`) streams an agent message; the deltas join to the completed item's `text`. `item/commandExecution/outputDelta` and the reasoning deltas stream the same way.
|
|
- `thread/tokenUsage/updated` carries `last` and `total` token counts and `modelContextWindow`; `account/rateLimits/updated` the account's rate-limit windows (the same snapshot `account/rateLimits/read` returns on request, as `rateLimits`); `thread/status/changed`, `mcpServer/startupStatus/updated`, `account/updated` and `remoteControl/status/changed` also arrive.
|
|
- Lines that aren't a JSON object, don't parse, or carry a notification method outside `notificationParams` go to `Options.OnBadLine`; parsed notifications are still delivered as events. `Event.Decode` returns the notification's generated params type.
|
|
|
|
Besides `item/tool/call`, server requests (approvals, user input) shouldn't arrive with approvals off; if one does, command and file-change approvals are accepted and anything else gets a JSON-RPC error.
|
|
|
|
## Transcripts
|
|
|
|
`Options.Transcript` records every line in both directions as `{"dir":"send"|"recv","msg":...}`, deltas included, with any `"email"` value replaced by `[redacted]`. After a CLI update, run `go generate ./internal/codex` to regenerate the types.
|