Windowed module protocol: every op admitted inside a 3.4s window after the firmware's observed temp poll (cadence 3.5-4.2s measured, stuck-at-last-fetch stale mechanism proven and avoided), acquire/window with panic on dead heartbeat; noise column shows green on/off cycle phase, cable-missing state unchanged

docs/modules/fs/README.md

@@ -24,9 +24,9 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
 - **GETs must be invoked bare** — pre-writing *any* DATA register, not just the documented DATA1 display flag, leaves the handler executing as a no-op with results never written (sentinel pre-fills survived GET_SNR untouched, proven on hardware).
 - **STATUS must never be written** — any user value (0x0000 and 0x0008 both tried) closes the mailbox: the next command is silently ignored until firmware restores it.
 - **PASS cannot prove completion.** The previous command's PASS stays latched while a slow firmware still executes, and until then the DATA registers are firmware scratch — the die temperature turns up in them (0x004E–0x0051 observed as "SNR" and "EEE mode"; the campaign's "anomalous 0x0047" was this same race).
-- **Handler writes during firmware-busy windows wedge the µC permanently** — DATA/CMD writes landing while firmware does post-AN provisioning or training work (a window stretching ~10 s past relink) killed the SMI service four times; reads alone never once. Unmitigated — exposure accepted until the shared-mailbox contention is properly resolved.
+- **Handler writes during firmware-busy windows wedge the µC permanently** — DATA/CMD writes landing while firmware does post-AN provisioning or training work (a window stretching ~10 s past relink) killed the SMI service four times; reads alone never once. The windowed protocol keeps steady-state writes clear of the temp poll's busy windows; bringup-era writes run unwindowed (they fire pre-AN, where writes have never wedged) and post-AN provisioning exposure remains as the diag path's accepted risk.
 - **Bridge reads carry no fetch identity** — a fetch that outruns the delay leaves the previous transaction's data at 0xAD with no error. A killed host can also leave the pipeline latched a response behind *persistently* (PHY ID served a stuck stale value across processes until driver re-init); every appliance boot clears it by loading the driver, so only bench runs reusing a loaded driver see it, and the identity check at startup dies loudly on it.
-- **Stale reads poisoned the corrected-error channel** — caught with per-register instrumentation: 3.33 reads served 1.147's resident 0x0011 (decoding as exactly 17 errored blocks — the recurring "+17"), SNR DATA values (+21/22), and 1.147 reads served 3.33's 0x8000 (count 16, charged twice by the rollover delta) and the die temperature. Every corrected burst observed to date decodes as a neighbor register; none is proven cable stress, and the noise-correlation was the µC being busy (stale window), not physics. The corrected channel is untrustworthy until the stale-read problem is solved; the internal temp client is a prime suspect for the periodic busy windows and is not silenceable (firmware-reliability notes).
+- **Stale reads poisoned the corrected-error channel** — caught with per-register instrumentation: 3.33 reads served 1.147's resident 0x0011 (decoding as exactly 17 errored blocks — the recurring "+17"), SNR DATA values (+21/22), and 1.147 reads served 3.33's 0x8000 (count 16, charged twice by the rollover delta) and the die temperature. Every such burst decodes as a neighbor register. The source is the firmware's own temp poll and the fix is the windowed protocol (firmware-reliability notes): work confined to a fixed window after each observed poll ends the poisoning — reproduced at will by aiming reads at the poll, eliminated by avoiding it.
 - Poll STATUS ~100 ms; frozen up to 2 s during 10GBASE-T training — only run after link-up.
 
 ## Proven diagnostics
@@ -55,7 +55,11 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
 
 ## Firmware-reliability notes
 
-**The firmware is its own mailbox client — the handler is a shared, unarbitrated resource.** Watched read-only on an idle module: CMD sits at 0x0031 (`CMD_GET_CURRENT_TEMP`, bit 15 consumed) and DATA1 tracks the live die temperature (0x4F–0x52 ≈ 79–82 °C under load), refreshed every ~3.5 s — the firmware issues its own temperature command through the same CMD/STATUS/DATA registers the host uses, with no arbitration. Every mystery this explains: temperature appearing in DATA1 (it is that command's output — the campaign's "anomalous 0x0047" included), PASS satisfying a host poll when it belongs to the internal command, and the µC wedges (two writers colliding on CMD/DATA, likeliest when internal management activity spikes after AN events). The host side keeps exactly one conversation open at a time (one per-module lock around every whole operation, `phy.go`) and confines writes to quiet windows; **contention with the internal client is otherwise unresolved** — a host GET can still be preempted (~10% per command at the temp cadence) and its results replaced. Any GET whose answer lives in DATA1 is unusable; corroborate through IEEE registers or the wire. The hard rule that stands: write every DATA register explicitly before any SET — the handler executes stale DATA. **The internal poll is not silenceable through the handler**: disabling the temperature warning (SET 0x8032 `(0x007D, 5, 0, 0, 0)` and 0x8034 `(0xFFD8, 5, 0, 0, 0)` — DATA3 = 0 = warning off — both PASS) leaves CMD resident at 0x0031 with DATA1 tracking the die within seconds of the host going quiet; the ~3.5 s poll is independent of the warning machinery. GET 0x8033/0x8035 read DATA1–3 = 0 before and after those SETs (DATA4/5 leftover scratch) — DATA1-answer readbacks, unusable as verification either way.
+**The firmware is its own mailbox client — the handler is a shared, unarbitrated resource.** Watched read-only on an idle module: CMD sits at 0x0031 (`CMD_GET_CURRENT_TEMP`, bit 15 consumed) and DATA1 tracks the live die temperature (0x4F–0x52 ≈ 79–82 °C under load) — the firmware issues its own temperature command through the same CMD/STATUS/DATA registers the host uses, with no arbitration. Every mystery this explains: temperature appearing in DATA1 (it is that command's output — the campaign's "anomalous 0x0047" included), PASS satisfying a host poll when it belongs to the internal command, and the µC wedges (two writers colliding on CMD/DATA, likeliest when internal management activity spikes after AN events). Any GET whose answer lives in DATA1 is unusable; corroborate through IEEE registers or the wire. The hard rule that stands: write every DATA register explicitly before any SET — the handler executes stale DATA. **The internal poll is not silenceable through the handler**: disabling the temperature warning (SET 0x8032 `(0x007D, 5, 0, 0, 0)` and 0x8034 `(0xFFD8, 5, 0, 0, 0)` — DATA3 = 0 = warning off — both PASS) leaves CMD resident at 0x0031 with DATA1 tracking the die within seconds of the host going quiet; the poll is independent of the warning machinery. GET 0x8033/0x8035 read DATA1–3 = 0 before and after those SETs (DATA4/5 leftover scratch) — DATA1-answer readbacks, unusable as verification either way.
+
+**The internal poll's clock, measured on hardware.** One poll every 3.49–4.22 s, regime-dependent but rock-stable within a regime (±15 ms): ≈3.50 s warm and idle, ≈3.70 s, ≈4.01 s cool and idle, ≈4.21 s under continuous host mailbox load — host traffic *defers* the poll, never advances it, and nothing observed brings two polls closer than 3.49 s. The poll is link-independent: it runs straight through an AN restart and the whole retrain without missing a beat. Around each poll's service the bridge serves stale data for ~50 ms: every read returns the last value successfully fetched before the µC went busy (stuck-at-last-fetch, not one-behind), which is exactly how 3.33 inherits 1.147's 0x0011. The window closes *before* the poll is detectable — a CMD read returning the fresh resident 0x0031 is itself proof the µC serviced the fetch, so flip detection doubles as the all-clear.
+
+**The windowed protocol (`phy.go` `admit`) — contention resolved by time-division.** Since the poll cannot be silenced, the host schedules around it: after bringup, every operation is admitted only inside a fixed 3.4 s window following each observed poll (CMD flipping to resident 0x0031, polled at 10 ms); at the cutoff the host goes quiet until the next flip. 3.4 s sits below every observed cadence, so a regime switch mid-run lands harmlessly in vacated time — no prediction, no period tracking, just the last observed flip plus two constants. The host's own commands re-arm detection (they leave CMD ≠ 0x0031); a missing heartbeat free-runs one window rather than stall (never observed — the heartbeat survives retrains); bringup runs before windowing is enabled and never waits. Validated on hardware in `~/work/phydiag-work/bcm_phaselock_bench.py`: aiming read bursts at the poll reproduces the poisoning on 100% of polls (the "+17" manufactured on demand); windowed operation ran 2,881 back-to-back batches — 16× the production rate, riding the cutoff — with zero stale values, and the production 1 Hz stream is unperturbed (blackout hides inside natural gaps, worst sample gap ~1.6 s). Corrected bursts since windowing no longer decode as neighbor registers (historically every one did) and correlate with amber SNR margin under full-rate traffic — real line corrections; the graded-noise correlation run is the confirmation path.
 
 ## ECD — recovered from the OpenBCM SDK, proven on hardware
 

docs/open-questions.md

@@ -22,16 +22,7 @@ The register question is answered (post-FEC vs corrected-by-iteration histogram
 
 No confirmed-safe path exists (every candidate lands in the µC danger window). The open decision is whether the capability is worth the NDA route or a sacrificial unit — the product doesn't need it for length ([modules/wiitek/](modules/wiitek/README.md), [modules/README.md](modules/README.md)).
 
-## 5. Corrected-error channel under the internal temp client
-
-Every corrected-error burst observed decodes as a stale neighbor register served under a busy
-µC, and the firmware's internal ~3.5 s GET_CURRENT_TEMP poll — the prime suspect for the busy
-windows — is not silenceable through the handler (temp-warning disable leaves it running,
-proven on hardware — [modules/fs/](modules/fs/README.md)). The corrected channel stays
-untrustworthy; the remaining path is the register-docs ask (Wiitek request sent; the FS
-missing-chapter asks pending).
-
-## 6. X520 bench divergences — features to restore on the product NIC
+## 5. X520 bench divergences — features to restore on the product NIC
 
 Running on the X520 (BCM development) required parking product-NIC capabilities the 82599 lacks. Each stays parked only until the ConnectX-5 is in; none is a settled design change:
 

docs/state.md

@@ -2,7 +2,7 @@
 
 ## Committed tree
 
-AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet-MAC-rx-stamped rate buckets (`SO_TIMESTAMPING` cmsg, `rx_filter=ALL` as a hard host check — nics/README.md for what that demands of the NIC; **temporarily bypassed** in `ts.go` so BCM work can run on the X520, which cannot stamp — the check reports yellow and the panel rates read zero there; restore to fatal for the product NIC); read-time-stamped NIC-counter rates; test interfaces pinned to MTU 9000 with a 9018-byte jumbo in the size mix (the modules' jumbo path is exercised, not assumed); BCM module diagnostics (`phy.go`, over the patched-ixgbe `sff_i2c` debugfs, compound-op framing; one per-module lock around every whole operation; the firmware's own mailbox use still contends, unresolved — modules/fs/ for the full trap list): bringup identifies both modules and forces EEE off and jumbo on every boot — no trustworthy readback exists and no cable is guaranteed to probe through; the ECD — per-pair verdicts, lengths and pair maps are the length/wiring path — runs through one async path at startup and on every reset, never blocking the UI, with counters re-baselining only after the diag's own link blip so it is never charged to the run; a 1 Hz poller feeds per-pair SNR margin (vs the ≈26.5 dB operating point; green ≥ 3 dB, amber ≥ 1 dB — provisional until the graded-noise run) and the corrected-error set (PCS 3.33 errored blocks/BER, PMA 1.147 fast-retrain count) to the panel and console; framebuffer UI; harness.
+AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet-MAC-rx-stamped rate buckets (`SO_TIMESTAMPING` cmsg, `rx_filter=ALL` as a hard host check — nics/README.md for what that demands of the NIC; **temporarily bypassed** in `ts.go` so BCM work can run on the X520, which cannot stamp — the check reports yellow and the panel rates read zero there; restore to fatal for the product NIC); read-time-stamped NIC-counter rates; test interfaces pinned to MTU 9000 with a 9018-byte jumbo in the size mix (the modules' jumbo path is exercised, not assumed); BCM module diagnostics (`phy.go`, over the patched-ixgbe `sff_i2c` debugfs, compound-op framing; one per-module lock around every whole operation, and steady-state work time-division-multiplexed with the firmware's own mailbox client — every operation admitted only inside a 3.4 s window after each observed internal temp poll, which ends the stale-read poisoning of the corrected channel — modules/fs/ for the full trap list and validation): bringup identifies both modules and forces EEE off and jumbo on every boot — no trustworthy readback exists and no cable is guaranteed to probe through; the ECD — per-pair verdicts, lengths and pair maps are the length/wiring path — runs through one async path at startup and on every reset, never blocking the UI, with counters re-baselining only after the diag's own link blip so it is never charged to the run; a 1 Hz poller feeds per-pair SNR margin (vs the ≈26.5 dB operating point; green ≥ 3 dB, amber ≥ 1 dB — provisional until the graded-noise run) and the corrected-error set (PCS 3.33 errored blocks/BER, PMA 1.147 fast-retrain count) to the panel and console; framebuffer UI; harness.
 
 ## Stashes