From a00f2216b707dbb9cf10ac7f9b7d501f3facd23b Mon Sep 17 00:00:00 2001 From: flamingcow Date: Wed, 12 Aug 2026 21:39:46 -0700 Subject: [PATCH] BCM bridge tolerates MCIA offset-write-then-read framing, proven on hardware via bcm_mcia_emu.py; MCIA question narrows to firmware acceptance of 0x56 and offset/framing details --- docs/nics/connectx-5/README.md | 15 +++++++++++++-- docs/open-questions.md | 6 +++--- docs/state.md | 2 +- 3 files changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/nics/connectx-5/README.md b/docs/nics/connectx-5/README.md index b139f0e..0924524 100644 --- a/docs/nics/connectx-5/README.md +++ b/docs/nics/connectx-5/README.md @@ -16,10 +16,21 @@ The public PRM documents only the data path plus the `ACCESS_REGISTER` envelope - **Every read is offset-addressed**: 16-bit `device_address` + 16-bit `size`, 12 data dwords (48 B max per transaction). There is **no raw-read encoding** — a read with no preceding offset cannot even be requested. - **`l` lock bit** — locks the module across consecutive transactions; MCIA's own answer to the two-master interleaving problem. +## BCM SMI through MCIA framing — module side proven + +MCIA has no raw-read encoding (every read emits an offset write first), so SMI reads work only if the BCM bridge tolerates that framing. **It does — proven on hardware** (X520 bit-bang emulation of MCIA's write-offset-then-read, `~/work/phydiag-work/bcm_mcia_emu.py` against the FS BCM): + +- A 1-byte offset write (0x00 and 0xFF tried) injected between SMI command and read phase leaves the pending read data intact — immediately and after 3 ms; the expected register value (PHY ID 0x3590/0x5081) reads back correctly every time. +- A cold offset-write+read returns the last-latched SMI data — the offset byte is ignored, not parsed as a new command frame. +- Normal SMI operation recovers cleanly afterward; no lasting disturbance. +- Caveat: emulated with STOP-separated transactions; a repeated-START combined transfer (if that's what firmware emits) is untested. + +SMI *writes* fit MCIA's model directly if firmware emits a 1-byte offset: an MCIA write emits `[offset][data…]` as one I2C write, so choosing `device_address` = first SMI frame byte synthesizes the 3–5 byte command frame exactly. + ## Open questions (decide on arrival) -- Whether firmware MCIA accepts **arbitrary I2C device addresses** — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51 (`MLX5_I2C_ADDR_LOW/HIGH`). -- Whether the BCM SMI **read data phase** survives MCIA's framing. The format settles half the question: a raw no-offset read has no encoding, so the only path is the bridge *tolerating* an offset write before the read phase. That is a module-side property — **testable now on the X520** by bit-bang-emulating MCIA's write-offset-then-read framing against the FS BCM, before the card arrives. +- Whether firmware MCIA accepts **arbitrary I2C device addresses** — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51 (`MLX5_I2C_ADDR_LOW/HIGH`). The register format allows it and the per-access status byte makes rejection observable. +- Whether MCIA writes emit exactly **one offset byte** (needed for the SMI-frame synthesis above), and whether reads use STOP-separated or repeated-START framing. - RollBall (0x51-resident, offset-model) fits MCIA — low risk. - If MCIA can't reach the BCM: product diagnostics via the Marvell/Aquantia modules only. The X520 is **not** a fallback product NIC — it has no all-packet exact-timestamp path, which a cable tester's rate buckets require ([../x520/README.md](../x520/README.md)). diff --git a/docs/open-questions.md b/docs/open-questions.md index c6d1d44..a4871e2 100644 --- a/docs/open-questions.md +++ b/docs/open-questions.md @@ -6,9 +6,9 @@ The genuinely open, thinking-worthy problems — nothing here is resolved. Analy The decisive unknown for product diagnostics coverage ([nics/connectx-5/](nics/connectx-5/README.md)): -- Does firmware MCIA accept **arbitrary I2C device addresses**? The register format allows it (8-bit field, per-access status — verified in `mlx5_ifc.h`); firmware acceptance is the unknown. -- Does the BCM bridge **tolerate an offset write before its SMI read phase**? MCIA has no raw-read encoding, so this module-side property is the whole question — and it is testable on the X520 before the card arrives. -- If no to either: product diagnostics via the Marvell/Aquantia modules only. +- Does firmware MCIA accept **arbitrary I2C device addresses**? The register format allows it (8-bit field, per-access status — verified in `mlx5_ifc.h`); firmware acceptance is the unknown. The module side is settled: the BCM bridge tolerates MCIA's offset-write-then-read framing, proven on hardware ([nics/connectx-5/](nics/connectx-5/README.md)). +- Does MCIA emit **one offset byte per write** (which would let SMI command frames be synthesized exactly), and STOP-separated or repeated-START reads? +- If firmware says no: product diagnostics via the Marvell/Aquantia modules only. ## 2. ECD run disturbance diff --git a/docs/state.md b/docs/state.md index ae1255c..2cdc7dc 100644 --- a/docs/state.md +++ b/docs/state.md @@ -41,7 +41,7 @@ AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet In `~/work/` alongside the phydiag artifacts, ready to fold into the repo's `kernel/`: - **Patched ixgbe** (`~/work/ixgbe-sff/`): `ixgbe_i2c_raw_write`/`ixgbe_i2c_raw_read` in `ixgbe_phy.c` (arbitrary START…STOP transactions from the existing bit-bang primitives, swfw-bracketed) + `sff_i2c` debugfs file (`w ` / `r ` / `x ` single-hold compound). One real bug found on hardware: multi-byte reads returned only byte 0 — the master's ACK left SDA driven low and nothing released it (stock 82599 paths never clock in more than one byte, so the missing release was invisible). `raw_read` now releases SDA after each ACK, mirroring stock's own release block. The patch also carries: **ETQF steering** (ETHER_FLOW ntuple inserts mapped onto ETQF/ETQS slots in `ixgbe_ethtool.c`, restore-on-up/clear-on-close hooked, FCoE slot-2 write moved behind its enable guard); **`reg_ops` read-returns-value** and a **`ts_bench`** debugfs command (in-kernel latch poll/re-arm for the timestamp bench pokes). `load-ixgbe` rebuilds + swaps the module, passing `allow_unsupported_sfp=1`. -- **Bench tools** (`~/work/phydiag-work/`): `etqfbench/` (Go; proves ETQF steering + the fdir-can't-steer-raw-L2 negative), `x520poke/` (Go; drives `reg_ops`/`ts_bench` for the timestamp/counter pokes), `compound_test.py` (proves the `x` compound op against the FS BCM). +- **Bench tools** (`~/work/phydiag-work/`): `etqfbench/` (Go; proves ETQF steering + the fdir-can't-steer-raw-L2 negative), `x520poke/` (Go; drives `reg_ops`/`ts_bench` for the timestamp/counter pokes), `compound_test.py` (proves the `x` compound op against the FS BCM), `bcm_mcia_emu.py` (proves the BCM bridge tolerates MCIA offset-write-then-read framing — nics/connectx-5/). - **BCM client** (`~/work/phydiag-work/bcm_ixgbe.py`): proven end-to-end on the FS — EEPROM, PHY ID 0x3590:5081, handler STATUS, per-pair SNR ≈ [32, 27.5, 30, 27.6] dB via `CMD_GET_SNR = 0x8030` invoked bare. Code-table trap and stale-DATA1 trap: modules/fs/. - **Exploration probes** (`bcm_explore.py`, `bcm_eee_off.py`): full GET sweep + the AutogrEEEn force-off recipe (results: modules/fs/). - **RollBall client** (`~/work/phydiag-work/rollball_ixgbe.py`): same transport; unlock/page/mailbox, per-pair IEEE SNR. Untested on this card. The `*.0x??64` brick blacklist is a hard guard that raises *before* touching hardware — the client structurally cannot repeat the kill.