diff --git a/docs/goals.md b/docs/goals.md index 2744910..14fce56 100644 --- a/docs/goals.md +++ b/docs/goals.md @@ -6,7 +6,7 @@ The target of measurement is the cable, not throughput. Line rate is a means to 1. **Loss and error attribution** — reception gaps, link errors, NIC/driver counters as first-class output alongside application loss. Baseline loss must be exactly zero before a run counts; any host-side loss masks real cable faults. 2. **Noise tolerance** — a deliberately-bad "noise" cable intertwined with the test cable, driven by link up/down cycling, stresses the cable under test with alien crosstalk. -3. **Per-pair SNR margin** from the module PHYs (IEEE 802.3an standard registers) — the leading indicator of a marginal cable before it drops frames. +3. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (transports.md). 4. **Cable length** — sanity check and fault localization. Sources, in preference order: - Module PHY DSP estimate — works on a linked cable. - PHY TDR — localizes opens/shorts both-ended; healthy-cable length only single-ended. diff --git a/docs/state.md b/docs/state.md index 07d36f5..6057af0 100644 --- a/docs/state.md +++ b/docs/state.md @@ -15,17 +15,18 @@ AF_PACKET raw sockets everywhere (`sock.go`), flow-director steering, per-packet - 1× Fibergaga SFP-10G-T-30M (Aquantia, RollBall, the proven oracle module). - 1× 10Gtek, EEPROM claims SFP-10G-SR — still a copper RJ45 module; filler, not part of the test set. -## X520 prep — built ahead of the card, untested (no hardware yet) -In `~/work/` alongside the ice-sff/phydiag-work artifacts, ready to fold into the repo's `kernel/` once validated on real hardware: -- **Patched ixgbe** (`~/work/ixgbe-sff/`, built against the running kernel, vermagic matches): adds `ixgbe_i2c_raw_write`/`ixgbe_i2c_raw_read` in `ixgbe_phy.c` (arbitrary START…STOP transactions built from the existing bit-bang primitives, swfw-semaphore bracketed) and an `sff_i2c` debugfs file with `w ` / `r ` commands, result read back from the fd. Modeled on the proven ice `sff_i2c`; ixgbe has no firmware to refuse, so arbitrary framing works. `load-ixgbe` rebuilds + swaps the module. -- **BCM client** (`~/work/phydiag-work/bcm_ixgbe.py`): BCM SMI framing on those two ops (read = write `[001+devad,RegH,RegL]`, >1 ms delay, read 2 B; write = 5-byte frame) plus the full command handler (`_wait_idle` → DATA → CMD|bit15 → poll PASS/ERROR → read DATA), `CMD_GET_SNR` decoded to per-pair dB, and EEPROM/PHY-ID/STATUS sanity reads. -- **RollBall client for the Wiiteks** (`~/work/phydiag-work/rollball_ixgbe.py`): the same `sff_i2c` transport, RollBall unlock/page/mailbox, per-pair IEEE SNR (PMA 1.133–1.136). The `*.0x??64` (high-byte ≥ 0x80) brick blacklist is a hard guard that raises *before* touching hardware — the client structurally cannot repeat the kill. +## X520 diagnostics path — validated on hardware +In `~/work/` alongside the ice-sff/phydiag-work artifacts, ready to fold into the repo's `kernel/`: +- **Patched ixgbe** (`~/work/ixgbe-sff/`): adds `ixgbe_i2c_raw_write`/`ixgbe_i2c_raw_read` in `ixgbe_phy.c` (arbitrary START…STOP transactions built from the existing bit-bang primitives, swfw-semaphore bracketed) and an `sff_i2c` debugfs file with `w ` / `r ` commands, result read back from the fd. **Proven on the card after one real bug**: multi-byte reads returned only byte 0 — the master's ACK left SDA driven low and nothing released it (stock 82599 paths never clock in more than one byte, so the missing release was invisible until now); `raw_read` now releases SDA after each ACK, mirroring the release block stock uses after clocking out a byte. `load-ixgbe` rebuilds + swaps the module and passes `allow_unsupported_sfp=1` (mandatory for the FS). +- **BCM client** (`~/work/phydiag-work/bcm_ixgbe.py`): BCM SMI framing proven end-to-end on the FS — EEPROM, PHY ID 0x3590:5081, handler STATUS, and per-pair SNR ≈ [32, 27.5, 30, 27.6] dB on the bench cable via `CMD_GET_SNR = 0x8030` invoked with no DATA1 write (see transports.md for the code-table trap and the pause-mode side quest). +- **RollBall client for the Wiiteks** (`~/work/phydiag-work/rollball_ixgbe.py`): the same `sff_i2c` transport, RollBall unlock/page/mailbox, per-pair IEEE SNR (PMA 1.133–1.136), untested on this card. The `*.0x??64` (high-byte ≥ 0x80) brick blacklist is a hard guard that raises *before* touching hardware — the client structurally cannot repeat the kill. ## Bringup plan when the X520 arrives -1. Card installed and modules seated (done). Next: `./load-ixgbe` with `allow_unsupported_sfp=1` (mandatory — the FS's honest 10GBASE-T EEPROM fails Intel qualification and kills the port probe; `ixgbe_main.c:165`), confirm `sff_i2c` appears. -2. FS/BCM modules first: `bcm_ixgbe.py` sanity reads (EEPROM, PHY ID, STATUS) to prove the SMI transport, then the command handler (CMD_GET_SNR vs the IEEE registers) on the test cable set. +1. Card installed, modules seated, `./load-ixgbe` with `allow_unsupported_sfp=1` (mandatory — the FS's honest 10GBASE-T EEPROM fails Intel qualification and kills the port probe; `ixgbe_main.c:165`), `sff_i2c` present. Done. +2. FS/BCM proven: SMI transport, EEPROM/PHY-ID/STATUS sanity, command handler (temp, voltage, per-pair SNR). The IEEE-register SNR path is dead on the BCM — the command handler is the source. Done. 3. Replacement Wiiteks: IEEE-standard registers only (SNR, latches) unless/until a documented recipe exists for more; the VCT templates in modules.md are candidates for single-shot targeted probes on a sacrificial unit only. 4. Re-derive host tuning on ixgbe (coalescing/ring syntax differs). +5. RX steering: program/verify the ETQF path (open-questions.md §1) before cabletest traffic runs. ## Open: RX flow-steering on ixgbe needs serious investigation before the X520 carries test traffic cabletest fans RX across queues by steering **raw ethertype** (0x88b5 base) to distinct queues with Flow Director rules (`system.go`, ETHTOOL_SRXCLSRLINS, ETHER_FLOW). The obvious path fails: ixgbe/82599's `ixgbe_flowspec_to_flow_type` accepts only TCP/UDP/SCTP-v4 and IPv4 — **no ETHER_FLOW** — so an ETHER_FLOW rule insert is rejected (EINVAL) and the tool fails at startup (no fallback, by design). This is a datapath issue affecting *both* module types, independent of diagnostics. diff --git a/docs/transports.md b/docs/transports.md index 1755772..8b988a9 100644 --- a/docs/transports.md +++ b/docs/transports.md @@ -49,9 +49,13 @@ Docs are in this folder (`10GBase-T...BCM84891.pdf` transport, `BCM84891L-MDIO C 4. Poll STATUS for PASS/ERROR. 5. Read DATAn for results. - Poll ~100 ms; STATUS is frozen up to 2 s during 10GBASE-T training, so only run after link up. -- **Commands**: - - **CMD_GET_SNR** (GET commands 0x8000+; detail §1.25.1.24): DATA2/10..DATA5/10 = per-pair SNR dB (channels A–D). - - Also pair swap/skew/polarity, die temp, voltage, 1588. +- **Commands** (codes verified on hardware — the PDF's command-table columns misalign under text extraction; the check that settled it was GET_CURRENT_VOLTAGE = 0x802F returning the 0.8/1.88 V rails in tenths of mV): + - **CMD_GET_SNR = 0x8030** — invoke with **no DATA1 write**; writing the documented display flag makes this firmware return zeros. Bare invocation returns DATA2–5 = per-pair SNR (channels A–D) in tenths of dB. Bench: 27–32 dB absolute, ~0.1–0.4 dB read-to-read jitter. This is *absolute* receiver SNR, not margin (10GBASE-T operating point ≈ 26.5 dB). + - The IEEE PMA SNR registers 1.133–1.140 read a constant 0x8080 on the BCM — never populated, display flag inert. **On FS modules SNR comes from the command handler, not standard registers.** + - **CMD_GET_CURRENT_TEMP = 0x8031** (die runs ~70 °C on bench), **GET_CURRENT_VOLTAGE = 0x802F**. + - **Trap: 0x8020 is SET_PAUSE_FRAME_MODE** — a config *write*, and the handler executes whatever stale DATA1 holds as its argument. Module pause is explicitly set to 0 (off — the state cabletest requires); GET_PAUSE_FRAME_MODE (0x801F) returns junk on this firmware, so treat the setting as write-only. + - The >1 ms SMI read delay: 3 ms works reliably. + - Also pair swap/skew/polarity, 1588. - **Enhanced Cable Diagnostics** (shorts/opens/**cable length**) is a headline feature, but its invocation is NOT in the command-code table (0x8000–0x805B) — it's a separate ECD register mechanism whose chapter isn't in the docs FS sent. This is the one outstanding ask to FS. **Correct BCM SMI framing over the E810 topo-I2C** (recorded in case a write-capable multi-byte transport reappears): read = write `[001+devad,RegH,RegL]` offset_size=1, STOP, >1 ms, then read 2 B offset_size=0; write = offset_size=2, addr = devad<<8|RegH, data = `[RegL,DataH,DataL]`.