32 lines
4.5 KiB
Markdown
32 lines
4.5 KiB
Markdown
# Intel X520-DA2 (82599, ixgbe)
|
||
|
||
The only *certain* arbitrary-framing module-I2C transport (bit-banged, no firmware in the path) — the diagnostics workhorse, and **only** that. It cannot run the measurement path and is not a fallback product NIC: a cable tester needs exact per-frame RX timestamps, the 82599 has one serial PTP-only latch and no all-packet path, and aggregate counters/rates are honest only when the receive is well-behaved — useless for catching the fault ([../../open-questions.md](../../open-questions.md) §2). The ConnectX-5 is the product NIC.
|
||
|
||
- PCIe Gen2 ×8 (5 GT/s, 32 Gb/s) — enough for 2×10G full duplex at the default mix.
|
||
- **`allow_unsupported_sfp=1` is mandatory** (`ixgbe_main.c:165`): the FS module's honest 10GBASE-T EEPROM fails Intel qualification and kills the whole port probe (error -95, no netdev). `load-ixgbe` passes it.
|
||
|
||
## `sff_i2c` diagnostics transport (patched driver, validated)
|
||
|
||
Patched ixgbe (`~/work/ixgbe-sff/`) adds `ixgbe_i2c_raw_write`/`ixgbe_i2c_raw_read` (`ixgbe_phy.c`) — arbitrary START…STOP transactions from the driver's bit-bang primitives, swfw-semaphore bracketed — behind `/sys/kernel/debug/ixgbe/<pci>/sff_i2c` (`w <addr8> <bytes…>` / `r <addr8> <n>`).
|
||
|
||
- **SDA-release fix (required)**: multi-byte reads returned only byte 0 — the master's ACK left SDA driven low and nothing released it (stock 82599 paths never clock in more than one byte, so the missing release was invisible). `raw_read` releases SDA after each ACK, mirroring stock's release block after clocking out a byte.
|
||
- **Two-master windows closed by the compound op**: each simple `w`/`r` takes the semaphore separately, so multi-op protocol sequences have unlocked gaps where the driver's event-driven I2C (SFP identify, link setup, `ethtool -m`) can interleave. The `x <waddr> <raddr> <delay_us> <n> <wbytes…>` command (`ixgbe_i2c_raw_xfer`) runs write→delay→read under a single swfw hold, removing the window. Proven on the FS BCM bridge ([../../open-questions.md](../../open-questions.md) §4).
|
||
|
||
## Register/timestamp bench transport (patched driver)
|
||
|
||
For the §2 exploration the driver also carries `reg_ops` `read <reg>` returning the value through the file (not just dmesg) and a `ts_bench` command running the latch poll/re-arm loop in-kernel (`arm <ctl>`, `control`, `latchrate <ms>`) — needed because the BAR is unmappable from userspace under `IO_STRICT_DEVMEM`. Poke driver: `~/work/phydiag-work/x520poke`.
|
||
|
||
## Timestamping facts
|
||
|
||
**No all-packet exact RX timestamping, and no path to it — this disqualifies the X520 as product NIC.** A cable tester needs each frame's exact arrival time; the 82599 cannot give it for the 0x88b5 test traffic.
|
||
|
||
- RX latches **PTP frames only** — confirmed on hardware. Writing `TYPE_ALL` (0x08) into TSYNCRXCTL sticks in the register but never arms the latch under raw-frame flood; `TSIP_UT_EN` (0x00800000) will not even hold (both are X550+; `HWTSTAMP_FILTER_ALL` → -ERANGE, `ixgbe_ptp.c:1043`). One serial latch pair (RXSTMPL/H, locked until RXSTMPH read), attribute capture (RXSATRL/H — stream seq lands in RXSATRH[31:16] byte-swapped), no RX-descriptor timestamp field.
|
||
- The single latch services ~383 k stamps/s in a dedicated in-kernel poll+re-arm loop (measured) against ~1.78 M pps/dir line rate — serial and PTP-only, so it stamps a fraction of frames at best and none of the raw traffic. Its only surviving use is the low-volume length probes (§2/§3).
|
||
- Per-port free-running SYSTIME (~6.4 ns at 10G), one shared crystal — two-way exchange cancels the inter-port offset (length-probe use only).
|
||
- **Per-queue counters are not a substitute**: they're a rate, honest only in zero-loss steady state, and a cable tester measures the receive precisely when it is not. Measured facts (RQSMR maps 1:1; QPRC/QBRC exact, read-clear; the 2 s stats watchdog is a competing reader, `ixgbe_main.c:7780,7789,7822`) are kept in [../../open-questions.md](../../open-questions.md) §2, but the route is rejected.
|
||
|
||
## RX steering facts
|
||
|
||
- Stock ntuple ETHER_FLOW is rejected (`ixgbe_flowspec_to_flow_type`: TCP/UDP/SCTP-v4 and IPv4 only). The patched driver maps exact-ethertype ETHER_FLOW inserts onto ETQF slots, so the tool runs unchanged.
|
||
- **ETQF/ETQS L2 EtherType steering is proven on hardware**: dedicated ethertype→queue stage ahead of RSS/fdir; 7 streams land on 7 queues, unfiltered ethertypes fall to queue 0. Flow Director cannot match non-IP frames at all (a flex-word rule on raw 0x88b5 steers nothing — confirmed). Details and the FCoE-slot fix: [../../open-questions.md](../../open-questions.md) §1.
|