4.5 KiB
Intel X520-DA2 (82599, ixgbe)
The only certain arbitrary-framing module-I2C transport (bit-banged, no firmware in the path) — the diagnostics workhorse, and only that. It cannot run the measurement path and is not a fallback product NIC: a cable tester needs exact per-frame RX timestamps, the 82599 has one serial PTP-only latch and no all-packet path, and aggregate counters/rates are honest only when the receive is well-behaved — useless for catching the fault (../../open-questions.md §2). The ConnectX-5 is the product NIC.
- PCIe Gen2 ×8 (5 GT/s, 32 Gb/s) — enough for 2×10G full duplex at the default mix.
allow_unsupported_sfp=1is mandatory (ixgbe_main.c:165): the FS module's honest 10GBASE-T EEPROM fails Intel qualification and kills the whole port probe (error -95, no netdev).load-ixgbepasses it.
sff_i2c diagnostics transport (patched driver, validated)
Patched ixgbe (~/work/ixgbe-sff/) adds ixgbe_i2c_raw_write/ixgbe_i2c_raw_read (ixgbe_phy.c) — arbitrary START…STOP transactions from the driver's bit-bang primitives, swfw-semaphore bracketed — behind /sys/kernel/debug/ixgbe/<pci>/sff_i2c (w <addr8> <bytes…> / r <addr8> <n>).
- SDA-release fix (required): multi-byte reads returned only byte 0 — the master's ACK left SDA driven low and nothing released it (stock 82599 paths never clock in more than one byte, so the missing release was invisible).
raw_readreleases SDA after each ACK, mirroring stock's release block after clocking out a byte. - Two-master windows closed by the compound op: each simple
w/rtakes the semaphore separately, so multi-op protocol sequences have unlocked gaps where the driver's event-driven I2C (SFP identify, link setup,ethtool -m) can interleave. Thex <waddr> <raddr> <delay_us> <n> <wbytes…>command (ixgbe_i2c_raw_xfer) runs write→delay→read under a single swfw hold, removing the window. Proven on the FS BCM bridge (../../open-questions.md §4).
Register/timestamp bench transport (patched driver)
For the §2 exploration the driver also carries reg_ops read <reg> returning the value through the file (not just dmesg) and a ts_bench command running the latch poll/re-arm loop in-kernel (arm <ctl>, control, latchrate <ms>) — needed because the BAR is unmappable from userspace under IO_STRICT_DEVMEM. Poke driver: ~/work/phydiag-work/x520poke.
Timestamping facts
No all-packet exact RX timestamping, and no path to it — this disqualifies the X520 as product NIC. A cable tester needs each frame's exact arrival time; the 82599 cannot give it for the 0x88b5 test traffic.
- RX latches PTP frames only — confirmed on hardware. Writing
TYPE_ALL(0x08) into TSYNCRXCTL sticks in the register but never arms the latch under raw-frame flood;TSIP_UT_EN(0x00800000) will not even hold (both are X550+;HWTSTAMP_FILTER_ALL→ -ERANGE,ixgbe_ptp.c:1043). One serial latch pair (RXSTMPL/H, locked until RXSTMPH read), attribute capture (RXSATRL/H — stream seq lands in RXSATRH[31:16] byte-swapped), no RX-descriptor timestamp field. - The single latch services ~383 k stamps/s in a dedicated in-kernel poll+re-arm loop (measured) against ~1.78 M pps/dir line rate — serial and PTP-only, so it stamps a fraction of frames at best and none of the raw traffic. Its only surviving use is the low-volume length probes (§2/§3).
- Per-port free-running SYSTIME (~6.4 ns at 10G), one shared crystal — two-way exchange cancels the inter-port offset (length-probe use only).
- Per-queue counters are not a substitute: they're a rate, honest only in zero-loss steady state, and a cable tester measures the receive precisely when it is not. Measured facts (RQSMR maps 1:1; QPRC/QBRC exact, read-clear; the 2 s stats watchdog is a competing reader,
ixgbe_main.c:7780,7789,7822) are kept in ../../open-questions.md §2, but the route is rejected.
RX steering facts
- Stock ntuple ETHER_FLOW is rejected (
ixgbe_flowspec_to_flow_type: TCP/UDP/SCTP-v4 and IPv4 only). The patched driver maps exact-ethertype ETHER_FLOW inserts onto ETQF slots, so the tool runs unchanged. - ETQF/ETQS L2 EtherType steering is proven on hardware: dedicated ethertype→queue stage ahead of RSS/fdir; 7 streams land on 7 queues, unfiltered ethertypes fall to queue 0. Flow Director cannot match non-IP frames at all (a flex-word rule on raw 0x88b5 steers nothing — confirmed). Details and the FCoE-slot fix: ../../open-questions.md §1.