Files
cabletest/docs/nics/x520/README.md
T

4.5 KiB
Raw Blame History

Intel X520-DA2 (82599, ixgbe)

The only certain arbitrary-framing module-I2C transport (bit-banged, no firmware in the path) — the diagnostics workhorse, and only that. It cannot run the measurement path and is not a fallback product NIC: a cable tester needs exact per-frame RX timestamps, the 82599 has one serial PTP-only latch and no all-packet path, and aggregate counters/rates are honest only when the receive is well-behaved — useless for catching the fault (../../open-questions.md §2). The ConnectX-5 is the product NIC.

  • PCIe Gen2 ×8 (5 GT/s, 32 Gb/s) — enough for 2×10G full duplex at the default mix.
  • allow_unsupported_sfp=1 is mandatory (ixgbe_main.c:165): the FS module's honest 10GBASE-T EEPROM fails Intel qualification and kills the whole port probe (error -95, no netdev). load-ixgbe passes it.

sff_i2c diagnostics transport (patched driver, validated)

Patched ixgbe (~/work/ixgbe-sff/) adds ixgbe_i2c_raw_write/ixgbe_i2c_raw_read (ixgbe_phy.c) — arbitrary START…STOP transactions from the driver's bit-bang primitives, swfw-semaphore bracketed — behind /sys/kernel/debug/ixgbe/<pci>/sff_i2c (w <addr8> <bytes…> / r <addr8> <n>).

  • SDA-release fix (required): multi-byte reads returned only byte 0 — the master's ACK left SDA driven low and nothing released it (stock 82599 paths never clock in more than one byte, so the missing release was invisible). raw_read releases SDA after each ACK, mirroring stock's release block after clocking out a byte.
  • Two-master windows closed by the compound op: each simple w/r takes the semaphore separately, so multi-op protocol sequences have unlocked gaps where the driver's event-driven I2C (SFP identify, link setup, ethtool -m) can interleave. The x <waddr> <raddr> <delay_us> <n> <wbytes…> command (ixgbe_i2c_raw_xfer) runs write→delay→read under a single swfw hold, removing the window. Proven on the FS BCM bridge (../../open-questions.md §4).

Register/timestamp bench transport (patched driver)

For the §2 exploration the driver also carries reg_ops read <reg> returning the value through the file (not just dmesg) and a ts_bench command running the latch poll/re-arm loop in-kernel (arm <ctl>, control, latchrate <ms>) — needed because the BAR is unmappable from userspace under IO_STRICT_DEVMEM. Poke driver: ~/work/phydiag-work/x520poke.

Timestamping facts

No all-packet exact RX timestamping, and no path to it — this disqualifies the X520 as product NIC. A cable tester needs each frame's exact arrival time; the 82599 cannot give it for the 0x88b5 test traffic.

  • RX latches PTP frames only — confirmed on hardware. Writing TYPE_ALL (0x08) into TSYNCRXCTL sticks in the register but never arms the latch under raw-frame flood; TSIP_UT_EN (0x00800000) will not even hold (both are X550+; HWTSTAMP_FILTER_ALL → -ERANGE, ixgbe_ptp.c:1043). One serial latch pair (RXSTMPL/H, locked until RXSTMPH read), attribute capture (RXSATRL/H — stream seq lands in RXSATRH[31:16] byte-swapped), no RX-descriptor timestamp field.
  • The single latch services ~383 k stamps/s in a dedicated in-kernel poll+re-arm loop (measured) against ~1.78 M pps/dir line rate — serial and PTP-only, so it stamps a fraction of frames at best and none of the raw traffic. Its only surviving use is the low-volume length probes (§2/§3).
  • Per-port free-running SYSTIME (~6.4 ns at 10G), one shared crystal — two-way exchange cancels the inter-port offset (length-probe use only).
  • Per-queue counters are not a substitute: they're a rate, honest only in zero-loss steady state, and a cable tester measures the receive precisely when it is not. Measured facts (RQSMR maps 1:1; QPRC/QBRC exact, read-clear; the 2 s stats watchdog is a competing reader, ixgbe_main.c:7780,7789,7822) are kept in ../../open-questions.md §2, but the route is rejected.

RX steering facts

  • Stock ntuple ETHER_FLOW is rejected (ixgbe_flowspec_to_flow_type: TCP/UDP/SCTP-v4 and IPv4 only). The patched driver maps exact-ethertype ETHER_FLOW inserts onto ETQF slots, so the tool runs unchanged.
  • ETQF/ETQS L2 EtherType steering is proven on hardware: dedicated ethertype→queue stage ahead of RSS/fdir; 7 streams land on 7 queues, unfiltered ethertypes fall to queue 0. Flow Director cannot match non-IP frames at all (a flex-word rule on raw 0x88b5 steers nothing — confirmed). Details and the FCoE-slot fix: ../../open-questions.md §1.