Files
cabletest/docs/nics/connectx-5/README.md
T

1.5 KiB

ConnectX-5 (mlx5) — the product NIC candidate

Dual SFP28, PCIe x8. mlx5 is the one driver meeting the full requirement set:

  • CQE-stamps every RX packet (HWTSTAMP_FILTER_ALL) — the committed bucketing and probe.go run unchanged.
  • One shared clock across both portsprobe.go's shared-PHC assumption holds.
  • Native ETHER_FLOW ntuple steering — no driver patch, no encap needed.
  • Mature driver; SFP+ modules drop into SFP28 cages at 10G.

Open questions (decide on arrival)

  • Whether firmware MCIA accepts arbitrary I2C device addresses — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51.
  • Whether the BCM SMI read data phase (raw 2-byte read, no offset write) can be framed through MCIA's offset model.
  • RollBall (0x51-resident, offset-model) fits MCIA — low risk.
  • If MCIA can't reach the BCM: product diagnostics via the Marvell/Aquantia modules only, X520 kept as an offline BCM rig. The X520 is not a fallback product NIC — it has no all-packet exact-timestamp path, which a cable tester's rate buckets require (../x520/README.md).

Arrival notes

  • Ports may ship in InfiniBand mode: mlxconfig -d <pci> set LINK_TYPE_P1=2 LINK_TYPE_P2=2, reboot.
  • Check whether auto-loading mlx5_ib interferes with channel changes (the irdma lesson, hardware.md).
  • Force 10G if SFP28 autoneg is stubborn: ethtool -s <dev> speed 10000 autoneg off.
  • Mellanox is permissive with third-party modules (no Intel-style qualification lockout) — verify with the lying-EEPROM clones anyway.