docs/nics/connectx-5/README.md

# ConnectX-5 (mlx5) — the product NIC candidate

Dual SFP28, PCIe x8. mlx5 is the one driver meeting the full requirement set:

- **CQE-stamps every RX packet** (`HWTSTAMP_FILTER_ALL`) — the committed bucketing and `probe.go` run unchanged.
- **One shared clock across both ports** — `probe.go`'s shared-PHC assumption holds.
- **Native ETHER_FLOW ntuple steering** — no driver patch, no encap needed.
- Mature driver; SFP+ modules drop into SFP28 cages at 10G.

## Open questions (decide on arrival)

- Whether firmware MCIA accepts **arbitrary I2C device addresses** — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51.
- Whether the BCM SMI **read data phase** (raw 2-byte read, no offset write) can be framed through MCIA's offset model.
- RollBall (0x51-resident, offset-model) fits MCIA — low risk.
- If MCIA can't reach the BCM: product diagnostics via the Marvell/Aquantia modules only, X520 kept as an offline BCM rig. The X520 is **not** a fallback product NIC — it has no all-packet exact-timestamp path, which a cable tester's rate buckets require ([../x520/README.md](../x520/README.md)).

## Arrival notes

- Ports may ship in InfiniBand mode: `mlxconfig -d <pci> set LINK_TYPE_P1=2 LINK_TYPE_P2=2`, reboot.
- Check whether auto-loading `mlx5_ib` interferes with channel changes (the irdma lesson, hardware.md).
- Force 10G if SFP28 autoneg is stubborn: `ethtool -s <dev> speed 10000 autoneg off`.
- Mellanox is permissive with third-party modules (no Intel-style qualification lockout) — verify with the lying-EEPROM clones anyway.
Restructure docs: per-device directories under modules/ and nics/ with co-located datasheets, state centralized in state.md, prose tightened into lists and tables 2026-08-12 19:23:02 -07:00			`# ConnectX-5 (mlx5) — the product NIC candidate`

			`Dual SFP28, PCIe x8. mlx5 is the one driver meeting the full requirement set:`

			- CQE-stamps every RX packet (`HWTSTAMP_FILTER_ALL`) — the committed bucketing and `probe.go` run unchanged.
			- One shared clock across both ports — `probe.go`'s shared-PHC assumption holds.
			`- Native ETHER_FLOW ntuple steering — no driver patch, no encap needed.`
			`- Mature driver; SFP+ modules drop into SFP28 cages at 10G.`

			`## Open questions (decide on arrival)`

			`- Whether firmware MCIA accepts arbitrary I2C device addresses — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51.`
			`- Whether the BCM SMI read data phase (raw 2-byte read, no offset write) can be framed through MCIA's offset model.`
			`- RollBall (0x51-resident, offset-model) fits MCIA — low risk.`
Dissolve resolved analyses into standalone device docs; open-questions.md keeps only the four live problems; discard NIC-timestamp length path (BCM ECD is proven) and Aquantia length trivia (30m part, oracle-only) 2026-08-12 21:30:16 -07:00			`- If MCIA can't reach the BCM: product diagnostics via the Marvell/Aquantia modules only, X520 kept as an offline BCM rig. The X520 is not a fallback product NIC — it has no all-packet exact-timestamp path, which a cable tester's rate buckets require ([../x520/README.md](../x520/README.md)).`
Restructure docs: per-device directories under modules/ and nics/ with co-located datasheets, state centralized in state.md, prose tightened into lists and tables 2026-08-12 19:23:02 -07:00
			`## Arrival notes`

			- Ports may ship in InfiniBand mode: `mlxconfig -d <pci> set LINK_TYPE_P1=2 LINK_TYPE_P2=2`, reboot.
			- Check whether auto-loading `mlx5_ib` interferes with channel changes (the irdma lesson, hardware.md).
			- Force 10G if SFP28 autoneg is stubborn: `ethtool -s <dev> speed 10000 autoneg off`.
			`- Mellanox is permissive with third-party modules (no Intel-style qualification lockout) — verify with the lying-EEPROM clones anyway.`