# ConnectX-5 (mlx5) — the product NIC candidate

Dual SFP28, PCIe x8. mlx5 is the one driver meeting the full requirement set:

- **CQE-stamps every RX packet** (`HWTSTAMP_FILTER_ALL`) — the committed bucketing and `probe.go` run unchanged.
- **One shared clock across both ports** — `probe.go`'s shared-PHC assumption holds.
- **Native ETHER_FLOW ntuple steering** — no driver patch, no encap needed.
- Mature driver; SFP+ modules drop into SFP28 cages at 10G.

## MCIA register format (verified in kernel headers)

The public PRM documents only the data path plus the `ACCESS_REGISTER` envelope — MCIA's definition lives in NDA space; `struct mlx5_ifc_mcia_reg_bits` (`include/linux/mlx5/mlx5_ifc.h`) is the working reference:

- **`i2c_device_address` is a full 8-bit field** — 0x56 is expressible in the register; the open part is purely whether firmware accepts it.
- **8-bit `status` per access** — firmware reports the result, so an address rejection or I2C failure is observable, not silent.
- **Every read is offset-addressed**: 16-bit `device_address` + 16-bit `size`, 12 data dwords (48 B max per transaction). There is **no raw-read encoding** — a read with no preceding offset cannot even be requested.
- **`l` lock bit** — locks the module across consecutive transactions; MCIA's own answer to the two-master interleaving problem.

## Open questions (decide on arrival)

- Whether firmware MCIA accepts **arbitrary I2C device addresses** — the BCM bridge sits at 0x56; kernel paths only ever use 0x50/0x51 (`MLX5_I2C_ADDR_LOW/HIGH`).
- Whether the BCM SMI **read data phase** survives MCIA's framing. The format settles half the question: a raw no-offset read has no encoding, so the only path is the bridge *tolerating* an offset write before the read phase. That is a module-side property — **testable now on the X520** by bit-bang-emulating MCIA's write-offset-then-read framing against the FS BCM, before the card arrives.
- RollBall (0x51-resident, offset-model) fits MCIA — low risk.
- If MCIA can't reach the BCM: product diagnostics via the Marvell/Aquantia modules only. The X520 is **not** a fallback product NIC — it has no all-packet exact-timestamp path, which a cable tester's rate buckets require ([../x520/README.md](../x520/README.md)).

## Arrival notes

- Ports may ship in InfiniBand mode: `mlxconfig -d <pci> set LINK_TYPE_P1=2 LINK_TYPE_P2=2`, reboot.
- Check whether auto-loading `mlx5_ib` interferes with channel changes (the irdma lesson, hardware.md).
- Force 10G if SFP28 autoneg is stubborn: `ethtool -s <dev> speed 10000 autoneg off`.
- Mellanox is permissive with third-party modules (no Intel-style qualification lockout) — verify with the lying-EEPROM clones anyway.