Answer open questions from desk analysis: ETQF steering, E820 is post-FEC, compound I2C op, forced co-location

This commit is contained in:
flamingcow
2026-08-10 11:51:39 -07:00
parent 8c8e0c7d32
commit 73074040e0
2 changed files with 48 additions and 4 deletions
+5 -4
View File
@@ -28,10 +28,11 @@ In `~/work/` alongside the ice-sff/phydiag-work artifacts, ready to fold into th
## Open: RX flow-steering on ixgbe needs serious investigation before the X520 carries test traffic
cabletest fans RX across queues by steering **raw ethertype** (0x88b5 base) to distinct queues with Flow Director rules (`system.go`, ETHTOOL_SRXCLSRLINS, ETHER_FLOW). The obvious path fails: ixgbe/82599's `ixgbe_flowspec_to_flow_type` accepts only TCP/UDP/SCTP-v4 and IPv4 — **no ETHER_FLOW** — so an ETHER_FLOW rule insert is rejected (EINVAL) and the tool fails at startup (no fallback, by design). This is a datapath issue affecting *both* module types, independent of diagnostics.
But 82599 has hardware steering machinery the current code doesn't use, and **there is good reason to think we can keep true raw-L2 ethertype steering without falling back to encapsulation.** This needs bench investigation on the actual card; do not assume the answer is UDP encap. Leads, most promising first:
- **Flow Director flex-byte match.** The driver exposes a 2-byte `flex_bytes` filter (via the `user-def`/FLOW_EXT `vlan_etype` field), and `ixgbe_reinit_fdir_tables_82599` already programs `FDIRCTRL_FLEX_SHIFT = 0x6` — offset 12 bytes, **exactly the ethertype field** of an Ethernet frame. So the hardware is already sampling the ethertype. Open question to settle on hardware: whether a non-IP raw frame classifies into an fdir flow_type the rule engine will match, given the driver requires an IP base flow_type in the rule spec — may need a small driver change to pair flex_bytes with a permissive/L2 flow type.
- **MAC → VMDq pool steering.** 82599 assigns unicast MAC (RAR) entries to VMDq pools and pools to RX queues (`hw_set_rar` VMDq pool/queue selection). Distinct dest MACs per stream → distinct queues, fully raw Ethernet, no IP at all. Worth prototyping alongside the flex-byte path.
- **RSS on a flexible field** — 82599 RSS hashes IP tuples, not raw L2, so unlikely to help without encap; noted only to rule out.
But 82599 has hardware steering machinery the current code doesn't use, and **there is good reason to think we can keep true raw-L2 ethertype steering without falling back to encapsulation.** This needs bench investigation on the actual card; do not assume the answer is UDP encap. Leads, most promising first (full analysis in open-questions.md §1):
- **L2 EtherType queue filters (ETQF/ETQS) — the answer on paper.** A dedicated ethertype→RX-queue stage checked at L2 ahead of RSS/fdir; 8 slots, 7 free in our config (index 3 reserved for 1588), already used by ixgbe for 0x88F7/FCoE/LLDP. Patch: map exact-ethertype ETHER_FLOW ntuple inserts onto free ETQF slots so `system.go` works unchanged. Bench-confirm on the card.
- **Flow Director flex-byte match — ruled out on paper.** 82599 fdir classifies IPv4/IPv6 only; the flex word narrows an IP-typed match but can't create a non-IP one, `FDIRCTRL_FLEX_SHIFT = 0x6` (offset 12 = ethertype) notwithstanding — non-IP frames never enter the fdir lookup. One-shot bench confirmation alongside the ETQF test.
- **MAC → VMDq pool steering.** 82599 assigns unicast MAC (RAR) entries to VMDq pools and pools to RX queues (`hw_set_rar` VMDq pool/queue selection). Distinct dest MACs per stream → distinct queues, fully raw Ethernet, no IP at all. Fallback if ETQF disappoints.
- **RSS on a flexible field** — 82599 RSS hashes IP tuples, not raw L2; non-IP frames hash to 0 and land on queue 0. Ruled out (but explains the no-filter baseline).
Fallbacks if raw-L2 steering genuinely can't be made to work: minimal bare-IPv4 framing (no UDP) steered by IP_USER_FLOW src/dst IP (lighter than the old UDP-encap plan), or single-queue RX (caps near the old ~1.6 Mpps single-NAPI ceiling, loses small-frame headroom and per-stream isolation). These are last resorts, not the plan.