Files

3.2 KiB
Raw Permalink Blame History

Host rules

Box contents, card placement, and interface assignments live in state.md. This file carries the rules that stay true regardless of what's plugged in.

Interfaces

  • Test pair: the two ports whose modules hold the cable under test. Find by driver name, never by ethN — names shift with kernel link order.
  • Noise pair: driven bad on purpose, link up/down cycled, intertwined with the test cable to inject alien crosstalk.
  • LAN uplink: carries the default route. Never repurpose or down it; identify by default route, not name.
  • Test-path ports live in NetworkManager's unmanaged list (/etc/NetworkManager/conf.d/99-unmanaged-10g.conf), up with no IPv4.

Module identities, EEPROM honesty (and why the X520 needs allow_unsupported_sfp=1), and the physical-layer model live in modules/.

Host tuning (resets every reboot; re-apply before trusting results)

Required for zero baseline loss; without them baseline loss is 0.020.3% and masks cable faults.

  1. CPU governor → performance on all cores. Biggest single contributor: under powersave, cores idle at 400 MHz, ramp too slowly, and drop frames at startup.
  2. Disable adaptive coalescing: ethtool -C <dev> adaptive-rx off adaptive-tx off rx-usecs 25 tx-usecs 25. Syntax/values differ per driver — re-derive per NIC.
  3. Rings deep, raised toward the max via ethtool -G (RX 8160 / TX 4096 on the ice pair). Effect was not isolated in testing; don't claim it as a proven win.
  4. Socket buffers: no sysctl change needed — the tool forces SO_RCVBUFFORCE/SO_SNDBUFFORCE (needs root) past the 4 MB rmem_max clamp.

Verify a clean run shows tx frames == rx frames exactly before a run counts.

irdma (dev host only)

irdma autoloads and binds ice ports, making ETHTOOL_SCHANNELS fail EBUSY ("Cannot change channels when RDMA is active") and failing the channels host check. sudo rmmod irdma (usage count 0; returns on reboot). The appliance kernel has no irdma. ice also refuses channel changes while ntuple rules exist — cabletest clears its own first.

Running

Through ./harness: sudo go run ./harness -for 8s -at 4s -- -a <portA> -b <portB>.

  • The harness starts go run . in its own process group and kills the whole group on exit, confirming with kill(-pgid,0)==ESRCH. Do not wrap in timeout (kills go run but orphans the compiled binary to PID 1); do not follow runs with pgrep orphan checks — the group kill is the guarantee.
  • It dumps the panel to shots/<offset>.png for remote assessment.
  • The tell for two competing instances (an orphan, or the user's own hand-run): RX pps > TX pps from cross-received frames on shared ethertypes/MACs. Ask rather than killing something that might be theirs.
  • Default config only — no -sizes/-streams overrides, no editing the hardcoded frameSizes/numStreams/batchSize (same violation). The default is the config that must work and the only one comparable across runs; small-frame runs measure the host, not the cable. Ask before running a specific non-default case.
  • The first ~5 s of any run is a settling transient (flow rules, rings, workers coming up) — not residual error.