Files
cabletest/docs/hardware.md
T

3.2 KiB
Raw Blame History

Host rules

Box contents, card placement, and interface assignments live in state.md. This file carries the rules that stay true regardless of what's plugged in.

Interfaces

  • Test pair: the two ports whose modules hold the cable under test. Find by driver name, never by ethN — names shift with kernel link order.
  • Noise pair: driven bad on purpose, link up/down cycled, intertwined with the test cable to inject alien crosstalk.
  • LAN uplink: carries the default route. Never repurpose or down it; identify by default route, not name.
  • Test-path ports live in NetworkManager's unmanaged list (/etc/NetworkManager/conf.d/99-unmanaged-10g.conf), up with no IPv4.

Module identities, EEPROM honesty (and why the X520 needs allow_unsupported_sfp=1), and the physical-layer model live in modules/.

Host tuning (resets every reboot; re-apply before trusting results)

Required for zero baseline loss; without them baseline loss is 0.020.3% and masks cable faults.

  1. CPU governor → performance on all cores. Biggest single contributor: under powersave, cores idle at 400 MHz, ramp too slowly, and drop frames at startup.
  2. Disable adaptive coalescing: ethtool -C <dev> adaptive-rx off adaptive-tx off rx-usecs 25 tx-usecs 25. Syntax/values differ per driver — re-derive per NIC.
  3. Rings deep, raised toward the max via ethtool -G (RX 8160 / TX 4096 on the ice pair). Effect was not isolated in testing; don't claim it as a proven win.
  4. Socket buffers: no sysctl change needed — the tool forces SO_RCVBUFFORCE/SO_SNDBUFFORCE (needs root) past the 4 MB rmem_max clamp.

Verify a clean run shows tx frames == rx frames exactly before a run counts.

irdma (dev host only)

irdma autoloads and binds ice ports, making ETHTOOL_SCHANNELS fail EBUSY ("Cannot change channels when RDMA is active") and failing the channels host check. sudo rmmod irdma (usage count 0; returns on reboot). The appliance kernel has no irdma. ice also refuses channel changes while ntuple rules exist — cabletest clears its own first.

Running

Through ./harness: sudo go run ./harness -for 8s -at 4s -- -a <portA> -b <portB>.

  • The harness starts go run . in its own process group and kills the whole group on exit, confirming with kill(-pgid,0)==ESRCH. Do not wrap in timeout (kills go run but orphans the compiled binary to PID 1); do not follow runs with pgrep orphan checks — the group kill is the guarantee.
  • It dumps the panel to shots/<offset>.png for remote assessment.
  • The tell for two competing instances (an orphan, or the user's own hand-run): RX pps > TX pps from cross-received frames on shared ethertypes/MACs. Ask rather than killing something that might be theirs.
  • Default config only — no -sizes/-streams overrides, no editing the hardcoded frameSizes/numStreams/batchSize (same violation). The default is the config that must work and the only one comparable across runs; small-frame runs measure the host, not the cable. Ask before running a specific non-default case.
  • The first ~5 s of any run is a settling transient (flow rules, rings, workers coming up) — not residual error.