34 lines
3.1 KiB
Markdown
34 lines
3.1 KiB
Markdown
# Host rules
|
||
|
||
Box contents, card placement, and interface assignments live in [state.md](state.md). This file carries the rules that stay true regardless of what's plugged in.
|
||
|
||
## Interfaces
|
||
|
||
- **Test pair**: the two ports whose modules hold the cable under test. Find by driver name, never by ethN — names shift with kernel link order.
|
||
- **LAN uplink**: carries the default route. **Never repurpose or down it**; identify by default route, not name.
|
||
- Test-path ports live in NetworkManager's unmanaged list (`/etc/NetworkManager/conf.d/99-unmanaged-10g.conf`), up with no IPv4.
|
||
|
||
Module identities, EEPROM honesty (and why the X520 needs `allow_unsupported_sfp=1`), and the physical-layer model live in [modules/](modules/README.md).
|
||
|
||
## Host tuning (resets every reboot; re-apply before trusting results)
|
||
|
||
Required for zero baseline loss (without them baseline loss is 0.02–0.3% and masks cable faults) and for the power budget.
|
||
|
||
1. **CPU governor → `performance`** on all cores. Biggest single contributor: under powersave, cores idle at 400 MHz, ramp too slowly, and drop frames at startup.
|
||
2. **CPU max frequency → 2.0 GHz** (`scaling_max_freq`, all cores). Under the performance governor this is a pinned clock, not a ceiling, and it is the power budget: ~30 W package at line rate against ~80 W unbounded (measurement.md, "Power"). The value is host-silicon-bound — 1.6 GHz still holds line rate with thinner margin, 1.2 GHz cannot fill the wire — so re-derive it on new host silicon and re-qualify zero-loss.
|
||
3. **Disable adaptive coalescing**: `ethtool -C <dev> adaptive-rx off adaptive-tx off rx-usecs 25 tx-usecs 25`. Syntax/values differ per driver — re-derive per NIC.
|
||
4. **Rings deep**, raised toward the max via `ethtool -G` (the tool asks RX 8160 / TX 4096; ixgbe takes it as-is). Effect was not isolated in testing; don't claim it as a proven win.
|
||
5. **Socket buffers**: no sysctl change needed — the tool forces `SO_RCVBUFFORCE`/`SO_SNDBUFFORCE` (needs root) past the 4 MB `rmem_max` clamp.
|
||
|
||
Verify a clean run shows `tx frames == rx frames` exactly before a run counts.
|
||
|
||
## Running
|
||
|
||
Through `./harness`: `sudo go run ./harness -for 8s -at 4s -- -a <portA> -b <portB>`.
|
||
|
||
- The harness starts `go run .` in its own process group and kills the whole group on exit, confirming with `kill(-pgid,0)==ESRCH`. Do not wrap in `timeout` (kills `go run` but orphans the compiled binary to PID 1); do not follow runs with pgrep orphan checks — the group kill is the guarantee.
|
||
- It dumps the panel to `shots/<offset>.png` for remote assessment.
|
||
- The tell for two competing instances (an orphan, or the user's own hand-run): `RX pps > TX pps` from cross-received frames on shared ethertypes/MACs. Ask rather than killing something that might be theirs.
|
||
- **Default config only** — never edit the hardcoded `frameSizes`/`numStreams`/`batchSize` for a run. The only flags are `-a`/`-b`; the compiled-in config is the one that must work and the only one comparable across runs; small-frame runs measure the host, not the cable. Ask before running a specific non-default case.
|
||
- The first ~5 s of any run is a settling transient (flow rules, rings, workers coming up) — not residual error.
|