Power: 2 streams and a 2.0 GHz clock pinned by a new checkMaxFreq SETTINGS check (performance governor + scaling_max_freq = fixed clock) take line rate from 80 W to 29.7 W package, qualified 96 s zero-loss; measured on the bench: streams buy scheduling headroom not watts (C1-parked threads, lonelier spinners boost harder), relaxed cpu_dma_latency never reaches deep idle (scheduler spreads wakeups, ~2% residency), offlining E-cores saves 1.7 W and is wire-invisible mid-run, the cliff is TX starvation below 1.6 GHz (1.2 GHz sags to ~18 G, zero errors)

docs/hardware.md

@@ -12,12 +12,13 @@ Module identities, EEPROM honesty (and why the X520 needs `allow_unsupported_sfp
 
 ## Host tuning (resets every reboot; re-apply before trusting results)
 
-Required for zero baseline loss; without them baseline loss is 0.02–0.3% and masks cable faults.
+Required for zero baseline loss (without them baseline loss is 0.02–0.3% and masks cable faults) and for the power budget.
 
 1. **CPU governor → `performance`** on all cores. Biggest single contributor: under powersave, cores idle at 400 MHz, ramp too slowly, and drop frames at startup.
-2. **Disable adaptive coalescing**: `ethtool -C <dev> adaptive-rx off adaptive-tx off rx-usecs 25 tx-usecs 25`. Syntax/values differ per driver — re-derive per NIC.
+2. **CPU max frequency → 2.0 GHz** (`scaling_max_freq`, all cores). Under the performance governor this is a pinned clock, not a ceiling, and it is the power budget: ~30 W package at line rate against ~80 W unbounded (measurement.md, "Power"). The value is host-silicon-bound — 1.6 GHz still holds line rate with thinner margin, 1.2 GHz cannot fill the wire — so re-derive it on new host silicon and re-qualify zero-loss.
-3. **Rings deep**, raised toward the max via `ethtool -G` (the tool asks RX 8160 / TX 4096; ixgbe takes it as-is). Effect was not isolated in testing; don't claim it as a proven win.
+3. **Disable adaptive coalescing**: `ethtool -C <dev> adaptive-rx off adaptive-tx off rx-usecs 25 tx-usecs 25`. Syntax/values differ per driver — re-derive per NIC.
-4. **Socket buffers**: no sysctl change needed — the tool forces `SO_RCVBUFFORCE`/`SO_SNDBUFFORCE` (needs root) past the 4 MB `rmem_max` clamp.
+4. **Rings deep**, raised toward the max via `ethtool -G` (the tool asks RX 8160 / TX 4096; ixgbe takes it as-is). Effect was not isolated in testing; don't claim it as a proven win.
+5. **Socket buffers**: no sysctl change needed — the tool forces `SO_RCVBUFFORCE`/`SO_SNDBUFFORCE` (needs root) past the 4 MB `rmem_max` clamp.
 
 Verify a clean run shows `tx frames == rx frames` exactly before a run counts.
 

docs/measurement.md

@@ -33,7 +33,7 @@ This rule covers the **NIC-counter** buckets and the per-stream receive buckets
 
 ## Measured performance
 
-The committed X520 path at the default mix: steady 20.00–20.04 Gb/s (line rate both directions, ~1.78 Mpps/dir), zero lost/corrupt/link/internal over a 90 s run, including startup.
+The committed X520 path at the default config (2 streams, 2.0 GHz pinned clock): steady 20.00–20.04 Gb/s (line rate both directions, ~0.75 Mpps/dir at the jumbo mix), zero lost/corrupt/link/internal over a 96 s run, including startup, at ~30 W CPU package on the bench host. The pre-jumbo mix ran ~1.78 Mpps/dir at the same rate.
 
 Host-side limits measured on the earlier E810 (ice) datapath — CPU facts, not NIC-specific:
 
@@ -45,6 +45,25 @@ Host-side limits measured on the earlier E810 (ice) datapath — CPU facts, not
 - RX cannot be parallelized by RSS — hardware RSS can't hash raw ethertypes. Flow-steering by ethertype to distinct queues is what gives multiple NAPI contexts (ETQF on the X520, Flow Director on the E810).
 - The AF_XDP experiment delivered per-packet MAC RX stamps via an XDP-metadata kfunc (E810 datapath only; the committed path uses no hardware stamps at all). With the test path off the E810 it is parked; its novelty was delivery mechanism and throughput, not the use of hardware stamps.
 
+## Power: the busy cores' voltage is the cost, not the idle cores
+
+Measured on the bench host (i9-13900H, 6P+8E, 20 threads), RAPL package power at line rate both directions:
+
+| Config | Package | Wire |
+|---|---|---|
+| 7 streams, unbounded clock | 80.0 W | clean |
+| 2 streams, unbounded clock (4.2–4.6 GHz) | 70.7 W | clean |
+| 2 streams @ 2.4 GHz | 35.0 W | clean |
+| **2 streams @ 2.0 GHz (shipped)** | **29.7 W** | clean, 43% busy |
+| 2 streams @ 1.6 GHz | 25.1 W | clean, 49% busy |
+| 2 streams @ 1.2 GHz | 20.4 W | **fails — wire sags to 17.5–19.8 G, zero errors: TX starvation, the honest failure mode** |
+| 7 streams @ 1.6 GHz | 29.8 W | line rate but 94% busy and the rate reading wobbles |
+| idle, no tool | 10.9 W | — |
+
+- The frequency pin is the lever: core power 56 → 16 W is V² savings on the busy cores. ~13.5 W of uncore (PCIe/memory fabric) is a floor no CPU knob touches.
+- Stream count buys scheduling headroom, not watts: 7→1 streams unbounded saves only ~13 W, because freed threads park in C1 (the 10 µs cpu_dma_latency cap) while the surviving TX spinners boost harder. At a fixed clock the cut is what makes the low clock safe — 49% busy at 2 streams where 7 needs 94%.
+- Every TX worker is a full-time spinner (the ENOBUFS spin is the flow control; there is no blocking send under PACKET_QDISC_BYPASS), so TX cost scales with stream count regardless of traffic; RX blocks in recvmmsg and scales with traffic.
+
 ## Dead ends — measured, do not re-attempt without new hardware
 
 Measured in the E810 era; the binding constraint was total CPU across ~28 goroutines on 20 threads.
@@ -53,6 +72,8 @@ Measured in the E810 era; the binding constraint was total CPU across ~28 gorout
 - **CPU pinning** — the Go scheduler beats manual placement; E-cores are poor at TX.
 - **Batch sizes above 64** — no gain, worse loss.
 - **Chasing the TX `sendmmsg` busy-spin on ENOBUFS** — it only exists when the TX ring is full, which means the wire is the ceiling; recovering that CPU buys no packets. POLLOUT is inert under PACKET_QDISC_BYPASS (skb freed on ENOBUFS, socket always reports writable).
+- **Letting idle cores sleep by relaxing cpu_dma_latency** (200 µs admits the 127 µs C2_ACPI) — steady-state deep-idle residency ~2% and power unchanged: the Go scheduler spreads wakeups across every thread, so no core idles long enough for the idle governor to pick a deep state.
+- **Offlining cores for power** (all 8 E-cores) — 1.7 W. The offline event mid-run is invisible on the wire (no blip at line rate), so parking cores is safe if ever wanted for another reason; it is not a power lever. Note the SETTINGS governor check fails loudly with offline cpus present — it reads every `cpu*/cpufreq/scaling_governor` and an offline cpu returns EBUSY.
 - **Alien-crosstalk injection from an intertwined noise cable** — bench-measured: two unshielded cables tightly wrapped moved the victim's per-pair SNR margin only 0.2–0.3 dB, far too little to stress even a marginal cable; 10GBASE-T's alien-noise budget absorbs it. The noise pair, its link-cycling, and the panel lane are removed from the tool.
 
 ## rxnfc flow-rule programming — three traps (found on ice; 1 and 2 are kernel-ABI-generic)

main.go

@@ -105,7 +105,7 @@ func totalRow(elapsed time.Duration, v view, target float64, phy phyDisplay) []s
 }
 
 const (
-	numStreams = 7
+	numStreams = 2
 	batchSize  = 64
 
 	testDriver = "ixgbe"

system.go

@@ -312,6 +312,7 @@ type ethtoolCoalesce struct {
 
 const (
 	wantGovernor      = "performance"
+	wantMaxFreqKHz    = "2000000"
 	wantCoalesceUsecs = 25
 	wantRxRing        = 8160
 	wantTxRing        = 4096
@@ -562,6 +563,50 @@ func checkGovernor(want string) checkResult {
 	return res
 }
 
+// Under the performance governor a max-frequency bound is a fixed clock, not a
+// ceiling the cores rove beneath, so the line-rate margin it was qualified with
+// is the margin every run gets.
+func checkMaxFreq(wantKHz string) checkResult {
+	res := checkResult{item: "cpu max frequency"}
+	paths, err := filepath.Glob("/sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq")
+	if err != nil || len(paths) == 0 {
+		res.err = fmt.Errorf("no cpufreq max frequencies found")
+		return res
+	}
+	var wrong []string
+	counts := map[string]int{}
+	for _, p := range paths {
+		b, err := os.ReadFile(p)
+		if err != nil {
+			res.err = err
+			return res
+		}
+		got := strings.TrimSpace(string(b))
+		counts[got]++
+		if got != wantKHz {
+			wrong = append(wrong, p)
+		}
+	}
+	if len(wrong) == 0 {
+		res.state = fmt.Sprintf("%s kHz on all %d cpus", wantKHz, len(paths))
+		return res
+	}
+	var found []string
+	for k, v := range counts {
+		found = append(found, fmt.Sprintf("%s:%d", k, v))
+	}
+	for _, p := range wrong {
+		if err := os.WriteFile(p, []byte(wantKHz), 0o644); err != nil {
+			res.err = err
+			res.state = fmt.Sprintf("could not set %s", p)
+			return res
+		}
+	}
+	res.fixed = true
+	res.state = fmt.Sprintf("was %s, now %s kHz on all %d cpus", strings.Join(found, " "), wantKHz, len(paths))
+	return res
+}
+
 // Bounds how far into idle any cpu may sink, so a worker or interrupt never
 // waits behind a deep state's exit. Ten microseconds admits C1, whose exit is
 // noise under the interrupt coalescing already in the path, and blocks the
@@ -690,7 +735,8 @@ func withIoctlSocket(fn func(fd int) []checkResult) []checkResult {
 // under test, so with no cable there is never going to be one.
 func configureSystem(ifnames []string, ethertypes []uint16) []checkResult {
 	return withIoctlSocket(func(fd int) []checkResult {
-		out := []checkResult{checkGovernor(wantGovernor), checkIdleLatency(wantIdleLatency)}
+		out := []checkResult{checkGovernor(wantGovernor), checkMaxFreq(wantMaxFreqKHz),
+			checkIdleLatency(wantIdleLatency)}
 		for _, ifname := range ifnames {
 			out = append(out, checkLinkUp(fd, ifname))
 			out = append(out, checkMTU(fd, ifname, wantMTU))