Noise ripped out after the bench verdict (tightly-wrapped unshielded cables move victim SNR only 0.2-0.3 dB, recorded in measurement.md dead ends): noise.go deleted with the noiser, phase clock and panel lane; configureNoise/checkPrivFlag gone so the X710 pair is untouched; console noise column, NOISE links rows and lane colors removed; driverPair rehomed to system.go, carrierUp to phy.go; history anchors to run start (frame-rate sampling and draw-clock roll kept — they fix display latency for every lane); goals/hardware/state/nics docs updated; verified on hardware (20.00G, zero errors, ECD 41m, SNR +7.6)
This commit is contained in:
+5
-4
@@ -7,10 +7,11 @@ The target of measurement is the cable, not throughput. Line rate is a means to
|
||||
## Outputs, in priority order
|
||||
|
||||
1. **Loss and error attribution** — reception gaps, link errors, NIC/driver counters as first-class output alongside application loss. Baseline loss must be exactly zero before a run counts; any host-side loss masks real cable faults.
|
||||
2. **Noise tolerance** — a deliberately-bad "noise" cable intertwined with the test cable, driven by link up/down cycling, stresses the cable under test with alien crosstalk.
|
||||
3. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (modules/fs/). Shown as margin over the operating point, classified good/marginal/bad, so nobody needs the magic numbers to read it (`phy.go`).
|
||||
4. **Cable length** — sanity check and fault localization. The BCM ECD is the product path: per-pair lengths in meters, healthy pairs included, proven meter-accurate on the bench (modules/fs/); run at bringup and on every reset, with pair-swap and fault verdicts on the panel.
|
||||
5. **Pre-FEC error visibility** — corrected-error counters that move before post-FEC loss appears. Vendor-specific; located on the Aquantia (modules/fibergaga/), unlocated elsewhere. Standard latched PCS counters (errored blocks, BER, block-lock loss) are the working proxy under noise stress.
|
||||
2. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (modules/fs/). Shown as margin over the operating point, classified good/marginal/bad, so nobody needs the magic numbers to read it (`phy.go`).
|
||||
3. **Cable length** — sanity check and fault localization. The BCM ECD is the product path: per-pair lengths in meters, healthy pairs included, proven meter-accurate on the bench (modules/fs/); run at bringup and on every reset, with pair-swap and fault verdicts on the panel.
|
||||
4. **Pre-FEC error visibility** — corrected-error counters that move before post-FEC loss appears. Vendor-specific; located on the Aquantia (modules/fibergaga/), unlocated elsewhere. Standard latched PCS counters (errored blocks, BER, block-lock loss) are the working proxy.
|
||||
|
||||
Alien-crosstalk noise injection was an output here and is measured out (measurement.md, "Dead ends"): an intertwined noise cable moves the victim's SNR margin only 0.2–0.3 dB.
|
||||
|
||||
## Design preferences that shaped the tool
|
||||
|
||||
|
||||
@@ -5,7 +5,6 @@ Box contents, card placement, and interface assignments live in [state.md](state
|
||||
## Interfaces
|
||||
|
||||
- **Test pair**: the two ports whose modules hold the cable under test. Find by driver name, never by ethN — names shift with kernel link order.
|
||||
- **Noise pair**: driven bad on purpose, link up/down cycled, intertwined with the test cable to inject alien crosstalk.
|
||||
- **LAN uplink**: carries the default route. **Never repurpose or down it**; identify by default route, not name.
|
||||
- Test-path ports live in NetworkManager's unmanaged list (`/etc/NetworkManager/conf.d/99-unmanaged-10g.conf`), up with no IPv4.
|
||||
|
||||
|
||||
@@ -53,6 +53,7 @@ Measured in the E810 era; the binding constraint was total CPU across ~28 gorout
|
||||
- **CPU pinning** — the Go scheduler beats manual placement; E-cores are poor at TX.
|
||||
- **Batch sizes above 64** — no gain, worse loss.
|
||||
- **Chasing the TX `sendmmsg` busy-spin on ENOBUFS** — it only exists when the TX ring is full, which means the wire is the ceiling; recovering that CPU buys no packets. POLLOUT is inert under PACKET_QDISC_BYPASS (skb freed on ENOBUFS, socket always reports writable).
|
||||
- **Alien-crosstalk injection from an intertwined noise cable** — bench-measured: two unshielded cables tightly wrapped moved the victim's per-pair SNR margin only 0.2–0.3 dB, far too little to stress even a marginal cable; 10GBASE-T's alien-noise budget absorbs it. The noise pair, its link-cycling, and the panel lane are removed from the tool.
|
||||
|
||||
## rxnfc flow-rule programming — three traps (found on ice; 1 and 2 are kernel-ABI-generic)
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
|
||||
| Per-pair SNR | `CMD_GET_SNR` 0x8030, **invoked bare** (writing the documented DATA1 display flag returns zeros; IEEE 1.133–1.140 never populate — constant 0x8080) | DATA2–5 = SNR A–D ×0.1 dB; ≈ 27–32 dB absolute on bench, 0.1–0.4 dB jitter. 10GBASE-T operating point ≈ 26.5 dB, so margin ≈ value − 26.5 — a derived number that never proved actionable, so **cabletest does not poll it** (the panel's SNR comes from an IEEE-margin end — modules/wiitek/ — and each GET costs handler traffic on this µC). **Each training re-converges differently**: back-to-back retrains with identical roles, temperature and noise moved one pair by 3.6 dB (roles pinned, launch power pinned — the convergence itself is the variable, and the two bench modules differ in stability: ≤1.5 dB spread on one, 3.6 dB on the other). A single training's absolute SNR is a weak qualification number; qualification wants median-of-N trainings |
|
||||
| Die temperature | `CMD_GET_CURRENT_TEMP` 0x8031 | ~68–70 °C on bench |
|
||||
| Supply rails | `GET_CURRENT_VOLTAGE` 0x802F | 0.8 V and 1.88 V rails, tenths of mV |
|
||||
| Error counters | IEEE PCS 3.32/3.33 — block lock, latched errored-block/BER, clear-on-read | The noise-stress error proxy |
|
||||
| Error counters | IEEE PCS 3.32/3.33 — block lock, latched errored-block/BER, clear-on-read | The corrected-error channel cabletest polls |
|
||||
| Retrain on demand | IEEE AN restart `7.0 \|= 0x1200` (enable + restart — the ECD can leave the enable bit cleared, see below) | Relinks in ~5–6 s |
|
||||
|
||||
## Probed configuration states
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ Which card is where: [../state.md](../state.md).
|
||||
|---|---|---|---|
|
||||
| [X520 / 82599 (ixgbe)](x520/README.md) | Host bit-bangs — full control, **proven**, fast (<25 ms RollBall) | No (single serial latch; 1588-event frames only) — no longer required | **The product NIC** under the smoothed-bucket rate plan: only card with unmediated host I2C, and the FS links here (mixed FS+Wiitek pair proven) |
|
||||
| [ConnectX-4/5 (mlx5)](connectx-5/README.md) | Firmware MCIA — writes via VSC/ICMD only (fwctl read-only), every mailbox read quantized to ~150 ms steps by firmware module traffic | Yes (CQE-stamps everything) | Retired: its stamps are no longer needed and its I2C mediation is the cost; the FS never links on it either (host-interface deadlock, device page) |
|
||||
| [X710 (i40e)](x710/README.md) | 1-byte-offset only | No | RollBall oracle path; noise pair |
|
||||
| [X710 (i40e)](x710/README.md) | 1-byte-offset only | No | RollBall oracle path; in the box unused |
|
||||
| [E810 (ice)](e810/README.md) | Multi-byte framing, writes EPERM-blocked | Yes | Dead for diagnostics writes; the below-the-AQ bit-bang idea (device page) is retired with the timestamp requirement |
|
||||
|
||||
## Architecture
|
||||
|
||||
+2
-2
@@ -22,7 +22,7 @@ Indices shift as stashes are pushed/popped — match by message, not number.
|
||||
| Interface | Device | Role (rules: hardware.md) |
|
||||
|---|---|---|
|
||||
| `enp1s0f0` + second port | X520 (ixgbe) | Test pair — FS + Wiitek, dispatched by EEPROM PN so either port may hold either module. **Stock ixgbe refuses the FS's honest EEPROM and its port has no netdev until `./load-ixgbe`** (allow_unsupported_sfp=1 + the sff_i2c/ETQF patches); the tool needs exactly 2 ixgbe interfaces |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Noise pair (has been `enp4s0f*` across reboots) — two Wiiteks, both qualified by the X710. The FS cannot live here (firmware rejects the honest EEPROM, no `allow_unsupported_sfp` in i40e) |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Unused by the tool (was the noise pair; alien crosstalk measured ineffective — measurement.md). Two Wiiteks still seated (has been `enp4s0f*` across reboots). The FS cannot live here (firmware rejects the honest EEPROM, no `allow_unsupported_sfp` in i40e) |
|
||||
| `enp88s0` | igc | LAN uplink, default route; sibling `enp89s0` is dark |
|
||||
|
||||
## Hardware
|
||||
@@ -32,7 +32,7 @@ Indices shift as stashes are pushed/popped — match by message, not number.
|
||||
| X520-DA2 | **Installed** in the single PCIe slot — **the product NIC** (nics/x520/) | Unmediated bit-bang I2C (patched-ixgbe `sff_i2c`), <25 ms RollBall, FS links and the ECD length path is proven; the smoothed-bucket rate design removes the need for hardware RX stamps |
|
||||
| ConnectX-5 | Out of the box — retired | Ran the two-Wiitek pair at near line rate with dual-end SNR (stashed `phy_mcia.go` MCIA transport), but the firmware quantizes every module mailbox read to ~150 ms steps and the FS never links behind mlx5 — nics/connectx-5/, "Status" |
|
||||
| FS SFP-10G-T-100 ×2 | One in the X520 test pair, one on the shelf | BCM84891L, documented, robust; ixgbe-only (unlinkable behind mlx5 — host-interface deadlock, confirmed module-typed, nics/connectx-5/ — and rejected by the X710). The FS+Wiitek mixed pair (FS length via ECD + Wiitek SNR) is the product configuration |
|
||||
| Replacement Wiiteks | One in the X520 test pair, two in the X710 noise pair, rest on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first. IEEE-standard registers only (modules/README.md safe set); VCT templates are single-shot candidates on a sacrificial unit only |
|
||||
| Replacement Wiiteks | One in the X520 test pair, two seated in the unused X710, rest on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first. IEEE-standard registers only (modules/README.md safe set); VCT templates are single-shot candidates on a sacrificial unit only |
|
||||
| Fibergaga SFP-10G-T-30M | On the shelf | Aquantia, RollBall, the documented oracle; the alternate length path if the pair mix ever changes |
|
||||
| 10Gtek | In hand | Claims SFP-10G-SR, still copper RJ45; filler, not in the test set |
|
||||
| E810 | Out of the box | Patched ice + `sff_i2c` remains useful only if it returns for read-side work |
|
||||
|
||||
Reference in New Issue
Block a user