Noise ripped out after the bench verdict (tightly-wrapped unshielded cables move victim SNR only 0.2-0.3 dB, recorded in measurement.md dead ends): noise.go deleted with the noiser, phase clock and panel lane; configureNoise/checkPrivFlag gone so the X710 pair is untouched; console noise column, NOISE links rows and lane colors removed; driverPair rehomed to system.go, carrierUp to phy.go; history anchors to run start (frame-rate sampling and draw-clock roll kept — they fix display latency for every lane); goals/hardware/state/nics docs updated; verified on hardware (20.00G, zero errors, ECD 41m, SNR +7.6)
This commit is contained in:
+5
-4
@@ -7,10 +7,11 @@ The target of measurement is the cable, not throughput. Line rate is a means to
|
||||
## Outputs, in priority order
|
||||
|
||||
1. **Loss and error attribution** — reception gaps, link errors, NIC/driver counters as first-class output alongside application loss. Baseline loss must be exactly zero before a run counts; any host-side loss masks real cable faults.
|
||||
2. **Noise tolerance** — a deliberately-bad "noise" cable intertwined with the test cable, driven by link up/down cycling, stresses the cable under test with alien crosstalk.
|
||||
3. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (modules/fs/). Shown as margin over the operating point, classified good/marginal/bad, so nobody needs the magic numbers to read it (`phy.go`).
|
||||
4. **Cable length** — sanity check and fault localization. The BCM ECD is the product path: per-pair lengths in meters, healthy pairs included, proven meter-accurate on the bench (modules/fs/); run at bringup and on every reset, with pair-swap and fault verdicts on the panel.
|
||||
5. **Pre-FEC error visibility** — corrected-error counters that move before post-FEC loss appears. Vendor-specific; located on the Aquantia (modules/fibergaga/), unlocated elsewhere. Standard latched PCS counters (errored blocks, BER, block-lock loss) are the working proxy under noise stress.
|
||||
2. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (modules/fs/). Shown as margin over the operating point, classified good/marginal/bad, so nobody needs the magic numbers to read it (`phy.go`).
|
||||
3. **Cable length** — sanity check and fault localization. The BCM ECD is the product path: per-pair lengths in meters, healthy pairs included, proven meter-accurate on the bench (modules/fs/); run at bringup and on every reset, with pair-swap and fault verdicts on the panel.
|
||||
4. **Pre-FEC error visibility** — corrected-error counters that move before post-FEC loss appears. Vendor-specific; located on the Aquantia (modules/fibergaga/), unlocated elsewhere. Standard latched PCS counters (errored blocks, BER, block-lock loss) are the working proxy.
|
||||
|
||||
Alien-crosstalk noise injection was an output here and is measured out (measurement.md, "Dead ends"): an intertwined noise cable moves the victim's SNR margin only 0.2–0.3 dB.
|
||||
|
||||
## Design preferences that shaped the tool
|
||||
|
||||
|
||||
@@ -5,7 +5,6 @@ Box contents, card placement, and interface assignments live in [state.md](state
|
||||
## Interfaces
|
||||
|
||||
- **Test pair**: the two ports whose modules hold the cable under test. Find by driver name, never by ethN — names shift with kernel link order.
|
||||
- **Noise pair**: driven bad on purpose, link up/down cycled, intertwined with the test cable to inject alien crosstalk.
|
||||
- **LAN uplink**: carries the default route. **Never repurpose or down it**; identify by default route, not name.
|
||||
- Test-path ports live in NetworkManager's unmanaged list (`/etc/NetworkManager/conf.d/99-unmanaged-10g.conf`), up with no IPv4.
|
||||
|
||||
|
||||
@@ -53,6 +53,7 @@ Measured in the E810 era; the binding constraint was total CPU across ~28 gorout
|
||||
- **CPU pinning** — the Go scheduler beats manual placement; E-cores are poor at TX.
|
||||
- **Batch sizes above 64** — no gain, worse loss.
|
||||
- **Chasing the TX `sendmmsg` busy-spin on ENOBUFS** — it only exists when the TX ring is full, which means the wire is the ceiling; recovering that CPU buys no packets. POLLOUT is inert under PACKET_QDISC_BYPASS (skb freed on ENOBUFS, socket always reports writable).
|
||||
- **Alien-crosstalk injection from an intertwined noise cable** — bench-measured: two unshielded cables tightly wrapped moved the victim's per-pair SNR margin only 0.2–0.3 dB, far too little to stress even a marginal cable; 10GBASE-T's alien-noise budget absorbs it. The noise pair, its link-cycling, and the panel lane are removed from the tool.
|
||||
|
||||
## rxnfc flow-rule programming — three traps (found on ice; 1 and 2 are kernel-ABI-generic)
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
|
||||
| Per-pair SNR | `CMD_GET_SNR` 0x8030, **invoked bare** (writing the documented DATA1 display flag returns zeros; IEEE 1.133–1.140 never populate — constant 0x8080) | DATA2–5 = SNR A–D ×0.1 dB; ≈ 27–32 dB absolute on bench, 0.1–0.4 dB jitter. 10GBASE-T operating point ≈ 26.5 dB, so margin ≈ value − 26.5 — a derived number that never proved actionable, so **cabletest does not poll it** (the panel's SNR comes from an IEEE-margin end — modules/wiitek/ — and each GET costs handler traffic on this µC). **Each training re-converges differently**: back-to-back retrains with identical roles, temperature and noise moved one pair by 3.6 dB (roles pinned, launch power pinned — the convergence itself is the variable, and the two bench modules differ in stability: ≤1.5 dB spread on one, 3.6 dB on the other). A single training's absolute SNR is a weak qualification number; qualification wants median-of-N trainings |
|
||||
| Die temperature | `CMD_GET_CURRENT_TEMP` 0x8031 | ~68–70 °C on bench |
|
||||
| Supply rails | `GET_CURRENT_VOLTAGE` 0x802F | 0.8 V and 1.88 V rails, tenths of mV |
|
||||
| Error counters | IEEE PCS 3.32/3.33 — block lock, latched errored-block/BER, clear-on-read | The noise-stress error proxy |
|
||||
| Error counters | IEEE PCS 3.32/3.33 — block lock, latched errored-block/BER, clear-on-read | The corrected-error channel cabletest polls |
|
||||
| Retrain on demand | IEEE AN restart `7.0 \|= 0x1200` (enable + restart — the ECD can leave the enable bit cleared, see below) | Relinks in ~5–6 s |
|
||||
|
||||
## Probed configuration states
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ Which card is where: [../state.md](../state.md).
|
||||
|---|---|---|---|
|
||||
| [X520 / 82599 (ixgbe)](x520/README.md) | Host bit-bangs — full control, **proven**, fast (<25 ms RollBall) | No (single serial latch; 1588-event frames only) — no longer required | **The product NIC** under the smoothed-bucket rate plan: only card with unmediated host I2C, and the FS links here (mixed FS+Wiitek pair proven) |
|
||||
| [ConnectX-4/5 (mlx5)](connectx-5/README.md) | Firmware MCIA — writes via VSC/ICMD only (fwctl read-only), every mailbox read quantized to ~150 ms steps by firmware module traffic | Yes (CQE-stamps everything) | Retired: its stamps are no longer needed and its I2C mediation is the cost; the FS never links on it either (host-interface deadlock, device page) |
|
||||
| [X710 (i40e)](x710/README.md) | 1-byte-offset only | No | RollBall oracle path; noise pair |
|
||||
| [X710 (i40e)](x710/README.md) | 1-byte-offset only | No | RollBall oracle path; in the box unused |
|
||||
| [E810 (ice)](e810/README.md) | Multi-byte framing, writes EPERM-blocked | Yes | Dead for diagnostics writes; the below-the-AQ bit-bang idea (device page) is retired with the timestamp requirement |
|
||||
|
||||
## Architecture
|
||||
|
||||
+2
-2
@@ -22,7 +22,7 @@ Indices shift as stashes are pushed/popped — match by message, not number.
|
||||
| Interface | Device | Role (rules: hardware.md) |
|
||||
|---|---|---|
|
||||
| `enp1s0f0` + second port | X520 (ixgbe) | Test pair — FS + Wiitek, dispatched by EEPROM PN so either port may hold either module. **Stock ixgbe refuses the FS's honest EEPROM and its port has no netdev until `./load-ixgbe`** (allow_unsupported_sfp=1 + the sff_i2c/ETQF patches); the tool needs exactly 2 ixgbe interfaces |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Noise pair (has been `enp4s0f*` across reboots) — two Wiiteks, both qualified by the X710. The FS cannot live here (firmware rejects the honest EEPROM, no `allow_unsupported_sfp` in i40e) |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Unused by the tool (was the noise pair; alien crosstalk measured ineffective — measurement.md). Two Wiiteks still seated (has been `enp4s0f*` across reboots). The FS cannot live here (firmware rejects the honest EEPROM, no `allow_unsupported_sfp` in i40e) |
|
||||
| `enp88s0` | igc | LAN uplink, default route; sibling `enp89s0` is dark |
|
||||
|
||||
## Hardware
|
||||
@@ -32,7 +32,7 @@ Indices shift as stashes are pushed/popped — match by message, not number.
|
||||
| X520-DA2 | **Installed** in the single PCIe slot — **the product NIC** (nics/x520/) | Unmediated bit-bang I2C (patched-ixgbe `sff_i2c`), <25 ms RollBall, FS links and the ECD length path is proven; the smoothed-bucket rate design removes the need for hardware RX stamps |
|
||||
| ConnectX-5 | Out of the box — retired | Ran the two-Wiitek pair at near line rate with dual-end SNR (stashed `phy_mcia.go` MCIA transport), but the firmware quantizes every module mailbox read to ~150 ms steps and the FS never links behind mlx5 — nics/connectx-5/, "Status" |
|
||||
| FS SFP-10G-T-100 ×2 | One in the X520 test pair, one on the shelf | BCM84891L, documented, robust; ixgbe-only (unlinkable behind mlx5 — host-interface deadlock, confirmed module-typed, nics/connectx-5/ — and rejected by the X710). The FS+Wiitek mixed pair (FS length via ECD + Wiitek SNR) is the product configuration |
|
||||
| Replacement Wiiteks | One in the X520 test pair, two in the X710 noise pair, rest on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first. IEEE-standard registers only (modules/README.md safe set); VCT templates are single-shot candidates on a sacrificial unit only |
|
||||
| Replacement Wiiteks | One in the X520 test pair, two seated in the unused X710, rest on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first. IEEE-standard registers only (modules/README.md safe set); VCT templates are single-shot candidates on a sacrificial unit only |
|
||||
| Fibergaga SFP-10G-T-30M | On the shelf | Aquantia, RollBall, the documented oracle; the alternate length path if the pair mix ever changes |
|
||||
| 10Gtek | In hand | Claims SFP-10G-SR, still copper RJ45; filler, not in the test set |
|
||||
| E810 | Out of the box | Patched ice + `sff_i2c` remains useful only if it returns for read-side work |
|
||||
|
||||
+2
-2
@@ -140,8 +140,8 @@ type histView struct {
|
||||
}
|
||||
|
||||
// The window's newest column is the slot containing now, taken from the draw
|
||||
// clock rather than the last sample: the grid then rolls on the slot boundary
|
||||
// itself — the same instant the noise cycle flips.
|
||||
// clock rather than the last sample, so the grid rolls on the slot boundary
|
||||
// itself.
|
||||
func (h *history) view(now time.Time) histView {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
|
||||
@@ -69,7 +69,6 @@ var intervalCols = []colSpec{
|
||||
{group: "NOW", title: "link", width: 7, right: true},
|
||||
{group: "NOW", title: "internal", width: 8, right: true},
|
||||
{group: "NOW", title: "corrected", width: 9, right: true},
|
||||
{group: "NOW", title: "noise", width: 7, right: true},
|
||||
{group: "OVERALL", title: "elapsed", width: 9, right: true},
|
||||
{group: "OVERALL", title: "packets", width: 9, right: true},
|
||||
{group: "OVERALL", title: "bytes", width: 9, right: true},
|
||||
@@ -82,9 +81,8 @@ var intervalCols = []colSpec{
|
||||
}
|
||||
|
||||
// The same figures the panel draws, in the same order: the last second as
|
||||
// rates and error flags with the noise cable riding at the end of them, then
|
||||
// everything since the reset.
|
||||
func totalRow(elapsed time.Duration, v view, target float64, phy phyDisplay, nv noiseView) []string {
|
||||
// rates and error flags, then everything since the reset.
|
||||
func totalRow(elapsed time.Duration, v view, target float64, phy phyDisplay) []string {
|
||||
return []string{
|
||||
rateCell(v.rxGbps*1e9, target*1e9),
|
||||
scaleSI(v.rxPPS),
|
||||
@@ -94,7 +92,6 @@ func totalRow(elapsed time.Duration, v view, target float64, phy phyDisplay, nv
|
||||
flagCell(v.window.link),
|
||||
flagCell(v.window.internal),
|
||||
correctedFlag(phy.recent),
|
||||
noiseCell(nv),
|
||||
scaleTime(elapsed),
|
||||
scaleCount(v.rxFrames),
|
||||
scaleCount(v.rxBytes),
|
||||
@@ -202,14 +199,7 @@ func run(aName, bName string) (err error) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
noise, err := newNoiser()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer noise.close()
|
||||
|
||||
a.tag, b.tag = "TEST A", "TEST B"
|
||||
noise.eps[0].tag, noise.eps[1].tag = "NOISE A", "NOISE B"
|
||||
ifnames := []string{a.name, b.name}
|
||||
|
||||
ethertypes := make([]uint16, numStreams)
|
||||
@@ -224,7 +214,6 @@ func run(aName, bName string) (err error) {
|
||||
|
||||
checks := configureSystem(ifnames, ethertypes)
|
||||
checks = append(checks, moduleChecks(modules, [2]string{a.name, b.name})...)
|
||||
checks = append(checks, configureNoise(noise.names())...)
|
||||
if err := reportChecks("SETTINGS", checks); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -261,13 +250,9 @@ func run(aName, bName string) (err error) {
|
||||
}()
|
||||
|
||||
var linkRows [][]string
|
||||
for i, e := range []endpoint{a, b, noise.eps[0], noise.eps[1]} {
|
||||
mod := ""
|
||||
if i < len(moduleIDs) {
|
||||
mod = moduleIDs[i]
|
||||
}
|
||||
for i, e := range []endpoint{a, b} {
|
||||
linkRows = append(linkRows, []string{
|
||||
paint(e.tag, cCyan), e.name, e.macString(), fmt.Sprintf("%d", e.mtu), mod,
|
||||
paint(e.tag, cCyan), e.name, e.macString(), fmt.Sprintf("%d", e.mtu), moduleIDs[i],
|
||||
})
|
||||
}
|
||||
fmt.Println(renderBox("LINKS",
|
||||
@@ -295,14 +280,6 @@ func run(aName, bName string) (err error) {
|
||||
defer holdPanic()
|
||||
samp.run(&done, startTx)
|
||||
}()
|
||||
// Not gated on startTx: the cycle and the connected verdict are wanted the
|
||||
// moment the panel is, and nothing it does touches the measurement.
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
defer holdPanic()
|
||||
noise.run(&done)
|
||||
}()
|
||||
for _, m := range modules {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
@@ -359,9 +336,7 @@ func run(aName, bName string) (err error) {
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
// Anchored to the same clock as the noise cycle, so slot boundaries and
|
||||
// phase transitions coincide.
|
||||
hist := newHistory(rateEpochStart)
|
||||
hist := newHistory(start)
|
||||
// A reset resets at the press, then re-measures; the measure's failures
|
||||
// are suppressed at their sources while it runs, so there is nothing to
|
||||
// hide or revert afterwards.
|
||||
@@ -405,7 +380,7 @@ func run(aName, bName string) (err error) {
|
||||
phy, measuring := phyView(diag, modules)
|
||||
hist.sample(now, v, phy, measuring, target)
|
||||
if err := disp.render(v, now.Sub(start), phy, measuring,
|
||||
noise.view(), hist.view(now), target); err != nil {
|
||||
hist.view(now), target); err != nil {
|
||||
return err
|
||||
}
|
||||
case now := <-tick.C:
|
||||
@@ -420,8 +395,7 @@ func run(aName, bName string) (err error) {
|
||||
fmt.Println(stats.rule(n))
|
||||
}
|
||||
}
|
||||
for _, line := range stats.emit(totalRow(elapsed, v, target, phy,
|
||||
noise.view())) {
|
||||
for _, line := range stats.emit(totalRow(elapsed, v, target, phy)) {
|
||||
fmt.Println(line)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,227 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
"unsafe"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// The noise cable: a deliberately bad cable twisted around the one under test,
|
||||
// there to radiate into it. Its ports are driven for the interference they
|
||||
// produce, not measured: nothing is ever received from them, and nothing they
|
||||
// count reaches the error columns.
|
||||
//
|
||||
// The wire cannot be quieted by going idle, because without EEE the PHYs
|
||||
// signal at full power whether or not frames flow, and these PHYs live inside
|
||||
// the SFP+ modules where no EEE control reaches them. Closing the port is the
|
||||
// one switch the host actually has, so the cycle is built on it: links up and
|
||||
// carrying frames for a spell, then administratively down for one. Every wake
|
||||
// re-runs 10GBASE-T training, which is as loud as this wire ever gets.
|
||||
const (
|
||||
noiseDriver = "i40e"
|
||||
noiseFrameLen = 1514
|
||||
noiseFrameGap = 10 * time.Millisecond
|
||||
|
||||
noiseEther uint16 = etherBase + numStreams
|
||||
)
|
||||
|
||||
// Kernel names shift with which drivers are built in, since ethN is handed out
|
||||
// in link order rather than by slot. The driver name is the one label a port
|
||||
// keeps across kernel configs, so pairs are found by it rather than named.
|
||||
func driverPair(driver string) (string, string, error) {
|
||||
ents, err := os.ReadDir("/sys/class/net")
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
var names []string
|
||||
for _, e := range ents {
|
||||
link, err := os.Readlink("/sys/class/net/" + e.Name() + "/device/driver")
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if filepath.Base(link) == driver {
|
||||
names = append(names, e.Name())
|
||||
}
|
||||
}
|
||||
if len(names) != 2 {
|
||||
return "", "", fmt.Errorf("want 2 %s interfaces, found %d [%s]",
|
||||
driver, len(names), strings.Join(names, " "))
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names[0], names[1], nil
|
||||
}
|
||||
|
||||
type noisePort struct {
|
||||
name string
|
||||
fd int
|
||||
frame []byte
|
||||
}
|
||||
|
||||
// The cycle is a pure function of the shared slot clock: even slots up — ports
|
||||
// admin-up and the wire loud, training or frames — odd slots down. Everything
|
||||
// that shows the phase computes it from the clock through here rather than
|
||||
// sampling the actuator, whose ioctls lag the boundary.
|
||||
func slotUp(seq int64) bool { return seq%2 == 0 }
|
||||
|
||||
func noiseUp(now time.Time) bool {
|
||||
return slotUp(int64(now.Sub(rateEpochStart) / histSlotSpan))
|
||||
}
|
||||
|
||||
type noiser struct {
|
||||
eps [2]endpoint
|
||||
ports [2]noisePort
|
||||
|
||||
// Whether the cable is judged present: both carriers seen during an up
|
||||
// phase. Latched across the down phase, where the missing carrier is our
|
||||
// own doing and says nothing about the cable.
|
||||
connected atomic.Bool
|
||||
}
|
||||
|
||||
func newNoiser() (*noiser, error) {
|
||||
aName, bName, err := driverPair(noiseDriver)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("noise: %w", err)
|
||||
}
|
||||
a, err := lookupEndpoint(aName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("noise: %w", err)
|
||||
}
|
||||
b, err := lookupEndpoint(bName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("noise: %w", err)
|
||||
}
|
||||
|
||||
n := &noiser{eps: [2]endpoint{a, b}}
|
||||
// The grid-locked cycle can start mid-down-slot, where absence would be
|
||||
// our own doing: presence is granted until the first up phase delivers a
|
||||
// verdict.
|
||||
n.connected.Store(true)
|
||||
for i, p := range [][2]endpoint{{a, b}, {b, a}} {
|
||||
fd, err := openTxSocket(p[0].idx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("noise tx socket %s: %w", p[0].name, err)
|
||||
}
|
||||
// The payload is left zero: the PCS scrambles everything on the wire,
|
||||
// so no pattern radiates differently from any other. The frame exists
|
||||
// to occupy the link, not to say anything.
|
||||
frame := make([]byte, noiseFrameLen)
|
||||
copy(frame[0:6], p[1].mac[:])
|
||||
copy(frame[6:12], p[0].mac[:])
|
||||
binary.BigEndian.PutUint16(frame[12:14], noiseEther)
|
||||
n.ports[i] = noisePort{name: p[0].name, fd: fd, frame: frame}
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func (n *noiser) names() []string {
|
||||
return []string{n.eps[0].name, n.eps[1].name}
|
||||
}
|
||||
|
||||
type noiseView struct {
|
||||
missing uint64
|
||||
on bool
|
||||
}
|
||||
|
||||
// missing is zero while the cable was there at the last verdict, one while it
|
||||
// was not: the shape the error cells already colour by, so absence paints as
|
||||
// the fault it is. on is the commanded phase, straight from the clock.
|
||||
func (n *noiser) view() noiseView {
|
||||
v := noiseView{on: noiseUp(time.Now())}
|
||||
if !n.connected.Load() {
|
||||
v.missing = 1
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// The ports were reachable when the noiser was built, so one that stops taking
|
||||
// the ioctl now is the interface going away underneath us, the same fault the
|
||||
// counter reads stop for.
|
||||
func (n *noiser) setLinks(fd int, up bool) {
|
||||
for i := range n.ports {
|
||||
var ifr flagsIfreq
|
||||
copy(ifr.name[:], n.ports[i].name)
|
||||
if _, _, errno := unix.Syscall(unix.SYS_IOCTL, uintptr(fd),
|
||||
uintptr(unix.SIOCGIFFLAGS), uintptr(unsafe.Pointer(&ifr))); errno != 0 {
|
||||
panic(fmt.Sprintf("reading %s flags: %v", n.ports[i].name, errno))
|
||||
}
|
||||
if up {
|
||||
ifr.flags |= unix.IFF_UP
|
||||
} else {
|
||||
ifr.flags &^= unix.IFF_UP
|
||||
}
|
||||
if _, _, errno := unix.Syscall(unix.SYS_IOCTL, uintptr(fd),
|
||||
uintptr(unix.SIOCSIFFLAGS), uintptr(unsafe.Pointer(&ifr))); errno != 0 {
|
||||
panic(fmt.Sprintf("setting %s flags: %v", n.ports[i].name, errno))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Down reads as EINVAL rather than zero, and either way the answer is the
|
||||
// same: no carrier here now.
|
||||
func carrierUp(name string) bool {
|
||||
v, ok := readUint("/sys/class/net/" + name + "/carrier")
|
||||
return ok && v == 1
|
||||
}
|
||||
|
||||
func (n *noiser) bothUp() bool {
|
||||
return carrierUp(n.ports[0].name) && carrierUp(n.ports[1].name)
|
||||
}
|
||||
|
||||
// Send results are deliberately dropped: the cable is bad on purpose, the link
|
||||
// comes and goes under the cycle, and a frame this side declined to send is as
|
||||
// good as one the wire mangled. What matters is only ever what the test cable
|
||||
// counted.
|
||||
//
|
||||
// The cycle is locked to the panel's history grid — up on even 5 s slots,
|
||||
// down on odd, transitions on the shared slot boundaries — so a history cell
|
||||
// never straddles a phase and flickers between the two answers.
|
||||
func (n *noiser) run(done *atomic.Bool) {
|
||||
fd, err := unix.Socket(unix.AF_INET, unix.SOCK_DGRAM, 0)
|
||||
if err != nil {
|
||||
panic(fmt.Sprintf("noise ioctl socket: %v", err))
|
||||
}
|
||||
defer unix.Close(fd)
|
||||
|
||||
tick := time.NewTicker(noiseFrameGap)
|
||||
defer tick.Stop()
|
||||
|
||||
applied, wasUp, linked := false, false, false
|
||||
for !done.Load() {
|
||||
<-tick.C
|
||||
up := noiseUp(time.Now())
|
||||
if !applied || up != wasUp {
|
||||
// A whole up phase with no link is many times the ~1s the wire
|
||||
// needs to train, so at its end the silence is the cable's answer.
|
||||
if applied && wasUp {
|
||||
n.connected.Store(linked)
|
||||
}
|
||||
n.setLinks(fd, up)
|
||||
applied, wasUp, linked = true, up, false
|
||||
}
|
||||
if !up || !n.bothUp() {
|
||||
continue
|
||||
}
|
||||
linked = true
|
||||
n.connected.Store(true)
|
||||
for i := range n.ports {
|
||||
unix.Write(n.ports[i].fd, n.ports[i].frame)
|
||||
}
|
||||
}
|
||||
// Left up rather than wherever the cycle stopped, so a run never strands
|
||||
// the ports down for whoever looks next.
|
||||
n.setLinks(fd, true)
|
||||
}
|
||||
|
||||
func (n *noiser) close() {
|
||||
for i := range n.ports {
|
||||
unix.Close(n.ports[i].fd)
|
||||
}
|
||||
}
|
||||
@@ -506,6 +506,13 @@ func openModules(names [2]string) ([]*phyModule, [2]string, error) {
|
||||
return mods, idents, nil
|
||||
}
|
||||
|
||||
// Down reads as EINVAL rather than zero, and either way the answer is the
|
||||
// same: no carrier here now.
|
||||
func carrierUp(name string) bool {
|
||||
v, ok := readUint("/sys/class/net/" + name + "/carrier")
|
||||
return ok && v == 1
|
||||
}
|
||||
|
||||
func waitCarrier(names [2]string, done *atomic.Bool) {
|
||||
deadline := time.Now().Add(linkWaitSpan)
|
||||
for !(carrierUp(names[0]) && carrierUp(names[1])) {
|
||||
|
||||
@@ -304,18 +304,6 @@ func correctedFlag(v uint64) string {
|
||||
return paint("warn", cYellow)
|
||||
}
|
||||
|
||||
// Green either way while the cable is present — the cycle's phase is state,
|
||||
// not health. Red stays what it was: the cable missing.
|
||||
func noiseCell(nv noiseView) string {
|
||||
switch {
|
||||
case nv.missing > 0:
|
||||
return paint("ERR", cRed)
|
||||
case nv.on:
|
||||
return paint("on", cGreen)
|
||||
}
|
||||
return paint("off", cGreen)
|
||||
}
|
||||
|
||||
// Per-interval rates jitter by a couple of percent at line rate, so green has
|
||||
// to cover that. Yellow means a real shortfall, red means badly off.
|
||||
const (
|
||||
|
||||
@@ -6,12 +6,39 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"unsafe"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// Kernel names shift with which drivers are built in, since ethN is handed out
|
||||
// in link order rather than by slot. The driver name is the one label a port
|
||||
// keeps across kernel configs, so pairs are found by it rather than named.
|
||||
func driverPair(driver string) (string, string, error) {
|
||||
ents, err := os.ReadDir("/sys/class/net")
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
var names []string
|
||||
for _, e := range ents {
|
||||
link, err := os.Readlink("/sys/class/net/" + e.Name() + "/device/driver")
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if filepath.Base(link) == driver {
|
||||
names = append(names, e.Name())
|
||||
}
|
||||
}
|
||||
if len(names) != 2 {
|
||||
return "", "", fmt.Errorf("want 2 %s interfaces, found %d [%s]",
|
||||
driver, len(names), strings.Join(names, " "))
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names[0], names[1], nil
|
||||
}
|
||||
|
||||
const (
|
||||
ethtoolGRXRINGS = 0x2d
|
||||
ethtoolGRXCLSRLCNT = 0x2e
|
||||
@@ -367,7 +394,6 @@ func getCoalesce(fd int, ifname string) (ethtoolCoalesce, error) {
|
||||
|
||||
const (
|
||||
ethSSStats = 1
|
||||
ethSSPrivFlags = 2
|
||||
ethGstringLen = 32
|
||||
)
|
||||
|
||||
@@ -660,67 +686,6 @@ func withIoctlSocket(fn func(fd int) []checkResult) []checkResult {
|
||||
return fn(fd)
|
||||
}
|
||||
|
||||
type ethtoolValue struct {
|
||||
cmd uint32
|
||||
data uint32
|
||||
}
|
||||
|
||||
// i40e leaves the module transmitting when a port is closed, so the peer never
|
||||
// sees anything happen and the copper stays trained. This flag is what makes
|
||||
// an administrative down reach the wire, and the noise cycle is switched
|
||||
// entirely through it.
|
||||
func checkPrivFlag(fd int, ifname, flag string) checkResult {
|
||||
res := checkResult{item: ifname + " " + flag}
|
||||
names, err := stringSetNames(fd, ifname, ethSSPrivFlags)
|
||||
if err != nil {
|
||||
res.err = err
|
||||
return res
|
||||
}
|
||||
bit := -1
|
||||
for i, s := range names {
|
||||
if s == flag {
|
||||
bit = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if bit < 0 {
|
||||
res.err = fmt.Errorf("driver has no private flag %q", flag)
|
||||
return res
|
||||
}
|
||||
v := ethtoolValue{cmd: unix.ETHTOOL_GPFLAGS}
|
||||
if err := ethtoolCall(fd, ifname, unsafe.Pointer(&v)); err != nil {
|
||||
res.err = err
|
||||
return res
|
||||
}
|
||||
if v.data&(1<<bit) != 0 {
|
||||
res.state = "on"
|
||||
return res
|
||||
}
|
||||
v.cmd = unix.ETHTOOL_SPFLAGS
|
||||
v.data |= 1 << bit
|
||||
if err := ethtoolCall(fd, ifname, unsafe.Pointer(&v)); err != nil {
|
||||
res.err = err
|
||||
res.state = "could not set"
|
||||
return res
|
||||
}
|
||||
res.fixed = true
|
||||
res.state = "was off, now on"
|
||||
return res
|
||||
}
|
||||
|
||||
// The noise ports get none of the test pair's tuning: nothing is measured on
|
||||
// them, so rings, coalescing and steering are all beside the point. All they
|
||||
// need is for a closed port to really drop the link.
|
||||
func configureNoise(ifnames []string) []checkResult {
|
||||
return withIoctlSocket(func(fd int) []checkResult {
|
||||
var out []checkResult
|
||||
for _, ifname := range ifnames {
|
||||
out = append(out, checkPrivFlag(fd, ifname, "link-down-on-close"))
|
||||
}
|
||||
return out
|
||||
})
|
||||
}
|
||||
|
||||
// Nothing here waits for a carrier: the two ports are the two ends of the cable
|
||||
// under test, so with no cable there is never going to be one.
|
||||
func configureSystem(ifnames []string, ethertypes []uint16) []checkResult {
|
||||
|
||||
@@ -19,8 +19,6 @@ var (
|
||||
uiCrit = rgb{0xd0, 0x3b, 0x3b}
|
||||
uiCellOK = rgb{0x0a, 0x5d, 0x0a}
|
||||
uiCellNA = rgb{0x3a, 0x3a, 0x37}
|
||||
uiNoise = rgb{0x1c, 0x5c, 0xab}
|
||||
uiNoise0 = rgb{0x24, 0x24, 0x23}
|
||||
uiAccent = rgb{0x39, 0x87, 0xe5}
|
||||
)
|
||||
|
||||
@@ -58,10 +56,9 @@ const (
|
||||
)
|
||||
|
||||
// The panel is the heat matrix: one row per check, columns for now, the last
|
||||
// 90 seconds at 5 s a cell, and since reset. Noise rides at the bottom as
|
||||
// context — state, not a verdict.
|
||||
// 90 seconds at 5 s a cell, and since reset.
|
||||
var panelRows = []string{
|
||||
"line rate", "lost", "corrupt", "link", "internal", "corrected", "SNR dB", "noise",
|
||||
"line rate", "lost", "corrupt", "link", "internal", "corrected", "SNR dB",
|
||||
}
|
||||
|
||||
type rect struct {
|
||||
@@ -418,7 +415,7 @@ func okOr(bad bool, c rgb) rgb {
|
||||
return uiCellOK
|
||||
}
|
||||
|
||||
func (d *display) drawRows(v view, phy phyDisplay, measuring bool, nv noiseView,
|
||||
func (d *display) drawRows(v view, phy phyDisplay, measuring bool,
|
||||
hv histView, target float64) {
|
||||
nowR := d.nowX + d.nowW
|
||||
resetR := d.resetX + d.resetW
|
||||
@@ -473,33 +470,6 @@ func (d *display) drawRows(v view, phy phyDisplay, measuring bool, nv noiseView,
|
||||
} else {
|
||||
d.colText(d.text, resetR, y, "-", uiMuted)
|
||||
}
|
||||
case "noise":
|
||||
state := "off"
|
||||
if nv.on {
|
||||
state = "on"
|
||||
}
|
||||
if nv.missing > 0 {
|
||||
d.colMark(nowR, y, markCross, uiCrit)
|
||||
} else {
|
||||
d.colText(d.text, nowR, y, state, uiInk2)
|
||||
}
|
||||
// The lane shows the commanded phase, a pure function of each
|
||||
// slot's parity; only slots the run never lived through are unknown.
|
||||
for k, s := range hv.slots {
|
||||
c := uiCellNA
|
||||
if s.sampled {
|
||||
c = uiNoise0
|
||||
if slotUp(s.seq) {
|
||||
c = uiNoise
|
||||
}
|
||||
}
|
||||
d.heatCell(y, k, c)
|
||||
}
|
||||
if nv.missing > 0 {
|
||||
d.colMark(resetR, y, markCross, uiCrit)
|
||||
} else {
|
||||
d.colMark(resetR, y, markCheck, uiGood)
|
||||
}
|
||||
default:
|
||||
f := -1
|
||||
for j, c := range faultClasses {
|
||||
@@ -553,14 +523,14 @@ func (d *display) drawFooter(v view, elapsed time.Duration) {
|
||||
}
|
||||
|
||||
func (d *display) render(v view, elapsed time.Duration, phy phyDisplay,
|
||||
measuring bool, nv noiseView, hv histView, target float64) error {
|
||||
measuring bool, hv histView, target float64) error {
|
||||
fb := d.fb
|
||||
fb.fill(uiBg)
|
||||
|
||||
d.drawHeader(v, phy, hv, measuring, time.Now())
|
||||
d.hairline(d.headerH - 1)
|
||||
d.drawColHeaders()
|
||||
d.drawRows(v, phy, measuring, nv, hv, target)
|
||||
d.drawRows(v, phy, measuring, hv, target)
|
||||
d.drawAxis()
|
||||
d.drawFooter(v, elapsed)
|
||||
d.drawResetButton()
|
||||
|
||||
Reference in New Issue
Block a user