Drop goals and open-questions docs, cut PTP from the appliance kernel
This commit is contained in:
flamingcow
+3
-3
@@ -2,14 +2,14 @@
|
||||
|
||||
Project knowledge that moves and versions with the repo.
|
||||
|
||||
cabletest finds Ethernet cables that misbehave under sustained full-duplex 10 Gbit load. It exchanges raw Ethernet frames (no IP, no ARP) across a variety of sizes and payloads between two directly-cabled ports on one host, and attributes every problem it can to the cable rather than the host. It runs as PID 1 on a dedicated appliance with a framebuffer UI; development happens on the same machine under Arch. The target of measurement is the cable, not throughput — line rate is a means to stress the physical layer.
|
||||
|
||||
General:
|
||||
|
||||
- [goals.md](goals.md) — what cabletest is for and what it must output.
|
||||
- [state.md](state.md) — where things are now: the box and what's in it, out-of-repo artifacts, every-boot steps. **All current-placement state lives here and only here.**
|
||||
- [state.md](state.md) — where things are now: the box and what's in it, the bench driver, every-boot steps. **All current-placement state lives here and only here.**
|
||||
- [hardware.md](hardware.md) — host rules: interface conventions, tuning, how to run.
|
||||
- [measurement.md](measurement.md) — how to measure correctly; performance results and dead ends.
|
||||
- [xdp.md](xdp.md) — the AF_XDP datapath experiment: what it proved, what rebuilding it needs, why it can't run on the X520.
|
||||
- [open-questions.md](open-questions.md) — the genuinely open problems, as pointers into the docs that hold their detail.
|
||||
|
||||
Per-device (each directory: device doc + its datasheets):
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
# Goals
|
||||
|
||||
cabletest finds Ethernet cables that misbehave under sustained full-duplex 10 Gbit load. It exchanges raw Ethernet frames (no IP, no ARP) across a variety of sizes and payloads between two directly-cabled ports on one host, and attributes every problem it can to the cable rather than the host. It runs as PID 1 on a dedicated appliance with a framebuffer UI; development happens on the same machine under Arch.
|
||||
|
||||
The target of measurement is the cable, not throughput. Line rate is a means to stress the physical layer.
|
||||
|
||||
## Outputs, in priority order
|
||||
|
||||
1. **Loss and error attribution** — reception gaps, link errors, NIC/driver counters as first-class output alongside application loss. Baseline loss must be exactly zero before a run counts; any host-side loss masks real cable faults.
|
||||
2. **Per-pair SNR** from the module PHYs — the leading indicator of a marginal cable before it drops frames. IEEE 802.3an standard registers on the Marvells; the BCM leaves those unpopulated and reports through its vendor command handler instead (modules/fs/). Shown as margin over the operating point, classified good/marginal/bad, so nobody needs the magic numbers to read it (`phy.go`).
|
||||
3. **Cable length** — sanity check and fault localization. The BCM ECD is the product path: per-pair lengths in meters, healthy pairs included, proven meter-accurate on the bench (modules/fs/); run at bringup and on every reset, with pair-swap and fault verdicts on the panel.
|
||||
4. **Pre-FEC error visibility** — corrected-error counters that move before post-FEC loss appears. Vendor-specific; located on the Aquantia (modules/fibergaga/), unlocated elsewhere. Standard latched PCS counters (errored blocks, BER, block-lock loss) are the working proxy.
|
||||
|
||||
Alien-crosstalk noise injection was an output here and is measured out (measurement.md, "Dead ends"): an intertwined noise cable moves the victim's SNR margin only 0.2–0.3 dB.
|
||||
|
||||
## Design preferences that shaped the tool
|
||||
|
||||
- Plain sockets first, escalate only after measuring: AF_PACKET reached line rate on the shipping mix; the AF_XDP experiment is parked.
|
||||
- Pre-populated frame contents.
|
||||
- Goroutine-heavy is fine — many cores to burn.
|
||||
- DPDK is off the table: it would kill the sysfs counters and link management the tester depends on.
|
||||
+1
-1
@@ -29,5 +29,5 @@ Through `./harness`: `sudo go run ./harness -for 8s -at 4s -- -a <portA> -b <por
|
||||
- The harness starts `go run .` in its own process group and kills the whole group on exit, confirming with `kill(-pgid,0)==ESRCH`. Do not wrap in `timeout` (kills `go run` but orphans the compiled binary to PID 1); do not follow runs with pgrep orphan checks — the group kill is the guarantee.
|
||||
- It dumps the panel to `shots/<offset>.png` for remote assessment.
|
||||
- The tell for two competing instances (an orphan, or the user's own hand-run): `RX pps > TX pps` from cross-received frames on shared ethertypes/MACs. Ask rather than killing something that might be theirs.
|
||||
- **Default config only** — no `-sizes`/`-streams` overrides, no editing the hardcoded `frameSizes`/`numStreams`/`batchSize` (same violation). The default is the config that must work and the only one comparable across runs; small-frame runs measure the host, not the cable. Ask before running a specific non-default case.
|
||||
- **Default config only** — never edit the hardcoded `frameSizes`/`numStreams`/`batchSize` for a run. The only flags are `-a`/`-b`; the compiled-in config is the one that must work and the only one comparable across runs; small-frame runs measure the host, not the cable. Ask before running a specific non-default case.
|
||||
- The first ~5 s of any run is a settling transient (flow rules, rings, workers coming up) — not residual error.
|
||||
|
||||
@@ -13,7 +13,7 @@ Both original Wiiteks were bricked **permanently** by register exploration — t
|
||||
- Single targeted reads only, on registers with a documented, verified-safe reason.
|
||||
- Recovery needs a vendor firmware image + the boot-download protocol (1.0xC050 boot status, 1.0xD0F0–D0F3 download) — we have no CUX3610 image, so bricks are effectively dead. A ghost I2C device at 7-bit 0x57 (returns zeros) is the ROM bootloader waiting for a download.
|
||||
|
||||
The `rollball_ixgbe.py` client hard-guards the blacklist and raises *before* touching hardware — it structurally cannot repeat the kill.
|
||||
cabletest cannot repeat the kill because it issues only the safe set below — no sweep, no vendor window, nothing outside a documented register. Any client that goes further needs the blacklist enforced before the transaction reaches the wire, not after: the mailbox writes the address bytes and the µC dies executing the access, so there is no failure to catch afterwards.
|
||||
|
||||
## Safe register set — proven live
|
||||
|
||||
@@ -57,7 +57,7 @@ This PHY offers no length capability the product needs (the length strategy live
|
||||
Routes that do not risk the module:
|
||||
|
||||
1. **The real VCT register map** from Marvell/Wiitek under NDA — the only route to a *confirmed*-safe sequence.
|
||||
2. **Deliberate sacrifice** of a spare to test the QCA-CDT-start hypothesis, `rb_forensic.py` running — only if the capability is judged worth a module.
|
||||
2. **Deliberate sacrifice** of a spare to test the QCA-CDT-start hypothesis — only if the capability is judged worth a module.
|
||||
|
||||
The Marvell 88E151x datasheet PDF in this directory is family reference material.
|
||||
|
||||
|
||||
@@ -57,12 +57,12 @@ The recipe (one SMI transaction = one MCIA SET, plus one MCIA GET for reads):
|
||||
- SMI write of `devad.reg=val`: MCIA SET with `device_address=devad, size=4, dword[0]=reg<<16|val`.
|
||||
- Live example: `mstreg -d 01:00.1 --reg_name MCIA --set "dword[0]=0x00020000" --indexes "module=1,i2c_device_address=0x56,page_number=0,device_address=0x21,size=2" --yes` then the GET → 0x3590.
|
||||
|
||||
## How cabletest reaches MCIA in production (`phy_mcia.go`)
|
||||
## How cabletest reaches MCIA
|
||||
|
||||
There are two host paths to MCIA, and the split is decisive:
|
||||
|
||||
- **`/dev/fwctl` (the kernel's sanctioned firmware back-channel) is read-only for MCIA.** ACCESS_REG reads work at scope `DEBUG_READ_ONLY`; **writes are silently dropped** — the RPC returns success at every layer (`errno=0`, command status 0, MCIA status 0) yet the register never changes, at every scope including `DEBUG_WRITE_FULL`. The firmware honours the ACCESS_REG opcode under fwctl's restricted user context (uid 2, caps `0x04000000`) but no-ops the actual I2C write. Proven live with full pre/post op logging. Since a RollBall register **read** requires **writes** (set page/devad/command), fwctl cannot run RollBall at all.
|
||||
- **The mstflint PCICONF path (VSC vendor capability → ICMD → FLASH_REG_ACCESS) can write**, so cabletest uses it. Per-dword gateway ops through the PCI config-space vendor capability reach ICMD space; a register-access command wrapped in operation+reg TLVs carries the MCIA transaction. Reimplemented in Go (`phy_mcia.go`), no mstflint dependency at runtime.
|
||||
- **The mstflint PCICONF path (VSC vendor capability → ICMD → FLASH_REG_ACCESS) can write**, so cabletest uses it. Per-dword gateway ops through the PCI config-space vendor capability reach ICMD space; a register-access command wrapped in operation+reg TLVs carries the MCIA transaction. Reimplemented in Go, no mstflint dependency at runtime.
|
||||
|
||||
Two things that are not optional and one that is not needed:
|
||||
|
||||
|
||||
@@ -1,19 +0,0 @@
|
||||
# Open questions
|
||||
|
||||
The genuinely open, thinking-worthy problems — nothing here is resolved. Analyses and settled facts live in the device docs; each entry points at the doc that holds its detail. Read `state.md` first for placement context.
|
||||
|
||||
## 1. ECD run disturbance
|
||||
|
||||
The BCM ECD run blips the link (PMA 1.1 latch-low catches a drop even with the break-link bit clear). Until the disturbance is characterized, length measurement is a between-runs operation, not a during-run one ([modules/fs/](modules/fs/README.md)).
|
||||
|
||||
## 2. Smear window sizing
|
||||
|
||||
`smearWindow` (4 buckets = 64 ms) is a chosen default, not a measured one. Measure the host read-stall distribution under load on the X520 datapath, then size the window comfortably past the worst stall — it is also the display latency, so no larger than needed ([nics/x520/](nics/x520/README.md)).
|
||||
|
||||
## 3. Pre-FEC correlation run
|
||||
|
||||
The register question is answered (post-FEC vs corrected-by-iteration histogram located); the graded-noise correlation run that verifies the expected ordering has not happened. Design and instrument: [modules/fibergaga/](modules/fibergaga/README.md).
|
||||
|
||||
## 4. Wiitek VCT — pursue or leave dead?
|
||||
|
||||
No confirmed-safe path exists (every candidate lands in the µC danger window). The open decision is whether the capability is worth the NDA route or a sacrificial unit — the product doesn't need it for length ([modules/wiitek/](modules/wiitek/README.md), [modules/README.md](modules/README.md)).
|
||||
+3
-5
@@ -2,11 +2,9 @@
|
||||
|
||||
## Where the code is right now
|
||||
|
||||
**The committed tree is the X520 product implementation**: AF_PACKET datapath with read-time rate buckets and the backward excess-fill smear (nics/x520/ — rate is a headline, never a verdict; `smearWindow` 4 buckets = 64 ms display latency, resize after measuring the stall distribution: open-questions), the mixed FS+Wiitek diagnostics with the BCM ECD length path, `testDriver` = ixgbe. No hardware-timestamp machinery anywhere. The CX-5 is retired (nics/connectx-5/, "Status") and the E810 bit-bang investigation with it (nics/e810/).
|
||||
**The committed tree is the X520 product implementation**: AF_PACKET datapath with read-time rate buckets and the backward excess-fill smear (nics/x520/ — rate is a headline, never a verdict; `smearWindow` 4 buckets = 64 ms display latency, resize after measuring the stall distribution), the mixed FS+Wiitek diagnostics with the BCM ECD length path, `testDriver` = ixgbe. No hardware-timestamp machinery anywhere. The CX-5 is retired (nics/connectx-5/, "Status") and the E810 bit-bang investigation with it (nics/e810/).
|
||||
|
||||
## Stashes
|
||||
|
||||
None. Every parked experiment is distilled into docs and its code dropped: the AF_XDP conversion in xdp.md, the CX-5 Wiitek-pair implementation in nics/connectx-5/ (including the MCIA gateway constants), the phydiag campaign across nics/e810/, nics/x710/, modules/fs/ and modules/fibergaga/.
|
||||
Every parked experiment is distilled into docs and its code dropped — no stashes, nothing outside the repo: the AF_XDP conversion in xdp.md, the CX-5 Wiitek-pair implementation in nics/connectx-5/ (including the MCIA gateway constants), the phydiag campaign across nics/e810/, nics/x710/, modules/fs/ and modules/fibergaga/.
|
||||
|
||||
## The box
|
||||
|
||||
@@ -35,7 +33,7 @@ None. Every parked experiment is distilled into docs and its code dropped: the A
|
||||
|
||||
## Bench driver
|
||||
|
||||
- **Patched ixgbe, bench build** (`kernel/ixgbe-sff/`, full driver source tracked; build artifacts gitignored) — out-of-tree module for the dev host: the product patch plus the bench-only extras (`reg_ops` value readback, `ts_bench`); `./load-ixgbe` rebuilds + swaps the module with `allow_unsupported_sfp=1`. The product subset (raw I2C + `sff_i2c`, ETQF steering, FCoE slot guard) is versioned at `kernel/ixgbe-sff.patch` and applied by `./mkimage` to every appliance kernel build — apply fails loudly when stable moves under it; port the patch then. Details: nics/x520/. Nothing else lives outside the repo; the retired diagnostics campaigns are distilled in these docs and their code is deleted.
|
||||
- **Patched ixgbe, bench build** (`kernel/ixgbe-sff/`, full driver source tracked; build artifacts gitignored) — out-of-tree module for the dev host: the product patch plus the bench-only extras (`reg_ops` value readback, `ts_bench`); `./load-ixgbe` rebuilds + swaps the module with `allow_unsupported_sfp=1`. The product subset (raw I2C + `sff_i2c`, ETQF steering, FCoE slot guard) is versioned at `kernel/ixgbe-sff.patch` and applied by `./mkimage` to every appliance kernel build — apply fails loudly when stable moves under it; port the patch then. Details: nics/x520/.
|
||||
|
||||
## Every boot
|
||||
|
||||
|
||||
@@ -38,13 +38,13 @@ cmdline="drm.edid_firmware=HDMI-A-1:$edid video=HDMI-A-1:e ixgbe.allow_unsupport
|
||||
wanty=(
|
||||
EFI_STUB BLK_DEV_INITRD INITRAMFS_COMPRESSION_NONE CMDLINE_BOOL
|
||||
PACKET INET DEVTMPFS PROC_FS SYSFS TMPFS
|
||||
IXGBE DEBUG_FS PTP_1588_CLOCK
|
||||
IXGBE DEBUG_FS
|
||||
DRM_I915 DRM_FBDEV_EMULATION DRM_LOAD_EDID_FIRMWARE DRM_PANIC
|
||||
FB_DEVICE FRAMEBUFFER_CONSOLE VT_CONSOLE
|
||||
USB_XHCI_PCI USB_HID HID_MULTITOUCH INPUT_EVDEV
|
||||
CPU_FREQ_GOV_PERFORMANCE X86_INTEL_PSTATE
|
||||
)
|
||||
wantn=(MODULES BLOCK IGC ICE I40E)
|
||||
wantn=(MODULES BLOCK IGC ICE I40E PTP_1588_CLOCK)
|
||||
|
||||
say() { printf '\n\033[36m== %s\033[0m\n' "$*"; }
|
||||
die() { printf '\033[31merror: %s\033[0m\n' "$*" >&2; exit 1; }
|
||||
@@ -105,6 +105,7 @@ cp "$repo/kernel/config" "$src/.config"
|
||||
--disable MODULES \
|
||||
--enable EXPERT --disable BLOCK \
|
||||
--enable IXGBE --disable ICE --disable I40E --disable IGC \
|
||||
--disable PTP_1588_CLOCK \
|
||||
--enable DRM_LOAD_EDID_FIRMWARE \
|
||||
--set-str EXTRA_FIRMWARE "$edid ${fw[*]}" \
|
||||
--set-str EXTRA_FIRMWARE_DIR "$fwdir" \
|
||||
|
||||
Reference in New Issue
Block a user