cabletest/docs/modules.md

Module-PHY diagnostics and the register maps

DANGER: Marvell CUX3610 (Wiitek UF-RJ45-10G-100) modules brick on certain reads

Both original Wiiteks were bricked permanently by register exploration — the death survives cold power cycles and reseats. Read this before touching any replacement.

The CUX3610 has an integrated microcontroller (Marvell Alaska-M, PHY ID 0x002B0BF4, firmware 12.1.5.0) that boots from onboard non-volatile storage. The high vendor register windows (3.0x8000+, 31.0xF000+) reached over RollBall are µC-mapped peripheral space, not PHY-core MDIO. A read of a trapped peripheral crashes the µC mid-operation and corrupts its persistent boot state, so it comes up FATAL forever after.

The trap: any register whose low byte is 0x64 in a high window (register high-byte ≥ 0x80), on any devad. Module 1 died reading 3.0x8064, module 2 reading 31.0xF064. The mailbox writes the address bytes, then the µC dies executing the access — the command never even needs to complete. 3.0x8064 is a known QCA808x CDT (cable-diagnostic) status register; the ODM firmware appears to emulate/intercept a CDT block there and crash on a cold read. A sloppy (reg & 0xFF) == 0x64 decode explains both deaths with one bug.

Rules for any Marvell/RollBall module:

• Never sweep vendor register windows. Never read *.0x??64 (high-byte ≥ 0x80).
• Single targeted reads only, on registers with a documented reason.
• Recovery after a brick needs a vendor firmware image + the boot-download protocol (registers 1.0xC050 boot status, 1.0xD0F0–D0F3 download) — we have no CUX3610 image, so bricks are effectively dead. A ghost i2c device at 7-bit 0x57 (returns zeros) is the ROM bootloader waiting for a download.

What the Marvell modules give (safely)

Proven on the CUX3610 before they died, and on the Fibergaga oracle — all IEEE 802.3an standard registers, safe on any Marvell module:

• Per-pair SNR margin — PMA 1.133–1.136 current, 1.137–1.140 minimum.
  • On the CUX3610 the value is reg − 0x8000 in 0.1 dB (saw A 9.1 / B 7.1 / C 7.7 / D 8.9 dB).
  • Retrain-dependent — only compare within one training session.
• Latched error/retrain detection — PCS 3.32/3.33.
  • Block-lock loss, BER saturate, errored blocks; clear-on-read.
• Link-partner / master-slave — 7.33.
• FW version 1.0xC011/12; boot status 1.0xC050.
• Forcing a retrain (to refresh SNR minimums):
  • RollBall write 7.0 |= bit9 (restart AN) — proven safe.
  • An ip link bounce does not drop the copper line, so it won't retrain.

Not available in safe space: pre-FEC counter, cable length. Those live in the µC vendor space that killed the modules.

What the FS/BCM gives

All via the SMI bridge + command handler (transports.md carries the codes and the invocation traps). Proven on the X520:

• Per-pair SNR — CMD_GET_SNR 0x8030 invoked bare: DATA2–5 = absolute receiver SNR in 0.1 dB (bench ≈ 27–32 dB; the 10GBASE-T operating point is ≈ 26.5 dB, so margin ≈ value − 26.5). The IEEE registers 1.133–1.140 stay at 0x8080 — handler only.
• Die temperature (0x8031) and supply rails (0x802F) — module health.
• Standard PCS 3.32/3.33 — block lock, latched errored-block/BER counters, clear-on-read: the noise-stress error proxy.
• Retrain — IEEE AN restart 7.0 |= bit9, relinks in ~6 s.

Probed on the bench unit (full command transcription: bcm84891l-mdio-commands.md); all GETs PASS; observed states:

• EEE / AutogrEEEn — force off at bringup, and trust the wire over the handler. GET_EEE_MODE (0x8008) once returned DATA1 = 0x0047 — decoding as 10G AutogrEEEn variable latency plus 5G/1G native EEE, Broadcom's local-only power saving that never shows in the IEEE advertisement registers (7.60/7.61 read 0 throughout) — then read 0x0000 later with no SET in between: either live-then-cleared or an unreliable return (caveat below). Mitigation applied and verified: SET_EEE_MODE (0x8009) with explicit all-off params (0, 0, 0x7A12, 0x480, 0) + AN restart → mode reads 0, link and SNR healthy. cabletest should apply this defensively at startup. Wire-truth check: arm SET_EEE_STATISTICS (0x801A) after link-up, read GET_EEE_STATISTICS (0x801B) — zero LPI events/time observed on the idle link; repeat under traffic.
• Fast retrain: enabled on 10G/5G/2.5G (0x800A → 1/1/1); IEEE 1.147 = 0x0019, retrain count bits zero since link-up. Keep it enabled and treat the 1.147 count as a per-run quality metric — a marginal cable that fast-retrains instead of dropping still shows up in the count.
• Pair map: identity (GET_PAIR_SWAP 0x8000 → DATA2 = 0x00E4 = A/B/C/D straight through) — MDI wiring verification works.
• Faults pass through (GET_SUB_LF_RF_STATUS 0x800D → 0; SET/CLEAR at 0x8011/0x8010 could mask them as IDLE — leave off).
• Host interface clean: XFI polarity normal, TX FIR = main tap only (0x3C), KR/SyncE/USXGMII/WOL all disabled, MAC/PHY frequency locked, jumbo enabled at 9K, temp warnings off.
• 1588 disabled (0x8004 → 0); limited-reach mode disabled (0x8006 → 0), so its DATA3 linked-in-LR bit — the only handler-visible trace of the DSP length estimate — is unavailable without a config change.
• Pause-frame mode (0x801F/0x8020) is scoped to 2.5/5G idle-stuffing under 10G XFI WAP mode — inert in plain 10G operation.

Firmware-reliability caveat. Several documented DATA1 returns on this ODM firmware are untrustworthy: die-temperature-like values (0x43/0x44/0x46/0x47) appear in DATA1 of commands that should return modes, and repeat reads of the same GET disagree. Corroborate anything load-bearing through IEEE registers (7.60/7.61 for EEE advertisement) or wire behavior (EEE statistics under traffic), and write every DATA register explicitly before any SET (the handler never clears them).

No cable length. The handler catalog is complete (§1.25.1.1–45: pair swap through WOL) and contains no ECD, length, or skew command. Cable length, opens/shorts, pair skew, and polarity live in the separate ECD register mechanism whose chapter FS hasn't sent. The 1588 engine is the same story: the PHY is IEEE 1588-2008 v2-compliant with a one-command enable (0x8004/0x8005) but its operation registers are undocumented — in-PHY timestamping would measure path delay at the MDI, taking PHY-pipeline latency out of the length equation entirely (open-questions.md §2). Until either chapter lands, FS-side length comes only from the NIC timestamp path — in contrast to the Aquantia oracle, whose 1E.C884 length register is fully documented.

Marvell cable-length register candidates (unverified, single-shot only)

From kernel drivers for sibling parts — templates for a future single targeted probe on a sacrificial module, never a sweep, never near *.0x??64:

• TDR from the 88Q2220 (marvell-88q2xxx.c, same 0x002b0bXX family, recipe cited from "the Marvell API"), all in MMD 3:
  • TDR_RESET 0xFECA — init 0x0D90, clear 0x1D90.
  • Calibration: 0xFED9 = 0x010E, 0xFEDA = 0x00EB, 0xFEDE = 0x0058.
  • TDR_STATUS 0xFEDD — write 0x0002 to start; ~500 ms test.
  • Read TDR_STATUS: bits[1:0] = 01 done; [7:4] verdict (0x7 OK / 0xE open / 0x3 short / 0x5 noise); [15:8] distance in meters.
  • Single-pair on the Q2220; the 4-pair CUX3610 likely has per-pair variants nearby. Low bytes are nowhere near the 0x64 trap.
• QCA-style CDT (qcom/qcom.h):
  • 3.0x8064 CDT_STATUS, 3.0x8065–0x8068 per-pair results, 3.0x8074–0x807E thresholds.
  • This IS the trap address — only ever entered via the full start sequence, never a cold status read; treat as off-limits absent strong evidence.
• Next doc-mining lead for the 4-pair pattern: gtAdvVct.c in the Marvell qd-dsdt GPL dumps (e.g. github.com/andy928/xpenology arch/arm/plat-feroceon/mv_hal/qd-dsdt/).

Cable-length physics (applies to both vendors)

• TDR (classic VCT/CDT/ECD):
  • Locates opens/shorts both-ended.
  • But healthy-cable length needs a far-end reflection, and a plugged-in terminated far end absorbs the pulse — so "OK" pairs report no distance. Effectively single-ended for length.
  • Every kernel implementation only reports distance for fault results.
• DSP estimate:
  • Both vendors' PHYs compute linked-cable length continuously from insertion loss during training (Marvell "proactively determines cable length"; the BCM's limited-reach power mode switches "based on cable length" while linked).
  • Works in the product's both-ends-plugged topology — the open question is only whether the estimate is exposed in a readable register.
• NIC timestamp path-delay — the module-independent fallback if the DSP length isn't reachable:
  • Works linked.
  • Needs per-packet or PTP-latch hardware timestamps.
  • Needs a short-cable calibration to cancel fixed PHY latency (~4.8 ns/m one way; 50 m ≈ 240 ns).