Restructure docs: per-device directories under modules/ and nics/ with co-located datasheets, state centralized in state.md, prose tightened into lists and tables

2026-08-12 19:23:02 -07:00
parent dc1fa6538d
commit c57816345f
24 changed files with 511 additions and 311 deletions
@@ -0,0 +1,58 @@
+# Modules
+
+Per-module diagnostics: capabilities, register maps, hazards. Each device directory holds its doc plus its datasheets. NIC-side transports: [../nics/](../nics/README.md).
+
+> ⚠ **Before touching any Wiitek/Marvell module, read the brick trap in [wiitek/](wiitek/README.md).** Two modules died permanently from register exploration.
+
+## The modules
+
+Counts and current placement: [../state.md](../state.md).
+
+| Module | PHY | Doc | Role |
+|---|---|---|---|
+| FS SFP-10G-T-100 | Broadcom BCM84891L | [fs/](fs/README.md) | Test set; documented, robust |
+| Wiitek UF-RJ45-10G-100 | Marvell CUX3610 (Alaska-M) | [wiitek/](wiitek/README.md) | Test set; **brick-prone** |
+| Fibergaga SFP-10G-T-30M | Aquantia AQR | [fibergaga/](fibergaga/README.md) | The documented oracle |
+| 10Gtek | (claims SFP-10G-SR) | — | Copper RJ45 despite the SR part number; filler, not in the test set |
+
+## The media lies (mostly)
+
+**Every test module is a copper RJ45 10GBASE-T module** — none are fiber, whatever the EEPROM claims; the real media is a 10GBASE-T PHY inside each module. EEPROM honesty varies by vendor, with driver consequences:
+
+| Module | EEPROM claims | Stock ixgbe verdict |
+|---|---|---|
+| Wiitek | LC connector, 850 nm SR, multimode lengths, fake optical DOM (temperature real, "laser" powers theater) | Passes qualification |
+| FS | Honest: RJ45 (0x22), 10GBASE-T Short Reach, 100 m copper, no fake DOM | **Rejected** — kills the whole port probe (error -95, no netdev) |
+| 10Gtek | `SFP-10G-SR` part number | (not in the test set) |
+
+- `allow_unsupported_sfp=1` is mandatory on the X520 — and it's the *truthful* module that requires it ([../nics/x520/](../nics/x520/README.md)).
+- Physical-layer reasoning must use 10GBASE-T: PAM16, LDPC FEC, self-synchronizing scrambler, 4 twisted pairs, distance/temperature sensitive — never an optical model.
+- The module PHYs keep the copper link trained on their own: an admin `ip link set down` does NOT drop the wire unless the i40e `link-down-on-close` priv flag is set (peer sees the drop in ~200 ms, relinks in ~0.9 s).
+
+## IEEE 802.3an standard registers (safe on all copper module PHYs)
+
+Proven on the CUX3610 (before the originals died) and the Fibergaga; the BCM notably does **not** populate the SNR set ([fs/](fs/README.md)).
+
+| Registers | What | Notes |
+|---|---|---|
+| PMA 1.133–1.136 / 1.137–1.140 | Per-pair SNR margin, current / minimum | CUX3610 value = `reg − 0x8000` in 0.1 dB. Retrain-dependent — only compare within one training session |
+| PCS 3.32/3.33 | Block-lock loss, BER saturate, errored blocks | Latched, clear-on-read |
+| AN 7.33 | Link-partner status / master-slave | |
+| PMA 1.147 | Fast retrain status/count | Per-run quality metric |
+| AN 7.0 bit 9 | Restart AN — forces retrain, refreshes SNR minimums | Proven safe everywhere. An `ip link` bounce does not drop the copper line, so it won't retrain |
+
+## RollBall protocol (Marvell and Aquantia modules)
+
+Clause-45 access to the internal PHY over I2C A2h, all messages ≤ 4 B, offset-addressed:
+
+1. Unlock: password 0xFF×4 at A2h 0x7B.
+2. Page-select: A2h 0x7F → 3.
+3. Mailbox: A2h 0x80 (cmd) / 0x81 (data), poll for DONE (0x04).
+
+Split transactions (separate STOP per byte) are field-proven on both the Fibergaga and the original Wiiteks; the kernel `mdio-i2c` combined form is not required by the modules.
+
+## Cable-length physics (applies to all vendors)
+
+- **TDR** (classic VCT/CDT/ECD): locates opens/shorts both-ended. But *healthy-cable length* needs a far-end reflection, and a plugged-in terminated far end absorbs the pulse — "OK" pairs report no distance. Effectively single-ended for length; every kernel implementation only reports distance for *fault* results.
+- **DSP estimate**: both vendors' PHYs compute linked-cable length continuously from insertion loss during training (Marvell "proactively determines cable length"; the BCM's limited-reach power mode switches "based on cable length" while linked). Works in the product's both-ends-plugged topology — the question is only whether the estimate is exposed in a readable register (Aquantia: yes, `1E.C884`; BCM/Marvell: not in documented space).
+- **NIC timestamp path-delay** — the module-independent fallback: works linked, needs hardware timestamps and a short-cable calibration to cancel fixed PHY latency (~4.8 ns/m one way; 50 m ≈ 240 ns). Viability gated on PHY-latency stability across retrains ([../open-questions.md](../open-questions.md) §2).