59 lines
5.1 KiB
Markdown
59 lines
5.1 KiB
Markdown
# Modules
|
||
|
||
Per-module diagnostics: capabilities, register maps, hazards. Each device directory holds its doc plus its datasheets. NIC-side transports: [../nics/](../nics/README.md).
|
||
|
||
> ⚠ **Before touching any Wiitek/Marvell module, read the brick trap in [wiitek/](wiitek/README.md).** Two modules died permanently from register exploration.
|
||
|
||
## The modules
|
||
|
||
Counts and current placement: [../state.md](../state.md).
|
||
|
||
| Module | PHY | Doc | Role |
|
||
|---|---|---|---|
|
||
| FS SFP-10G-T-100 | Broadcom BCM84891L | [fs/](fs/README.md) | Test set; documented, robust |
|
||
| Wiitek UF-RJ45-10G-100 | Marvell CUX3610 (Alaska-M) | [wiitek/](wiitek/README.md) | Test set; **brick-prone** |
|
||
| Fibergaga SFP-10G-T-30M | Aquantia AQR | [fibergaga/](fibergaga/README.md) | The documented oracle |
|
||
| 10Gtek | (claims SFP-10G-SR) | — | Copper RJ45 despite the SR part number; filler, not in the test set |
|
||
|
||
## The media lies (mostly)
|
||
|
||
**Every test module is a copper RJ45 10GBASE-T module** — none are fiber, whatever the EEPROM claims; the real media is a 10GBASE-T PHY inside each module. EEPROM honesty varies by vendor, with driver consequences:
|
||
|
||
| Module | EEPROM claims | Stock ixgbe verdict |
|
||
|---|---|---|
|
||
| Wiitek | LC connector, 850 nm SR, multimode lengths, fake optical DOM (temperature real, "laser" powers theater) | Passes qualification |
|
||
| FS | Honest: RJ45 (0x22), 10GBASE-T Short Reach, 100 m copper, no fake DOM | **Rejected** — kills the whole port probe (error -95, no netdev) |
|
||
| 10Gtek | `SFP-10G-SR` part number | (not in the test set) |
|
||
|
||
- `allow_unsupported_sfp=1` is mandatory on the X520 — and it's the *truthful* module that requires it ([../nics/x520/](../nics/x520/README.md)).
|
||
- The X710 rejects the FS outright: i40e firmware disables Rx/Tx ("unsupported SFP module type", logged only at driver bind) and i40e has no override parameter — the FS can only live behind ixgbe. The rejection is silent at insertion time; a dead port with `ethtool` reporting no supported link modes is the symptom.
|
||
- Physical-layer reasoning must use 10GBASE-T: PAM16, LDPC FEC, self-synchronizing scrambler, 4 twisted pairs, distance/temperature sensitive — never an optical model.
|
||
- The module PHYs keep the copper link trained on their own: an admin `ip link set down` does NOT drop the wire unless the i40e `link-down-on-close` priv flag is set (peer sees the drop in ~200 ms, relinks in ~0.9 s).
|
||
|
||
## IEEE 802.3an standard registers (safe on all copper module PHYs)
|
||
|
||
Proven on the CUX3610 (before the originals died) and the Fibergaga; the BCM notably does **not** populate the SNR set ([fs/](fs/README.md)).
|
||
|
||
| Registers | What | Notes |
|
||
|---|---|---|
|
||
| PMA 1.133–1.136 / 1.137–1.140 | Per-pair SNR margin, current / minimum | CUX3610 value = `reg − 0x8000` in 0.1 dB. Retrain-dependent — only compare within one training session |
|
||
| PCS 3.32/3.33 | Block-lock loss, BER saturate, errored blocks | Latched, clear-on-read |
|
||
| AN 7.33 | Link-partner status / master-slave | |
|
||
| PMA 1.147 | Fast retrain status/count | Per-run quality metric |
|
||
| AN 7.0 | Restart AN (`\|= 0x1200`, enable + restart) — forces retrain, refreshes SNR minimums | Proven safe everywhere. Always force the enable bit too: the BCM ECD can leave 7.0.12 cleared ([fs/](fs/README.md)), and a bare restart preserves it. An `ip link` bounce does not drop the copper line, so it won't retrain |
|
||
|
||
## RollBall protocol (Marvell and Aquantia modules)
|
||
|
||
Clause-45 access to the internal PHY over I2C A2h, all messages ≤ 4 B, offset-addressed:
|
||
|
||
1. Unlock: password 0xFF×4 at A2h 0x7B.
|
||
2. Page-select: A2h 0x7F → 3.
|
||
3. Mailbox: A2h 0x80 (cmd) / 0x81 (data), poll for DONE (0x04).
|
||
|
||
The full mailbox map (0x80 CMD, 0x81 devad, 0x82/0x83 reg, 0x84/0x85 value) is vendor-confirmed in [wiitek/10gbase-t-phy-register-operation.md](wiitek/10gbase-t-phy-register-operation.md). Split transactions (separate STOP per byte) are field-proven on both the Fibergaga and the original Wiiteks; the kernel `mdio-i2c` combined form is not required by the modules. cabletest carries a Go client (`phy.go`): split ≤4-byte offset writes, single-hold compound reads, whitelist-guarded against the Wiitek brick space — proven live over the patched-ixgbe `sff_i2c` on the X520.
|
||
|
||
## Cable-length physics (applies to all vendors)
|
||
|
||
- **TDR** (classic VCT/CDT/ECD): locates opens/shorts both-ended. The generic concern — *healthy-cable length* needs a far-end reflection, and a plugged-in terminated far end absorbs the pulse, so "OK" pairs report no distance (every kernel implementation only reports distance for *fault* results) — is disproven for the BCM: its ECD reports per-pair length for healthy pairs on a plugged, terminated, linked cable, meter-accurate ([fs/](fs/README.md)). **The BCM ECD is the product length path.**
|
||
- **DSP estimate**: both vendors' PHYs compute linked-cable length continuously from insertion loss during training (Marvell "proactively determines cable length"; the BCM's limited-reach power mode switches "based on cable length" while linked). Works in the product's both-ends-plugged topology — the question is only whether the estimate is exposed in a readable register (Aquantia: yes, `1E.C884`; BCM/Marvell: not in documented space).
|