Mixed FS+Wiitek test pair: modules dispatched by EEPROM PN to a BCM or Go RollBall client (whitelist panics outside the wiitek proven-safe set), SNR now solely the wiitek's IEEE margins (BCM handler SNR dropped as unactionable), ECD-clears-AN-enable trap found and fixed (7.0 |= 0x1200 on every restart, explicit restart after each diag), mixed roles FS-slave/wiitek-auto-master, window re-arms at taken edge so the one CMD write per window sits far from the poll; X710 rejects the honest FS EEPROM so the noise pair runs Fibergaga+Wiitek
This commit is contained in:
@@ -26,6 +26,7 @@ Counts and current placement: [../state.md](../state.md).
|
||||
| 10Gtek | `SFP-10G-SR` part number | (not in the test set) |
|
||||
|
||||
- `allow_unsupported_sfp=1` is mandatory on the X520 — and it's the *truthful* module that requires it ([../nics/x520/](../nics/x520/README.md)).
|
||||
- The X710 rejects the FS outright: i40e firmware disables Rx/Tx ("unsupported SFP module type", logged only at driver bind) and i40e has no override parameter — the FS can only live behind ixgbe. The rejection is silent at insertion time; a dead port with `ethtool` reporting no supported link modes is the symptom.
|
||||
- Physical-layer reasoning must use 10GBASE-T: PAM16, LDPC FEC, self-synchronizing scrambler, 4 twisted pairs, distance/temperature sensitive — never an optical model.
|
||||
- The module PHYs keep the copper link trained on their own: an admin `ip link set down` does NOT drop the wire unless the i40e `link-down-on-close` priv flag is set (peer sees the drop in ~200 ms, relinks in ~0.9 s).
|
||||
|
||||
@@ -39,7 +40,7 @@ Proven on the CUX3610 (before the originals died) and the Fibergaga; the BCM not
|
||||
| PCS 3.32/3.33 | Block-lock loss, BER saturate, errored blocks | Latched, clear-on-read |
|
||||
| AN 7.33 | Link-partner status / master-slave | |
|
||||
| PMA 1.147 | Fast retrain status/count | Per-run quality metric |
|
||||
| AN 7.0 bit 9 | Restart AN — forces retrain, refreshes SNR minimums | Proven safe everywhere. An `ip link` bounce does not drop the copper line, so it won't retrain |
|
||||
| AN 7.0 | Restart AN (`\|= 0x1200`, enable + restart) — forces retrain, refreshes SNR minimums | Proven safe everywhere. Always force the enable bit too: the BCM ECD can leave 7.0.12 cleared ([fs/](fs/README.md)), and a bare restart preserves it. An `ip link` bounce does not drop the copper line, so it won't retrain |
|
||||
|
||||
## RollBall protocol (Marvell and Aquantia modules)
|
||||
|
||||
@@ -49,7 +50,7 @@ Clause-45 access to the internal PHY over I2C A2h, all messages ≤ 4 B, offset-
|
||||
2. Page-select: A2h 0x7F → 3.
|
||||
3. Mailbox: A2h 0x80 (cmd) / 0x81 (data), poll for DONE (0x04).
|
||||
|
||||
Split transactions (separate STOP per byte) are field-proven on both the Fibergaga and the original Wiiteks; the kernel `mdio-i2c` combined form is not required by the modules.
|
||||
The full mailbox map (0x80 CMD, 0x81 devad, 0x82/0x83 reg, 0x84/0x85 value) is vendor-confirmed in [wiitek/10gbase-t-phy-register-operation.md](wiitek/10gbase-t-phy-register-operation.md). Split transactions (separate STOP per byte) are field-proven on both the Fibergaga and the original Wiiteks; the kernel `mdio-i2c` combined form is not required by the modules. cabletest carries a Go client (`phy.go`): split ≤4-byte offset writes, single-hold compound reads, whitelist-guarded against the Wiitek brick space — proven live over the patched-ixgbe `sff_i2c` on the X520.
|
||||
|
||||
## Cable-length physics (applies to all vendors)
|
||||
|
||||
|
||||
@@ -33,11 +33,11 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
|
||||
|
||||
| Capability | How | Observed |
|
||||
|---|---|---|
|
||||
| Per-pair SNR | `CMD_GET_SNR` 0x8030, **invoked bare** (writing the documented DATA1 display flag returns zeros; IEEE 1.133–1.140 never populate — constant 0x8080) | DATA2–5 = SNR A–D ×0.1 dB; ≈ 27–32 dB absolute on bench, 0.1–0.4 dB jitter. 10GBASE-T operating point ≈ 26.5 dB, so margin ≈ value − 26.5. cabletest shows the margin classified green ≥ 3 dB / amber ≥ 1 dB / red below — provisional thresholds until the graded-noise correlation run. **Each training re-converges differently**: back-to-back retrains with identical roles, temperature and noise moved one pair by 3.6 dB (roles pinned, launch power pinned — the convergence itself is the variable, and the two bench modules differ in stability: ≤1.5 dB spread on one, 3.6 dB on the other). A single training's absolute SNR is a weak qualification number; qualification wants median-of-N trainings, live SNR is honest as a within-session tracker |
|
||||
| Per-pair SNR | `CMD_GET_SNR` 0x8030, **invoked bare** (writing the documented DATA1 display flag returns zeros; IEEE 1.133–1.140 never populate — constant 0x8080) | DATA2–5 = SNR A–D ×0.1 dB; ≈ 27–32 dB absolute on bench, 0.1–0.4 dB jitter. 10GBASE-T operating point ≈ 26.5 dB, so margin ≈ value − 26.5 — a derived number that never proved actionable, so **cabletest does not poll it** (the panel's SNR comes from an IEEE-margin end — modules/wiitek/ — and each GET costs handler traffic on this µC). **Each training re-converges differently**: back-to-back retrains with identical roles, temperature and noise moved one pair by 3.6 dB (roles pinned, launch power pinned — the convergence itself is the variable, and the two bench modules differ in stability: ≤1.5 dB spread on one, 3.6 dB on the other). A single training's absolute SNR is a weak qualification number; qualification wants median-of-N trainings |
|
||||
| Die temperature | `CMD_GET_CURRENT_TEMP` 0x8031 | ~68–70 °C on bench |
|
||||
| Supply rails | `GET_CURRENT_VOLTAGE` 0x802F | 0.8 V and 1.88 V rails, tenths of mV |
|
||||
| Error counters | IEEE PCS 3.32/3.33 — block lock, latched errored-block/BER, clear-on-read | The noise-stress error proxy |
|
||||
| Retrain on demand | IEEE AN restart `7.0 \|= bit9` | Relinks in ~5–6 s |
|
||||
| Retrain on demand | IEEE AN restart `7.0 \|= 0x1200` (enable + restart — the ECD can leave the enable bit cleared, see below) | Relinks in ~5–6 s |
|
||||
|
||||
## Probed configuration states
|
||||
|
||||
@@ -52,7 +52,7 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
|
||||
| Jumbo | 0x801C/0x801D | GET 0x801D answers in DATA1 — scratch-prone, unusable — and bringup cannot assume a cable to probe through, so cabletest forces enable every boot (SET 0x801C + AN restart); the running mix's 9018-byte frames are the standing wire truth (`phy.go`). The FS at its 9K option passes them at line rate with zero loss/corrupt |
|
||||
| 1588 | 0x8004 | Disabled (engine registers undocumented — see asks below) |
|
||||
| Limited reach | 0x8006 | Disabled — its DATA3 "linked in LR mode" bit, the only handler-visible trace of the DSP length estimate, is unavailable without a config change |
|
||||
| Master/slave role | IEEE 7.32 bits 15/14 | Left to AN the resolution is a per-training lottery; manual config writes stick exactly and the resolution follows deterministically (7.33.14, no config fault with complementary ends). cabletest forces A master / B slave every boot (`phy.go`) so sessions are measured under identical conditions |
|
||||
| Master/slave role | IEEE 7.32 bits 15/14 | Left to AN the resolution is a per-training lottery; manual config writes stick exactly and the resolution follows deterministically (7.33.14, no config fault with complementary ends). cabletest forces A master / B slave every boot when both ends are BCM; with a non-BCM partner (whose manual config is unreachable) the BCM is forced slave and the partner auto-resolves master — proven linking in ~3 s against the Wiitek (`phy.go`) |
|
||||
| Launch power | IEEE 1.130, 1.0.11 | 1.130 reads 0x0003 and is firmware-pinned: host writes to the short-reach bit revert instantly and the value never changes across retrains, so launch power is constant and not host-influenceable. PMA low-power bit clear. The handler catalog has no TX-power command |
|
||||
|
||||
## Firmware-reliability notes
|
||||
@@ -61,7 +61,7 @@ Protocol and full verified catalog: [bcm84891l-mdio-commands.md](bcm84891l-mdio-
|
||||
|
||||
**The internal poll's clock, measured on hardware.** One poll every 3.49–4.22 s, regime-dependent but rock-stable within a regime (±15 ms): ≈3.50 s warm and idle, ≈3.70 s, ≈4.01 s cool and idle, ≈4.21 s under continuous host mailbox load — host traffic *defers* the poll, never advances it, and nothing observed brings two polls closer than 3.49 s. The poll is link-independent: it runs straight through an AN restart and the whole retrain without missing a beat. Around each poll's service the bridge serves stale data for ~50 ms: every read returns the last value successfully fetched before the µC went busy (stuck-at-last-fetch, not one-behind), which is exactly how 3.33 inherits 1.147's 0x0011. The window closes *before* the poll is detectable — a CMD read returning the fresh resident 0x0031 is itself proof the µC serviced the fetch, so flip detection doubles as the all-clear.
|
||||
|
||||
**The windowed protocol (`phy.go`) — contention resolved by time-division.** Since the poll cannot be silenced, the host schedules around it. One goroutine per module owns every transport touch (requests execute one at a time on the loop — nothing else can reach the wire, by construction), and each request is admitted only inside a fixed 3.4 s window following an observed poll (CMD flipping to resident 0x0031, polled at 10 ms); at the cutoff the host goes quiet until the next flip. 3.4 s sits below every observed cadence, so a regime switch mid-run lands harmlessly in vacated time — no prediction, no period tracking, just the last observed flip plus two constants. **Re-lock — no window at assumed phase**: edge detection needs CMD ≠ 0x0031, and normal work leaves it armed (every handler command rewrites CMD), but after a gap with no commands — a diag plus its relink wait, an idle stretch, a poll that skipped its GET because the link bit read down — the resident is 0x0031 and the phase is unknown. Admission then arms first (a bare GET_PAIR_SWAP, resident 0x0000) and anchors only on the true edge that follows, so every window is edge-locked from first contact at boot; bringup pays one phase acquisition per module (~2–4 s). A missing heartbeat free-runs one window rather than stall (never observed — the heartbeat survives retrains). Before re-lock the post-diag window anchored blind on the stale resident, and every ghost observed under windowing sat in exactly that window; with re-lock, ghost-free across all bench diags to date. A poisoned batch is self-consistent and forensically visible: the stuck value clears the link bit (SNR skipped, panel shows "-", the ghost-SNR panic sidestepped) while 3.33 charges the same value as errored blocks (0x0011 → the "+17"); every corrected charge and link-down reading prints its raw 1.1/3.33/1.147 on the console, so events self-attribute — a ghost names its neighbor register, real correction decodes as nothing but itself. Validated on hardware in `~/work/phydiag-work/bcm_phaselock_bench.py`: aiming read bursts at the poll reproduces the poisoning on 100% of polls (the "+17" manufactured on demand); windowed operation ran 2,881 back-to-back batches — 16× the production rate, riding the cutoff — with zero stale values, and the production 1 Hz stream is unperturbed (blackout hides inside natural gaps, worst sample gap ~1.6 s).
|
||||
**The windowed protocol (`phy.go`) — contention resolved by time-division.** Since the poll cannot be silenced, the host schedules around it. One goroutine per module owns every transport touch (requests execute one at a time on the loop — nothing else can reach the wire, by construction), and each request is admitted only inside a fixed 3.4 s window following an observed poll (CMD flipping to resident 0x0031, polled at 10 ms); at the cutoff the host goes quiet until the next flip. 3.4 s sits below every observed cadence, so a regime switch mid-run lands harmlessly in vacated time — no prediction, no period tracking, just the last observed flip plus two constants. **Re-lock — no window at assumed phase**: edge detection needs CMD ≠ 0x0031, and handler commands leave it armed (every one rewrites CMD); whenever CMD sits at the resident 0x0031 at expiry the phase is unknown and admission arms first, then anchors on the true edge that follows. Steady-state polling issues no handler commands of its own (the SNR GET is dropped), so each taken edge immediately re-arms with a bare GET_PAIR_SWAP: the single CMD write per window lands at the start of the quiet period, maximally far from the next poll — the wedge risk is writes near the poll — and every later expiry reads the phase without writing. One command per ~4 s window replaces a handler GET per second. Admission then arms first (a bare GET_PAIR_SWAP, resident 0x0000) and anchors only on the true edge that follows, so every window is edge-locked from first contact at boot; bringup pays one phase acquisition per module (~2–4 s). A missing heartbeat free-runs one window rather than stall (never observed — the heartbeat survives retrains). Before re-lock the post-diag window anchored blind on the stale resident, and every ghost observed under windowing sat in exactly that window; with re-lock, ghost-free across all bench diags to date. A poisoned batch is self-consistent and forensically visible: the stuck value clears the link bit (SNR skipped, panel shows "-", the ghost-SNR panic sidestepped) while 3.33 charges the same value as errored blocks (0x0011 → the "+17"); every corrected charge and link-down reading prints its raw 1.1/3.33/1.147 on the console, so events self-attribute — a ghost names its neighbor register, real correction decodes as nothing but itself. Validated on hardware in `~/work/phydiag-work/bcm_phaselock_bench.py`: aiming read bursts at the poll reproduces the poisoning on 100% of polls (the "+17" manufactured on demand); windowed operation ran 2,881 back-to-back batches — 16× the production rate, riding the cutoff — with zero stale values, and the production 1 Hz stream is unperturbed (blackout hides inside natural gaps, worst sample gap ~1.6 s).
|
||||
|
||||
## ECD — recovered from the OpenBCM SDK, proven on hardware
|
||||
|
||||
@@ -76,6 +76,7 @@ The ECD register mechanism is absent from the handler catalog and the datasheet
|
||||
Observed on a plugged, linked, healthy cable: verdicts OK×4 and per-pair lengths of [45, 45, 41, 46] on a ~45 m cable — **meter-accurate with no calibration**, and this ECD reports length for healthy pairs, not just faults, resolving the terminated-far-end concern in [../README.md](../README.md). Caveats:
|
||||
|
||||
- The run blips the link (PMA 1.1 latch-low catches a drop even with the break-link bit clear) — do not run mid-measurement until the disturbance is characterized.
|
||||
- **The run can leave clause-45 AN disabled** (7.0.12 cleared — observed live after diags racing an in-progress retrain): with AN off the BCM emits no AN pulses, both ends report no link partner (7.1 LP-AN-able 0), and the link stays down through AN restarts, role changes and PMA resets until the bit is re-enabled. cabletest restarts AN with `7.0 |= 0x1200` (enable + restart) after every diag, and `devRestartAN` always forces the enable bit — a bare bit-9 restart preserves the cleared enable forever.
|
||||
- **An interrupted run can wedge the µC's SMI service** (observed after a process died mid-diag): the bridge ACKs I2C but serves 0x0000 for every register at any delay and silently drops MDIO writes — no in-band recovery, not even PMA reset — while the EEPROM path and the trained link keep working. Recovery: a driver unbind/rebind of the port (the SFP re-initialization resets the µC); a reseat would do the same.
|
||||
- Fault verdicts (open/short/inter-pair) are unexercised — deliberately: the product is a closed-loop tester, both ends always plugged.
|
||||
- Family constraints from the SDK: port must be enabled; unsupported at forced 100M.
|
||||
|
||||
@@ -21,14 +21,23 @@ The shared IEEE 802.3an set ([../README.md](../README.md)) plus two safe vendor
|
||||
|
||||
| Registers | What | Reading |
|
||||
|---|---|---|
|
||||
| PMA 1.133–1.136 / 1.137–1.140 | Per-pair SNR margin (offset-0x8000, 0.1 dB) | [8.8, 7.0, 7.3, 8.9] dB |
|
||||
| PMA 1.133–1.136 / 1.137–1.140 | Per-pair SNR margin (offset-0x8000, 0.1 dB) | [8.8, 7.0, 7.3, 8.9] dB (as auto-master vs FS manual-slave: [9.2, 7.2, 7.0, 8.7]) |
|
||||
| PCS 3.32 / 3.33 | Errored-block / block-lock latches | 0x1001 / 0x7f00 |
|
||||
| AN 7.33 | Master/slave, link partner | 0x3c62 |
|
||||
| AN 7.33 | Master/slave, link partner | 0x3c62 (as master against the FS: 0x7c62) |
|
||||
| 1.0xC011 / 12 | Firmware version | 0x0c01 / 0x0500 (12.1.5.0) |
|
||||
| 1.0xC050 | Boot status | 0x087e |
|
||||
| PMA 1.0 | Speed selection | 0x2040 (10G) — see the reset trap below |
|
||||
| AN 7.0 / 7.1 / 7.32 / 7.60 | AN control/status, 10GBASE-T control, EEE advert | 0x3000 / 0x0008 idle · 0x00ad linked / 0x11e1 (auto role, 10G advert) / 0x0000 (no EEE advertised) |
|
||||
| A2.250 (Table 3) | PHY part number ([10gbase-t-phy-register-operation.md](10gbase-t-phy-register-operation.md)) | 9 on the CUX3610 (1 is documented as 88X3310) |
|
||||
|
||||
Proven-safe writes, all IEEE standard space: AN restart `7.0 |= 0x1200`, and PMA reset `1.0 = 0x8000` — **the PMA reset clears the speed-selection bits** (1.0 reads 0x0000 after, no speed selected, 10G training impossible) and does not touch 7.x config; write 1.0 = 0x2040 back after any reset or the link never returns.
|
||||
|
||||
`rb_forensic.py` in phydiag-work performs this baseline and logs every sff_i2c op (PRE line fsync'd before each op, POST after) to `wiitek-ops.log` — a PRE with no POST names whatever killed the µC. `rollball_ixgbe.py` hard-guards the `*.0x??64` blacklist and raises before touching hardware.
|
||||
|
||||
## In the test set — mixed pair with the FS
|
||||
|
||||
cabletest drives this module through its own Go RollBall client (`phy.go`): module type dispatched from the EEPROM vendor PN before any protocol touch, mailbox ops as split ≤4-byte offset writes plus single-hold compound reads over the patched-ixgbe `sff_i2c`, and a **whitelist guard** — every MDIO read/write is checked against the proven-safe set above and panics before touching hardware on anything else, so the client structurally cannot repeat the kill. The 1 Hz poller reads link (1.1), per-pair IEEE margins (1.133–136, used directly — no operating-point subtraction), corrected latches (3.33) and fast-retrain count (1.147); margins ride ghost bounds of −10…+25 dB. **This module's IEEE margins are the panel's only SNR source** — the FS's handler SNR is not polled at all (the derived number was never actionable and each GET costs BCM handler traffic), so an all-BCM pair shows no SNR. Roles in the mixed pair: the FS is forced manual-slave and this module auto-resolves master (its 7.32 reads auto and cannot be safely forced) — links in ~3 s. EEE stays down one-sided (this unit advertises none; the FS forces none). Module-level jumbo cannot be forced here; the running mix's 9018-byte frames pass at line rate regardless. Length comes from the FS end's ECD with this module as the far end: healthy per-pair verdicts and lengths within a few meters of the FS↔FS baseline on the same cable.
|
||||
|
||||
Not available in safe space: pre-FEC counter, cable length — those live in the µC vendor space that killed the modules.
|
||||
|
||||
## VCT/cable length: no confirmed-safe path
|
||||
|
||||
+8
-8
@@ -2,7 +2,7 @@
|
||||
|
||||
## Committed tree
|
||||
|
||||
AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet-MAC-rx-stamped rate buckets (`SO_TIMESTAMPING` cmsg, `rx_filter=ALL` as a hard host check — nics/README.md for what that demands of the NIC; **temporarily bypassed** in `ts.go` so BCM work can run on the X520, which cannot stamp — the check reports yellow and the panel rates read zero there; restore to fatal for the product NIC); read-time-stamped NIC-counter rates; test interfaces pinned to MTU 9000 with a 9018-byte jumbo in the size mix (the modules' jumbo path is exercised, not assumed); BCM module diagnostics (`phy.go`, over the patched-ixgbe `sff_i2c` debugfs, compound-op framing; one loop goroutine per module owns every transport touch, and all work is time-division-multiplexed with the firmware's own mailbox client — each request admitted only inside a 3.4 s edge-locked window after an observed internal temp poll, re-acquiring phase whenever it was lost — which ends the stale-read poisoning of the corrected channel; corrected charges and link-down reads print their raw registers so events self-attribute — modules/fs/ for the full trap list and validation): bringup identifies both modules and forces EEE off, jumbo on and the master/slave roles (A master / B slave — left to AN the role is a per-training lottery) every boot — no trustworthy readback exists and no cable is guaranteed to probe through; the ECD — per-pair verdicts, lengths and pair maps are the length/wiring path — runs through one async path at startup and on every reset, never blocking the UI, with counters re-baselining only after the diag's own link blip so it is never charged to the run; a 1 Hz poller feeds per-pair SNR margin (vs the ≈26.5 dB operating point; green ≥ 3 dB, amber ≥ 1 dB — provisional until the graded-noise run) and the corrected-error set (PCS 3.33 errored blocks/BER, PMA 1.147 fast-retrain count) to the panel and console; framebuffer UI; harness.
|
||||
AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet-MAC-rx-stamped rate buckets (`SO_TIMESTAMPING` cmsg, `rx_filter=ALL` as a hard host check — nics/README.md for what that demands of the NIC; **temporarily bypassed** in `ts.go` so BCM work can run on the X520, which cannot stamp — the check reports yellow and the panel rates read zero there; restore to fatal for the product NIC); read-time-stamped NIC-counter rates; test interfaces pinned to MTU 9000 with a 9018-byte jumbo in the size mix (the modules' jumbo path is exercised, not assumed); module diagnostics (`phy.go`, over the patched-ixgbe `sff_i2c` debugfs; module type dispatched from the EEPROM vendor PN before any protocol touch, with a BCM client — compound-op framing, one loop goroutine per module owning every transport touch, all work time-division-multiplexed with the firmware's own mailbox client: each request admitted only inside a 3.4 s edge-locked window after an observed internal temp poll, re-acquiring phase whenever it was lost, which ends the stale-read poisoning of the corrected channel — and a RollBall client for the Wiitek — unlock/page/mailbox in Go, whitelist-guarded so any register outside the proven-safe set panics before touching hardware; corrected charges and link-down reads print their raw registers so events self-attribute — modules/fs/ and modules/wiitek/ for the trap lists and validation): bringup identifies both modules and forces EEE off, jumbo on and the master/slave roles on every BCM end every boot (A master / B slave when both ends are BCM; with a non-BCM partner the BCM is forced slave and the partner auto-resolves master — the proven mixed combination) — no trustworthy readback exists and no cable is guaranteed to probe through — and every AN restart forces the enable bit (`7.0 |= 0x1200`: the ECD can leave 7.0.12 cleared, which mutes AN entirely and holds the link down through restarts and resets), with an explicit AN restart on the ECD end after every diag; the ECD — per-pair verdicts, lengths and pair maps are the length/wiring path — runs through one async path at startup and on every reset, never blocking the UI, with counters re-baselining only after the diag's own link blip so it is never charged to the run; a 1 Hz poller feeds per-pair SNR margin and the corrected-error set (PCS 3.33 errored blocks/BER, PMA 1.147 fast-retrain count) to the panel and console — SNR comes only from ends that measure IEEE margins (the Wiitek's 1.133–136, the reason it is in the pair); the BCM's handler SNR is not polled (the derived number was never actionable and each GET costs handler traffic), so an all-BCM pair shows no SNR; green ≥ 3 dB, amber ≥ 1 dB — provisional until the graded-noise run; framebuffer UI; harness.
|
||||
|
||||
## Stashes
|
||||
|
||||
@@ -17,20 +17,20 @@ AF_PACKET raw sockets everywhere (`sock.go`); flow-director steering; per-packet
|
||||
|
||||
| Interface | Device | Role (rules: hardware.md) |
|
||||
|---|---|---|
|
||||
| `enp1s0f0` | X520 port 0 (ixgbe) | Test pair — FS module |
|
||||
| `enp1s0f1` | X520 port 1 (ixgbe) | Test pair — FS module |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Noise pair (has been `enp4s0f*` across reboots) |
|
||||
| `enp1s0f0` | X520 port 0 (ixgbe) | Test pair — Wiitek module (SNR end) |
|
||||
| `enp1s0f1` | X520 port 1 (ixgbe) | Test pair — FS module (length end) |
|
||||
| `enp3s0f0np0` / `enp3s0f1np1` | X710 (i40e) | Noise pair (has been `enp4s0f*` across reboots). **The FS cannot live here**: X710 firmware disables Rx/Tx on the honest FS EEPROM ("unsupported SFP module type") and i40e has no `allow_unsupported_sfp` — the noise pair needs lying modules. Seated: Fibergaga (f0) + Wiitek SN …X170U (f1), both qualified, noise cycle live |
|
||||
| `enp88s0` | igc | LAN uplink, default route; sibling `enp89s0` is dark |
|
||||
|
||||
## Hardware
|
||||
|
||||
| Item | Status | Notes |
|
||||
|---|---|---|
|
||||
| X520-DA2 | **Installed** in the single PCIe slot (E810 out); PCIe 5 GT/s ×8 | Both ports FS (port 1 SN S2433774168); cable linked at 10G. Stock ixgbe needs `allow_unsupported_sfp=1` — the *FS* trips qualification (hardware.md) |
|
||||
| X520-DA2 | **Installed** in the single PCIe slot (E810 out); PCIe 5 GT/s ×8 | Mixed test pair: port 0 Wiitek (SN WAMZ012606X039U), port 1 FS (SN S2433774168); cable linked at 10G. Stock ixgbe needs `allow_unsupported_sfp=1` — the *FS* trips qualification (hardware.md) |
|
||||
| ConnectX-5 | **Ordered** (dual SFP28, PCIe x8) — the product NIC candidate | mlx5 is the one driver meeting the full requirement set: stamps every packet, shared PHC across ports, native ETHER_FLOW steering. Open: MCIA diagnostics questions (nics/connectx-5/). Arrival notes: ports may ship in InfiniBand mode (`mlxconfig set LINK_TYPE_P1=2 LINK_TYPE_P2=2`); SFP+ drops into SFP28 cages at 10G; check `mlx5_ib` vs channel changes (the irdma lesson) |
|
||||
| Replacement Wiiteks | Arrived; on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first |
|
||||
| FS SFP-10G-T-100 ×2 | Both in the X520 test pair | BCM84891L, documented, robust. **2× FS at both ends is the expected product module config** unless mixed ends prove wanted |
|
||||
| Fibergaga SFP-10G-T-30M | In hand | Aquantia, RollBall, the documented oracle |
|
||||
| Replacement Wiiteks | One in the X520 test pair (port 0), rest on the shelf | Originals bricked by register exploration — modules/wiitek/ trap first |
|
||||
| FS SFP-10G-T-100 ×2 | One in the X520 test pair (port 1), one on the shelf (rejected by the X710 — ixgbe-only) | BCM84891L, documented, robust. The mixed FS+Wiitek pair is the configuration under evaluation: length from the FS ECD, IEEE per-pair SNR margins from the Wiitek — the best of both worlds a 2× FS pair cannot give (the BCM never populates IEEE SNR) |
|
||||
| Fibergaga SFP-10G-T-30M | In the X710 noise pair (f0) | Aquantia, RollBall, the documented oracle |
|
||||
| 10Gtek | In hand | Claims SFP-10G-SR, still copper RJ45; filler, not in the test set |
|
||||
| E810 | Out of the box | Patched ice + `sff_i2c` remains useful only if it returns for read-side work |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user