34 lines
5.2 KiB
Markdown
34 lines
5.2 KiB
Markdown
# Intel E810 (ice)
|
|
|
|
Out of the box. The measurement path was originally built against it — its datapath delivers per-packet MAC RX timestamps (`rx_filter=ALL`), which the committed rate design no longer needs ([../README.md](../README.md)). The `sff_i2c` ice patch for read-side diagnostics is vendored here as [ice-sff-i2c.patch](ice-sff-i2c.patch); to run it, apply to a copy of the kernel's ice source and build out of tree (`make -C /usr/lib/modules/$(uname -r)/build M=<dir> modules`, then `rmmod irdma ice; insmod`), with two out-of-tree fixups: `ice_trace.h` needs `TRACE_INCLUDE_PATH` set to the build dir's absolute path and `TRACE_INCLUDE_FILE ice_trace`, and the Makefile's object list must match the source snapshot's kernel version.
|
|
|
|
## Host notes (if it returns)
|
|
|
|
- `irdma` autoloads and binds ice ports, making `ETHTOOL_SCHANNELS` fail EBUSY ("Cannot change channels when RDMA is active") and failing the channels host check. `sudo rmmod irdma` (usage count 0; returns on reboot). The appliance kernel has no irdma.
|
|
- ice refuses channel changes while ntuple rules exist — cabletest clears its own stale rules first.
|
|
|
|
## RX error counter reference set
|
|
|
|
The richest per-frame RX error breakdown of the NICs used — `illegal_bytes` and the MAC faults move while every frame still arrives intact, as close to a bit error rate as a link reports. `ethtool -S` names: `rx_crc_errors.nic`, `rx_jabber.nic`, `rx_undersize.nic`, `rx_oversize.nic`, `rx_fragments.nic`, `rx_dropped.nic`, `illegal_bytes.nic`, `mac_local_faults.nic`, `mac_remote_faults.nic`. Read the driver's own array, never sysfs beside it: ice folds crc errors and jabbers into `rx_errors`, so summing `rx_errors` alongside `rx_crc_errors` charges every bad frame twice.
|
|
|
|
## Module I2C: multi-byte framing works, writes are policy-blocked
|
|
|
|
- topo-I2C (0x06E2/E3) has offset-size control: params bit[7] repeated-start, [6:5] address length, [3:0] data size. **Reads work perfectly.**
|
|
- Every write returns aq 1 (EPERM) — tested identically against EEPROM 0x50 and BCM 0x56, so the block is write-vs-read, not address-scoped. Intel policy, survives NVM update; NVM 5.01 hardened the 3.10 silent-drop into an explicit EPERM.
|
|
- Dead for anything needing writes: RollBall unlock needs a write; BCM SMI reads *start* with a write.
|
|
|
|
Read-side transport details (the patched ice exists for read-side work, so these stay load-bearing):
|
|
|
|
- CTX_OVERRIDE (ntc 0x50, handle 0) reaches the real wire; `index` is the raw I2C bus number and equals the port number (verified by serial), so one PF0 debugfs file reaches both cages. NACK reads report `err -5 aq 1`.
|
|
- Cage-node addressing (ntc 0x26/0x46, handle from the netlist) returns a **firmware cache** of the module EEPROM, not the wire: unknown regions read zeros, and the cache serves stale bytes after the wire changes. Don't use it.
|
|
- Override bus 5 ACKs every address with zeros — a firmware phantom.
|
|
- 16-bit-address mode (params bit4) turns the offset phase into a 2-byte write an SFF EEPROM can latch as data — never use it on 0x50/0x51.
|
|
- No offset-less read mode exists, so the plain mdio-i2c clause-45 read phase (bcm84881-style at 0x56) cannot be formed. RollBall fits fully (≤4 B native multi-byte writes).
|
|
- Latency is the wire-truth judge on this transport: on 3.10 firmware, silently-dropped 1-byte writes completed in ~63 µs while real wire traffic ran ~103 µs/byte (reads) and multi-ms ack-polling (writes), and `ok` returns even for writes at an empty address — protocol-level readback is the only delivery confirmation.
|
|
|
|
Firmware on the card: NVM 5.01 (release 31.2.2, EETRACK 0x80021FA2), flashed with `nvmupdate64e` 1.43.37.1a from Intel's `E810_NVMUpdatePackage_v5_01_Linux.tar.gz` (re-downloadable; nothing kept locally). Flash logistics: the tool needs `-AllowMissingCert` on this platform (the MS UEFI CA 2023 cert is absent — harmless, nothing boots from the NIC), writes no file backup even with `-b` ("Backup images created" means the card's inactive flash bank, where rollback lives: `nvmupdate64e -r -m B49691D578EC`), and the post-flash inventory fails until a power cycle activates the new bank — that failure is the expected tell, not an error.
|
|
|
|
## The bit-bang-below-the-AQ idea — retired with the timestamp requirement
|
|
|
|
The write block is at the **Admin Queue** (firmware) level — the `0x06E2/E3` topo-I2C commands are refused with EPERM. That is not the same as the module's SDA/SCL being unreachable: the 82599 reaches the module I2C **without firmware** by toggling SDA/SCL as GPIO bits in `I2CCTL` (the `kernel/ixgbe-sff/` bench driver bit-bangs arbitrary START…STOP from those primitives), and whether the E810 exposes an equivalent register-level / SDP-GPIO path below the AQ policy was never tested. The idea existed to marry all-packet stamps to raw I2C in one card; the smoothed-bucket rate plan ([../x520/README.md](../x520/README.md)) dissolves the stamps requirement, so the X520 covers everything and this investigation is not needed. It stays recorded as the E810's one open door if per-frame arrival times are ever wanted again — the investigation would start at the datasheet's SDP/GPIO and low-level I2C controller registers, asking whether the cage's SDA/SCL are host-addressable.
|