cabletest/docs/nics/e810/README.md

Intel E810 (ice)

Out of the box. The measurement path was originally built against it — its datapath delivers per-packet MAC RX timestamps (rx_filter=ALL), which the committed rate design no longer needs (../README.md). A patched ice + sff_i2c exists for read-side diagnostics work.

Host notes (if it returns)

• irdma autoloads and binds ice ports, making ETHTOOL_SCHANNELS fail EBUSY ("Cannot change channels when RDMA is active") and failing the channels host check. sudo rmmod irdma (usage count 0; returns on reboot). The appliance kernel has no irdma.
• ice refuses channel changes while ntuple rules exist — cabletest clears its own stale rules first.

RX error counter reference set

The richest per-frame RX error breakdown of the NICs used — illegal_bytes and the MAC faults move while every frame still arrives intact, as close to a bit error rate as a link reports. ethtool -S names: rx_crc_errors.nic, rx_jabber.nic, rx_undersize.nic, rx_oversize.nic, rx_fragments.nic, rx_dropped.nic, illegal_bytes.nic, mac_local_faults.nic, mac_remote_faults.nic. Read the driver's own array, never sysfs beside it: ice folds crc errors and jabbers into rx_errors, so summing rx_errors alongside rx_crc_errors charges every bad frame twice.

Module I2C: multi-byte framing works, writes are policy-blocked

• topo-I2C (0x06E2/E3) has offset-size control: params bit[7] repeated-start, [6:5] address length, [3:0] data size. Reads work perfectly.
• Every write returns aq 1 (EPERM) — tested identically against EEPROM 0x50 and BCM 0x56, so the block is write-vs-read, not address-scoped. Intel policy, survives NVM update; NVM 5.01 hardened the 3.10 silent-drop into an explicit EPERM.
• Dead for anything needing writes: RollBall unlock needs a write; BCM SMI reads start with a write.

Read-side transport details (the patched ice exists for read-side work, so these stay load-bearing):

• CTX_OVERRIDE (ntc 0x50, handle 0) reaches the real wire; index is the raw I2C bus number and equals the port number (verified by serial), so one PF0 debugfs file reaches both cages. NACK reads report err -5 aq 1.
• Cage-node addressing (ntc 0x26/0x46, handle from the netlist) returns a firmware cache of the module EEPROM, not the wire: unknown regions read zeros, and the cache serves stale bytes after the wire changes. Don't use it.
• Override bus 5 ACKs every address with zeros — a firmware phantom.
• 16-bit-address mode (params bit4) turns the offset phase into a 2-byte write an SFF EEPROM can latch as data — never use it on 0x50/0x51.
• No offset-less read mode exists, so the plain mdio-i2c clause-45 read phase (bcm84881-style at 0x56) cannot be formed. RollBall fits fully (≤4 B native multi-byte writes).
• Latency is the wire-truth judge on this transport: on 3.10 firmware, silently-dropped 1-byte writes completed in ~63 µs while real wire traffic ran ~103 µs/byte (reads) and multi-ms ack-polling (writes), and ok returns even for writes at an empty address — protocol-level readback is the only delivery confirmation.

Firmware on the card: NVM 5.01 (release 31.2.2, EETRACK 0x80021FA2). The update package and logs live in ~/work/e810nvm/; nvmupdate64e needs -AllowMissingCert on this platform (harmless — nothing boots from the NIC), writes no file backup, and rollback lives in the card's inactive flash bank (nvmupdate64e -r -m <MAC>).

The bit-bang-below-the-AQ idea — retired with the timestamp requirement

The write block is at the Admin Queue (firmware) level — the 0x06E2/E3 topo-I2C commands are refused with EPERM. That is not the same as the module's SDA/SCL being unreachable: the 82599 reaches the module I2C without firmware by toggling SDA/SCL as GPIO bits in I2CCTL (the ~/work/ixgbe-sff/ patch bit-bangs arbitrary START…STOP from those primitives), and whether the E810 exposes an equivalent register-level / SDP-GPIO path below the AQ policy was never tested. The idea existed to marry all-packet stamps to raw I2C in one card; the smoothed-bucket rate plan (../x520/README.md) dissolves the stamps requirement, so the X520 covers everything and this investigation is not needed. It stays recorded as the E810's one open door if per-frame arrival times are ever wanted again — the investigation would start at the datasheet's SDP/GPIO and low-level I2C controller registers, asking whether the cage's SDA/SCL are host-addressable.