Frequent MeshForwarder NoAck in an otherwise healthy mixed Apple/OTBR Thread network — expected verbosity or something to fix?

Configuration Matter/Thread
1

Setup

  • Thread network: MyHome67, Thread 1.3.0
  • Border Routers:
    • Home Assistant OTBR add-on, radio = Nabu Casa ZBT-2 (Silicon Labs EFR32MG24), connected via direct USB to a Minisforum MS-01 running HAOS in a Proxmox VM
    • 2x Apple TV (one historically Leader, BBR role has since moved between them)
    • 5x HomePod (mini/2nd gen mix)
  • Other mesh devices: IKEA (ALPSTUGA, GRILLPLATS) and Eve Thread routers/end devices, various Matter sensors
  • Backbone interface fix (multi-AIL) and TREL-disable already applied per earlier troubleshooting — network has otherwise been stable for months

Symptom

Over several days, the OTBR add-on’s log (via Loki/Grafana) shows a steady stream of:

MeshForwarder-: Failed to send IPv6 UDP msg, len:90, chksum:xxxx, ecn:no, to:0xXXXX, sec:yes, error:NoAck, prio:low, radio:15.4

and, less frequently:

Mle-----------: Failed to process Data Request: Security
Mle-----------: Failed to process Link Accept: Security
P-RadioSpinel-: Handle transmit done failed: ChannelAccessFailure

No user-visible symptoms accompany this — no Matter commands failing, no HomeKit “not responding” events tied to it, no devices dropping out of Home Assistant. The volume of NoAck lines alone is what prompted the investigation (roughly 100+ some days), after a one-off, clearly-explained incident (an Apple TV going briefly unreachable, restarted by us, mesh reshuffled around it) raised our attention to these logs in the first place.

What we ruled out

Going through this systematically rather than guessing:

  • Electrical/RF noise from appliances — a kitchen kettle boil once coincided with a NoAck burst. A same-day follow-up test with the kettle and microwave both running produced zero errors, and a separate burst occurred with no appliance in use nearby. Not a reproducible cause.
  • Wi-Fi channel interference / antenna placement — checked 2.4GHz channel plan, neighboring AP density, and ran a live RF spectrum scan on the APs nearest the affected devices. No abnormal noise floor, no non-Wi-Fi interference signature detected. The OTBR’s own radio (ZBT-2) is on its own stand, well clear of any equipment cluster.
  • Host/Proxmox performance — checked CPU usage on both the Proxmox host and the HA VM across the period in question. Flat and low (1–7%) throughout; no scheduling contention that would explain missed radio servicing.
  • OTBR add-on restarts / BBR–Leader role transitions — checked logs specifically for Backbone, BBR, partition, role changed, attach/detach, leader in the exact window of a NoAck burst. Zero matches. The burst was not preceded or accompanied by any logged role transition.
  • Radio/USB/Spinel transport reliability — this is the one that mattered most. Over ~30 hours, only 6 RadioSpinel-tagged lines appeared, all ChannelAccessFailure (a normal CSMA/CA “channel busy, backing off” report), and zero Spinel decode errors, CRC failures, timeouts, or “no response from RCP” — the interface between otbr-agent and the RCP firmware looks completely clean.

Baseline data collected

ot-ctl counters mac (captured after ~10.3 hours of RCP uptime):

TxTotal: 17194
    TxAckRequested: 12452
    TxAcked: 12418          <- only 34 frames (0.27%) never acked after retries
    TxErrCca: 1
    TxErrBusyChannel: 1
    TxDirectMaxRetryExpiry: 34
RxTotal: 66955
    RxDuplicated: 1501      <- consistent with occasional lost-ACK retries
    RxErrSec: 65            <- 0.1% of RxTotal
    RxErrFcs: 0

ot-ctl counters mle (same session):

Role Router:      37,203,365 ms  (essentially the entire tracked window)
Role Detached:              1  (281ms, once)
Partition Id Changes:       1  (the initial attach only)
Parent Changes:             0
Better Parent/Partition Attach Attempts: 0

So: a ~0.27% real Tx failure rate, negligible CCA/channel-busy events, negligible security failures, and a border router that has held its Router role essentially continuously with zero parent-hunting.

Where this leaves us

Given clean MAC counters, clean MLE stability counters, a clean RadioSpinel/USB transport layer, and no corroborating device-availability symptoms, the working conclusion is:

In an otherwise healthy mixed Apple/Home Assistant Thread network, MeshForwarder: Failed to send... error:NoAck appears frequently despite stable MLE counters, no parent changes, clean MAC error counters, and no user-visible issues. Is this expected — i.e., are these log lines describing routine first-attempt MAC retries that resolve on retransmission — or is there something about a mixed Apple-BR/OpenThread-BR topology specifically that would cause a higher background rate of these than a single-vendor mesh?

We suspect the answer is that this is simply routine 802.15.4 MAC-layer retry telemetry, made to look more alarming than it is by log verbosity, and that a heterogeneous mesh (multiple independent BRs, different vendor Thread stacks) naturally exercises more routing paths and produces more of this telemetry than a single-vendor setup — but we’d appreciate confirmation from anyone with visibility into the OpenThread/OTBR logging design, or anyone running a similarly mixed Apple + Home Assistant OTBR setup who’s compared notes on background NoAck rates.

Happy to share the full Loki query set or additional counter snapshots if useful.