Since I've started the local OTBR it's working stable. So it has something to do with communication to the Apple BR.
Unfortunately with local OTBR it's no good solution because OTBR and Apple BR leads to really bad performance when running both in parallel.
I know this might be a non-answer, but I've noticed a HUGE difference in performance with my 102 devices since moving to the Matter.js beta. I know it's a beta, but it's been a gamechanger in how reliable it is compared to the python Matter server.
I have 6 Apple TBRs and 102 Matter/Thread devices and I would have outages of devices all the time. After moving to the beta, I have no device drops for days or weeks. And when I have drops, it's for maybe 1 or 2 seconds and that's it.
It's worth trying the beta if you are still on the Python server. There have been many iterations of the matter.js server so far, so it's not like you are running something brand new.
I already use the Matter Beta. But the problem happens with beta and with the official version.
The problem is that Home Assistant can‘t reach the ipV6 network subsegment of Thread anymore. And it started with any update. So it seems to be a bug.
But I also seen that the beta runs in principle better.
Just some thoughts ... Its really up to the Apple HomePod Mini to issue Router Advertisements containing the routes to the Thread network. These routes will have certain lifetimes to them. Similarly, the Mini has to issue new routes if the Thread mesh routeable prefixes change, etc. So perhaps check out the route lifetimes seen by HA to see if they are the cause.
The only thing that comes to mind here, is that HA may be issuing a Router Solicitation after it restarts or update command is issued. This would request the Mini to send out a Router Advertisement which would contain the routes to the mesh.