@sol Take all this with a pinch of salt, but here are some of my experiences/opinions
I’ve been messing around with home networking for over 25 years now, and as such, my network has evolved over time. Back in the day, when the average household had less than 5 things on the internet, the router supplied by your broadband provider was more than capable of handling the traffic, with the network complexity and interactions were relatively low.
The issue I see these days is that the same broadband companies are supplying the same (very cheap) routers to households which unlikleynow have dozens of devices online. The complexity of these networks is going up, but, because of mass adoption, suppliers have to make the systems simpler to use, which usually means making them more intelligent…, however, this causes more problems due to highly complex interactions between all these smart systems. The result being that these networks are ultimately much harder to manage long term.
Clearly you have a complex network, and more that enough technical know how to setup all the devices you have connected. Its just time to give your infrastructure some love 
In my experience, that way to do this is to simplify how your network is managed, and use appropriate kit.
When does a home network turn into an enterprise network ?, Scale and complexity. From the look of your setup this is definitely at the very top end of home / entry level enterprise. e.g. Using a simple ‘deigned for home’ router at your scale will work, but not necessarily give you the performance and control that you need to get the best out of the network. I know the Fritz!Box you are using is by no means ‘basic’, but it still lacks enterprise type features for managing your network.
There are a number of open source ‘Universal Threat Management (UTM)’ systems which can be used to replace your router with something more flexible. I have used a bunch of these over the years, but for the last 10 years have been using Untangle. This is free, as are many other OS UTM distributions. I run this on a dedicated fanless Celeron based PC ($120), and its basically a full enterprise grade UTM you can use at home. As I said before, something like this is not necessary…, its just the next level.
For the network past the router, I recently moved from to a full SDN solution. This was because is was becoming too cumbersome to manage 4 access points and 3 switches all separately. Moving to full SDN means that I now manage the entire network from one interface, again, not necessary…, just next level.
With these 2 management interfaces (Untangle & Omada SDN), I can see in detail everything that is going on with my network, which then allows me to make changes, and compare the performance with hard metrics.
The issue I see in the future is that systems like mine are out of the reach of (probably) 99% of home users. From my point of view, my network is now simpler and more performant than ever…, but I’ve been doing this for 25 years. As the average home network complexity increases I think either people will need to become much more educated about how all this stuff works (unlikely…), or home networks will have to become ‘professionally’ managed in order to maintain the performance we all now expect.
