It is done.
My 5 x Home Assistant node installation is, after much unexpected delay, is finally complete. This realises a goal I started envisioning more than 2 years ago right here on this forum. To realise the high availability / fail-over functional ability commonly seen in the GSM / Telecommunications provider sphere and / or like was originally embraced in the IBM System/4 Pi computer used in aviation and on the Space Shuttle for flight control systems.
This evolution of design came from me looking at commercial over the counter aquarium controllers, and comparing their functionality and design principles in relation to their costs. I hated everything I saw - and kept thinking “If I designed one of the GSM networks I used to work on, this way… I’d be fired.” Worse was the price tags associated with these commercial controllers, particularly when you looked at some of high market solutions and their associated accessories often referred to as “Breakout Boxes” or “Remote Probe boxes”. One vendor in particular only uses SPI communications to a remote break out box, which only contains components that if I built it, would cost me maybe 10-20 euros. But the branded commercial product cost over over 250 euros at the time I priced it. Worse, I saw the repurposing of USB type connectors as SPI cables potential future Human Failure errors just waiting to happen. (“Hey, can I charge my phone here?” ZAP)
Besides, I was already using an original RaspberryPi Model 1B since 2012 as my aquarium light controller. It ran my own code for controlling 10 x Neol branded Ethernet enabled PowerSwitch4 strips & 3 x APC MasterSwitch 8-socket power bars, in use around the house and on all my aquariums. And in May 2017, I thought combined with a RaspberryPi 3B, that RaspberryPi Model 1 would make a far better “Remote Probe Box” than anything I could by commercially. And, looking into the architecture of Home Assistant at the time, I realised I could create my own High Availability architecture using a network of RaspberryPi’s.
Most importantly, I turned the model of the commercial “dumb” Remote Breakout boxes on it’s head: Each place where I needed remote probes and sensors, would become it’s own redundant instance of Home Assistant. Dumb breakout boxes would become very Intelligent breakout boxes in my design. Further, having Ethernet enabled Powersockets vs. those attached directly to a Controller (like in commercial products) let me abstract Powerswitching out and away from the Pi-based controllers I made, allowing any remaining Pi standing in a failure situation, able to take over power switching duties, be it for heaters, water pumps, chemistry doses, mixers & stirrers, lights, etc.
I even could (and did) set it up that the RPI’s could reboot each other, attempting to get the failed node back online, if possible and necessary, without my presence or intervention required.
In a way, albeit a bit primitive, I was creating my very own locally installed HA Cloud. And it was still far more feature rich and reliable than any commercial controller I could find at the time.
And, just a couple of weeks after bringing the 3rd Pi (2x RPI3B and 1x RPI1B) online, a submerged aquarium heater suffered a leak failure causing my GFCI to trip after my wife and I had gone to bed, two floors upstairs from the aquariums. We never heard a thing, but the RPI on the 1rst floor could determine that it still could see the newly brought online RPI downstairs, but couldn’t see the RPI or Powerbars in the fish room, and sent out a message informing the GFCI / Circuitbreaker had probably tripped. And it saved the life of over 200 baby clownfish just a couple of months old. In the last 2 years, there have been about 6 different events similar to this, occurring, which HA has saved the day and my fishes lives.
This convinced me, and most importantly - my wife, of the value of what I’d sat out to achieve: High Availability and a design principle that eschews “Safe Mode of Failure, by Design”. I believe this itself is an expansion of the original design principle which Paulus Schoutsen set out to realise - your home automation should not stop working just because the Cloud or Internet Link stops working. In my case, I believe your automation should not stop working just because Home Assistant stops working.
Even if it cannot recover from the failure itself, as in the case of a mains power failure, it can still let me know about it. And further, remaining Home Assistant installs still running can take over, for what their able to see and control.
Eventually, performance requirements would dictate that I broke the Database (postgresql) to it’s own dedicated RPI3. And after a year of that, I migrated the core HA on an RPI and it’s dedicated DB server RPI both to dedicated VM’s on a NUC I picked up last year. And I migrated the RPI3 that serves as a GPS NTP server in the attic to an HA node, and deployed a RPI HA node on the first floor as well.
But the original Model 1B in the fish room always remained a “dumb GPIO slave” relative to all the other HA installed nodes through the rest of the house. This, despite the fact, it is the most physically complex RPI in the house.
Until yesterday. Yesterday I spent the day swapping it out & debugging a few glitches in the pre-configured drop-in replacement, which contained a pre-customized installation of HA, which I’d worked on in previous days. And at 5:34pm yesterday, my 5-HA node dream of a system came online. And thanks to @mcclown who provided the code to make Seneye work on RPI’s - now I can finally plug my Seneyes straight into an RPI in my fish room - without a Seneye SWS or a Windows computer running Seneye SCA application.
And this new RPI can take over heating control and critical functions, in the event the fish room were to ever find itself in the strange and unique position that it was the only mains circuit with 220V power, while the rest of the house was dark. Which is exactly what is going to happen in the coming months when they come to replace our mains power electrical box. 
All of this high availably and safe failure design principles price tag came at less than the cost of two of those “Breakout Boxes” ( SPI bridges ) available from expensive commercial aquarium controller companies. Minus my own labor writing the code & building the hardware around it all - but that’s the fun part. 
Many thanks to everyone, from Paulus Schoutsen on downwards, who’s contributed to Home Assistant in some way, making Home Assistant the most awesome home and aquarium automation products out there. I bet I could use it to launch missiles & control power planets reliably, even. I’m even asking myself “What new problems can I solve with HA?”
Epic Home Assistant is truly epic.
