Backup server / round robin load balancing

OK I got it.
The ucarp itself is distributed.
So you’d have:
Host1 --> Docker --> ucarp
and
Host2 --> Docker --> ucarp

Exactly, and the 2 ucarp are accessible via the same IP, then redirecting the request to one or the other HASS instance.

I think you may be overcomplicating things. What you need is a mirror of your services on the pi (or at least a subset of them) and automatic failover/recovery with no changes on the clients. One way is to move the ip between the two hosts. Let’s say you use ip 1 now. Configure your main home automation with ip 2, the pi with ip 3, then configure keepalived to float the ip 1 for both servers: if 2 is up it also gets ip 1 and if 1 is down 2 gets the ip 1. Keepalived can run a script on selecting a host for an ip so you need to disable auto start on on boot for service and add a script to start them when the ip is floated to the host and stop them when ip is removed.

This would mean a small few seconds of downtime when shutting down main host, but you won’t have to worry about services/automations running twice.

I use the same system for 2 servers main/failover and a pi as a second failover for when the power goes out for more than 3 hours - running mqtt and a minimal list of services for critical systems…

Yes that’s basically what I want to accomplish, I’ll start the same docker container with my hass, mqtt, … server on both server.
Only the main one would be accessible with the hass ip, and if that one fail, the pi would take over under the same ip.

When you mean keepalived, is that a tool? Or a script you created?

Keepalived is an open source piece of software. It’s used in many places and it might be a bit hard to set up (the configuration is not really too user-friendly) but once setup it just works.

Examples:

https://raymii.org/s/tutorials/Keepalived-Simple-IP-failover-on-Ubuntu.html

https://www.digitalocean.com/community/tutorials/how-to-set-up-highly-available-web-servers-with-keepalived-and-floating-ips-on-ubuntu-14-04 (here you’d need to replace nginx with another process)

and here is an example on starting / stopping services on failover

https://raymii.org/s/tutorials/Keepalived_notify_script_execute_action_on_failover.html

Forgot to mention it can monitor your home assistant process (or mqtt, or both), and failover if one the services fail.

You can find countless examples with your favorite search engine.

keepalived is functionally about the same as ucarp. One advantage of keepalived would be the ability to execute as script or whatever when the failover happens.