Do you have a drawing you could share? It's not clear whether your interconnect is through a switch or p2p between each machine. The p2p connection would allow you to use CEPH as a federated fault tolerant mass storage. Do you use Proxmox HA for VM failover? It sounds like the answer is "no" since you describe splitting the HA VMs. It's unclear just what that means however. Are you running three concurrent/redundant HA instances?
I'm setting up Proxmox on multiple mini PCs and would like to make use of the fail over feature too. For now though I'm struggling to get remote access working using Pangolin, self hosted on a VPS.
Might be worth starting a topic for building systems using Proxmox?
I followed this video : https://www.youtube.com/watch?v=i0urJMFQ0yk
Hardware:
3 times:
- HP T640
- 64 Go ram
- SSD M2 128Go (internal)
- NVMe1To connected with USB 3.0 A or C (external)
- USB 3.0 A or C => 2.5 Gbps Lan adapter
All computer are connected on a manageable switch 2.5Gbps.
I used Ceph and follow the video attached. I vas not able to install it with proxmox 9. I'm reluctant to go thru as the initial process tooks me many attempt as I'm not IT guy.
If somebody can tell me how to upgrade to proxmox 9 as support of version 8 end in august.
1 Like
Thanks for the pointer. I wouldn't have thought of building a HAHA cluster using Proxmox running on Thin Clients. I have a pile of old but relatively high powered laptops the I was thinking of using as CEPH monitors and Proxmox HA last-resort hosts. Is your setup just for for running HA? It seems like the T640 should be able to handle this but it might max out pretty quickly if other applications got added beyond Home Assistant.
I wonder, too, about write endurance of the eMMC storage running CEPH.
I run a 3 node Proxmox cluster with Home Assistant in High Availability using ZFS replication. CEPH just seemed like too much for my needs. I'm running multiple VMs and LXCs using ZFS replication and it works great with an unmanaged 2.5GB switch for the replication traffic.