You can not use memory numbers on VMs from Proxmox.
You need to get them from inside the VM, like from inside HA.
That’s exactly what I did.
Proxmox can not see when memory are released again in a VM, so you get a picture of a huge memory usage, when there in fact is none.
EDIT: You did it correctly then
Edit2: It looks like you are pulling the data to your graphs from Docker still. Even though you are inside HA, then it is still docker data and they can not be used. You need to use HAs own.
By default, the logs available via the Home Assistant GUI only show activity since the last reboot. So you can’t see why the crash happened.
When you restart Home Assistant, the current homeassistant.log
file is renamed to homeassistant.log.1
, and a new homeassistant.log
file is created for new log entries. So, look at the last few entries in homeassistant.log.1 for a clue.
I have the same problem. I’ve never been able to do a successful backup since the new system was released. Me too on Proxmox. My homeassistant.log.1 ends like this:
2025-02-19 10:07:53.647 INFO (MainThread) [homeassistant.components.recorder.backup] Backup start notification, locking database for writes
2025-02-19 10:08:36.248 WARNING (MainThread) [homeassistant.components.automation.livello_lux_basso] Lux Esterno Basso: Already running
2025-02-19 10:08:36.249 WARNING (MainThread) [homeassistant.components.automation.livello_lux_basso] Lux Esterno Basso: Already running
And then a row of “NUL” characters.
homeassistan.log starts like this:
2025-02-19 10:10:18.449 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration nodered which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.450 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration xiaomi_gateway3 which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.451 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration localtuya which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.453 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.455 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration frigate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.456 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration scheduler which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.457 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration meteoswiss which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:18.459 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration device_tools which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2025-02-19 10:10:19.924 INFO (MainThread) [homeassistant.setup] Setup of domain logger took 0.17 seconds
2025-02-19 10:10:19.924 INFO (MainThread) [homeassistant.setup] Setup of domain system_log took 0.17 seconds
2025-02-19 10:10:19.924 INFO (MainThread) [homeassistant.bootstrap] Setting up frontend: {'backup', 'hassio', 'frontend'}
2025-02-19 10:10:19.925 INFO (MainThread) [homeassistant.setup] Setting up http
2025-02-19 10:10:19.925 INFO (MainThread) [homeassistant.setup] Setting up device_automation
2025-02-19 10:10:19.925 INFO (MainThread) [homeassistant.setup] Setup of domain device_automation took 0.00 seconds
I can reproduce this easily by running any kind of backup (I tried all combos).
Any Home Assistant backup starts with a local backup. So, of course all methods will act the same.
Do I recall that you said earlier that “MQTT still works”?
Good hint.
I just ran around in circles looking for a similar issue. MY Home Assistant automations stopped working, but Node Red was still running.
If this happens again, go to the console terminal on your host computer or SSH and enter:
ha core restart
.
If this gets you going again, look at the last line in the log.1 file. This is what your Home Assistant was doing when core stopped.
In my case, the problem was an error in one of my Node Red function nodes. But it’s only been 36 hours, so the jury is still out.
Wasn’t me… Not sure what MQTT has to do with this. My entire HA restarts immediately upon crash, automatically.
This is just throwing a band-aid on the problem. Not fixing it at all.
Is not even a band-aid… it does so automatically and it annoys me quite a bit as every time it resets many states.
I confused you for the OP.
Not sure how trustworthy the 24MB/s disk io number is, but it is at least 10x too slow for an SSD if it’s true. Something is wrong with your virtual disk setup for these backups to work. Have you ever taken a disk snapshot in proxmox?
vmstat
numbers look good for an idle system. Need to gather them for when the backup is running.
I just updated HA Core and had it make a backup.
While doing that, I ran iotop and iostat on the Proxmox host.
In iotop I saw a max of 180 MB/s and in iostat a max of 72 MB/s (yet most of the time at 25-30 MB/s). I did not see any iowait above 1%.
Then I triggered a manual general Home Assistant backup based on what’s configured in HA to run daily around 5:00 in the morning and which is the thing that usually kiils my HA VM. This job backups to my NAS (a VM on another server) and to NabuCasa cloud.
There I saw high iowait numbers.
Here are a few screenshots (iotop on top and iostat below).
No. Why do you ask ?
I had an issue (not on proxmox) where the snapshot would be qcow2 and it would be very slow, but if you never did it, obviously nothing to worry about.
No idea how exactly your NAS is set up, but if you have it mounted as a directory, it might be the reason. Try using any other way of moving the file, scp, sftp, rsync, whatever.
I have what may be a similar issue. HA running in a VMware Fusion VM on Apple Silicon. When I woke up this morning, HA was down. Vmware said the VM was stopped, and after starting it, HA was fine. From the logs, this all happened around the time an automatic backup could have been taken. My backups are purely to local storage, nothing fancy. They get copied off to external storage by other means, using SMB. But one wasn’t taken this morning at all. I ran a backup manually after HA restart, using the automatic settings, and it worked fine.
The Log 1 file shows absolutely nothing. Completely normal activity, and then suddenly nothing.
The VM has 4GB RAM allocated. Mac Mini has 16GB in total.
All a bit worrying!
EDIT: I just found that my Mac kernel panicked at this time. Not sure why yet. What fun!
I had the same problem, HAOS crashing when an automatic or manual backup was triggered.
My HAOS is running as a VM in Proxmox. The host has 12gb of RAM.
Before the problem started to occur, I increased the RAM of the HAOS instance from 6GB to 8GB.
Then the crashes started to happen and I could not start a manual backup.
Now I have decreased the RAM again and I can do backups again.
I have no idea why, maybe I took too much RAM from the host or something.
Maybe this helps somebody.
Oh my goodness… it worked! I lower from 12 to 8Gib and it backs up fine! Takes forever but it works.
Thank you!