Hello,
I’ve got a problem with my home assistant installation but don’t know how the figure out what’s wrong.
From time to time (every few days) my home assistant instance just becomes unresponsive / hangs up. It is completly unaccessible then and doesn’t do anything anymore. It also doesn’t record any data anymore, which I verified using the system monitor integration to collect some data points that are not reliying on the network connection.
My home assistant server is running on my TrueNAS core server. I’ve set it up as a virtual machine running HAOS. Logging into the terminal of that virtual machine didn’t show anything suspicious to me, but I am also not very familiar with the console and which information it can give me. Restarting the virtual machine brings home-assistant back up as usual until it locks up again after usually a few days of running.
Last time it crashed was between 2024-11-04 14:23 and 2024-11-04 14:30 (CEST) or 13:23 and 13:30 UTC. I pressed the restart button after noticing at 14:38 CEST.
I got these times as my instance is monitored by healthchecks.
These are some logs I got executing: ha host logs --boot -1 --lines 500
2024-11-04 13:02:45.113 homeassistant kernel: audit: type=1334 audit(1730725365.111:747): prog-id=303 op=LOAD
2024-11-04 13:02:45.113 homeassistant kernel: audit: type=1334 audit(1730725365.111:748): prog-id=304 op=LOAD
2024-11-04 13:02:45.113 homeassistant kernel: audit: type=1334 audit(1730725365.111:749): prog-id=305 op=LOAD
2024-11-04 13:02:45.121 homeassistant systemd[1]: Starting Hostname Service...
2024-11-04 13:02:45.299 homeassistant systemd[1]: Started Hostname Service.
2024-11-04 13:02:45.308 homeassistant kernel: audit: type=1334 audit(1730725365.306:750): prog-id=306 op=LOAD
2024-11-04 13:02:45.308 homeassistant kernel: audit: type=1334 audit(1730725365.306:751): prog-id=307 op=LOAD
2024-11-04 13:02:45.308 homeassistant kernel: audit: type=1334 audit(1730725365.306:752): prog-id=308 op=LOAD
2024-11-04 13:02:45.317 homeassistant systemd[1]: Starting Time & Date Service...
2024-11-04 13:02:45.476 homeassistant systemd[1]: Started Time & Date Service.
2024-11-04 13:03:15.339 homeassistant systemd[1]: systemd-hostnamed.service: Deactivated successfully.
2024-11-04 13:03:15.369 homeassistant kernel: audit: type=1334 audit(1730725395.368:753): prog-id=305 op=UNLOAD
2024-11-04 13:03:15.369 homeassistant kernel: audit: type=1334 audit(1730725395.368:754): prog-id=304 op=UNLOAD
2024-11-04 13:03:15.369 homeassistant kernel: audit: type=1334 audit(1730725395.368:755): prog-id=303 op=UNLOAD
2024-11-04 13:03:15.509 homeassistant systemd[1]: systemd-timedated.service: Deactivated successfully.
2024-11-04 13:03:15.515 homeassistant kernel: audit: type=1334 audit(1730725395.514:756): prog-id=308 op=UNLOAD
2024-11-04 13:03:15.515 homeassistant kernel: audit: type=1334 audit(1730725395.514:757): prog-id=307 op=UNLOAD
2024-11-04 13:03:15.515 homeassistant kernel: audit: type=1334 audit(1730725395.514:758): prog-id=306 op=UNLOAD
2024-11-04 13:26:52.187 homeassistant kernel: audit: type=1701 audit(1730726812.182:759): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=unconfined pid=383791 comm="python3" exe="/usr/local/bin/python3.12" sig=11 res=1
2024-11-04 13:26:52.191 homeassistant systemd[1]: Created slice Slice /system/systemd-coredump.
2024-11-04 13:26:52.194 homeassistant kernel: audit: type=1334 audit(1730726812.191:760): prog-id=309 op=LOAD
2024-11-04 13:26:52.194 homeassistant kernel: audit: type=1334 audit(1730726812.192:761): prog-id=310 op=LOAD
2024-11-04 13:26:52.194 homeassistant kernel: audit: type=1334 audit(1730726812.192:762): prog-id=311 op=LOAD
2024-11-04 13:26:52.199 homeassistant systemd[1]: Started Process Core Dump (PID 417786/UID 0).
2024-11-04 13:26:52.376 homeassistant systemd-coredump[417787]: Process 383791 (python3) of user 0 terminated abnormally without generating a coredump.
2024-11-04 13:26:52.378 homeassistant systemd[1]: [email protected]: Deactivated successfully.
2024-11-04 13:26:52.421 homeassistant kernel: audit: type=1334 audit(1730726812.419:763): prog-id=311 op=UNLOAD
2024-11-04 13:26:52.421 homeassistant kernel: audit: type=1334 audit(1730726812.419:764): prog-id=310 op=UNLOAD
2024-11-04 13:26:52.421 homeassistant kernel: audit: type=1334 audit(1730726812.419:765): prog-id=309 op=UNLOAD
2024-11-04 13:26:55.502 homeassistant systemd[1]: docker-6ab9353893ec096dbb76581c7557238575a24611e4599111ed375310296fdce5.scope: Deactivated successfully.
2024-11-04 13:26:55.502 homeassistant systemd[1]: docker-6ab9353893ec096dbb76581c7557238575a24611e4599111ed375310296fdce5.scope: Consumed 30min 36.137s CPU time.
2024-11-04 13:26:55.506 homeassistant kernel: audit: type=1334 audit(1730726815.504:766): prog-id=290 op=UNLOAD
2024-11-04 13:26:55.513 homeassistant dockerd[534]: time="2024-11-04T13:26:55.513907217Z" level=info msg="ignoring event" container=6ab9353893ec096dbb76581c7557238575a24611e4599111ed375310296fdce5 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
2024-11-04 13:26:55.544 homeassistant systemd[1]: var-lib-docker-overlay2-e0e4d68c1dd43fc4cb0cb2506bbb506c807be481c557b9f08f5d990a983cd267-merged.mount: Deactivated successfully.
2024-11-04 13:26:55.544 homeassistant systemd[1]: mnt-data-docker-overlay2-e0e4d68c1dd43fc4cb0cb2506bbb506c807be481c557b9f08f5d990a983cd267-merged.mount: Deactivated successfully.
2024-11-04 13:37:45.531 homeassistant systemd-logind[462]: Power key pressed short.
2024-11-04 13:37:45.532 homeassistant systemd-logind[462]: Powering off...
2024-11-04 13:37:45.534 homeassistant systemd-logind[462]: System is powering down.
These are the last logs from home-assistant.log.1, but I think they don’t give any valuable information for my specific problem:
2024-11-04 13:45:10.209 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.speedtest_download is taking over 10 seconds
2024-11-04 14:00:01.185 ERROR (MainThread) [homeassistant.components.speedtestdotnet.coordinator] Error fetching speedtestdotnet data: Unable to connect to servers to test latency.
2024-11-04 14:15:10.209 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.speedtest_download is taking over 10 seconds
home-assistant.log.fault was written at 2024-11-04 13:26:52 GMT, so right around when the crash supposedly happened. Unfortunately I am not really capable of reading that file.
Edit: Link to the contents of that file because its too long for this post:
https://pastecode.io/s/qimegnsv
Does anybody have an idea whats wrong with my home-assistant server, or how I could get some more information about the problem?