HAOS VM keeps crashing (out of memory)

My HAOS install, running in a VM on Truenas Scale, crashes approximately every 24 hours. The Core or Supervisor logs don’t mention anything. The VM logs say very little:

2026-01-07 19:18:37.822+0000: shutting down, reason=crashed

The Truenas Scale system logs (cat /var/log/syslog | grep -I vm) mention an out of memory error.

Jan  7 20:18:37 truenas kernel: [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
Jan  7 20:18:37 truenas kernel: Out of memory: Killed process 630484 (qemu-system-x86) total-vm:7183924kB, anon-rss:2129816kB, file-rss:304kB, shmem-rss:0kB, UID:986 pgtables:7960kB oom_score_adj:0

I have increased RAM from 2GB to 4GB (see screenshot of settings below) but this keeps happening.

Could data be building up somewhere causing this crash? Would you recommend a completly new install from backup or is there some settings somewhere that I could check first?

Here are the settings I’m using for this VM:

From time to time an integration can cause a memory leak.
First you should restart in safe mode, which will disable all third party integrations.
If the system is stable there, then you enable half of the integrations that is disabled.
If that works then you enable the next half and so on.
When it fails then you know in what group it is and then you half that again and so on.

I think there is a detailed guide in the community guide section for this.

1 Like

Please share that OOM log.

Sure, see original post

qemu-system-x86

This is an OOM on the TrueNAS side. It’s not HAOS running out of memory but your “hypervisor”.

On TrueNAS check

arc_summary -s arc
top -co %MEM
free -h

On HAOS use the glances addon.

Thanks, turns out middlewared is hogging several GiB of memory. Restarted it and will follow up on it for a while…