Haos randomly freezing in VirtualBox

Configuration
1

Hi all,

I’ve been running HAOS in VirtualBox (under Linux) for some time now. Lately, it has been randomly freezing, I’m unable to even stop it (it’s stuck at stopping, so I have to kill the VBOX process to restart it).

If I open the window (I run it in headless mode), I can see the default terminal - but the cursor is not not even blinking (totally frozen).

Some info:
Ubuntu Linux, virtualbox 7.0.16-dfsg-2ubuntu1, kernel 6.8.0-48-generic.
VM has 4 vCPU 6 gigs of RAM (all ARE available, no overprovisioning)
HASS: 13.2, homeassistant 2024.10.4

Virtualbox logs:
After normal boot messages, periodically this pops out (but the machine still works):
60:30:12.602484 TM: Giving up catch-up attempt at a 60 001 585 561 ns lag; new total: 134 881 914 270 588 ns

But when it’s frozen, I can see this:

60:44:42.234176 VMMDev: vmmDevHeartbeatFlatlinedTimer: Guest seems to be unresponsive. Last heartbeat received 4 seconds ago
60:44:48.247691 VMMDev: GuestHeartBeat: Guest is alive (gone 10 013 541 794 ns)
(same messages repeating)

At this point, the machine is frozen.

Unfortunately, I can’t find anything older than last boot in the UI under logs. :-/

Any ideas what might be happening / what could I do?

Thanks

2

That is a message about time skew.
Most often this will just result in automations being up to a few minutes off.
HA will periodically set the correct time, but only like 2-4 times per day, so with a high time skew you can still be off by minutes.
You can install the Chrony addon to get a constant time syncing, which should eliminate those messages.

Regarding the hangs, then look in the homeassistant.log.1 file in the config library.
It is the log from the previous run.

3

Same issue here. Brand new install. Same configs mentioned.

4

It’s taken me a bit because I rebooted it twice after the last freeze (system update). Today, it was frozen, again (even a cursor wouldn’t move in terminal!). Curiously, virtualbox process was at MAX CPU (400%, it has 4 vCPUs).
After rebooting, I can see hundreds of thousand errors due to “Dahua” integration (camera) for one specific camera which is currently disconnected due to a faulty cable.
I removed that device and it will see if it was that!

Errors:

2024-11-20 19:29:55.338 ERROR (MainThread) [custom_components.dahua] Failed to initialize device at 192.168.99.90
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1109, in _wrap_create_connection
    sock = await aiohappyeyeballs.start_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 104, in start_connection
    raise first_exception
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 82, in start_connection
    sock = await _connect_sock(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 174, in _connect_sock
    await loop.sock_connect(sock, address)
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 641, in sock_connect
    return await fut
           ^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 681, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
OSError: [Errno 113] Connect call failed ('192.168.99.90', 80)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/dahua/__init__.py", line 203, in _async_update_data
    self._max_streams = await self.client.get_max_extra_streams() + 1
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dahua/client.py", line 141, in get_max_extra_streams
    result = await self.get("/cgi-bin/magicBox.cgi?action=getProductDefinition&name=MaxExtraStream")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dahua/client.py", line 792, in get
    raise exception
  File "/config/custom_components/dahua/client.py", line 774, in get
    response = await auth.request("GET", url)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dahua/digest.py", line 46, in request
    response = await self.session.request(method, url, headers=headers, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 663, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 538, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1050, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1384, in _create_direct_connection
    raise last_exc
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1353, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1124, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 192.168.99.90:80 ssl:default [Connect call failed ('192.168.99.90', 80)]
2024-11-20 19:29:55.340 ERROR (MainThread) [custom_components.dahua] Unexpected error fetching dahua data
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1109, in _wrap_create_connection
    sock = await aiohappyeyeballs.start_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 104, in start_connection
    raise first_exception
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 82, in start_connection
    sock = await _connect_sock(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohappyeyeballs/impl.py", line 174, in _connect_sock
    await loop.sock_connect(sock, address)
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 641, in sock_connect
    return await fut
           ^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 681, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
OSError: [Errno 113] Connect call failed ('192.168.99.90', 80)