Home Assistant crashes every day

maurizio53 · February 13, 2023, 5:14pm

Almost every day my home assistant crashes and this is the log I can see:

2023-02-13 16:59:54.485 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hass_nabucasa/remote.py", line 395, in _reconnect_snitun
    await self._snitun.wait()
  File "/usr/local/lib/python3.10/site-packages/snitun/multiplexer/core.py", line 132, in _runner
    async with async_timeout.timeout(PEER_TCP_TIMEOUT):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 120, in __aenter__
    self._do_enter()
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 207, in _do_enter
    self._reschedule()
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 201, in _reschedule
    self._timeout_handler = self._loop.call_at(deadline, self._on_timeout, task)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 736, in call_at
    heapq.heappush(self._scheduled, timer)
RuntimeError: list changed size during iteration
2023-02-13 16:59:54.494 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/snitun/client/client_peer.py", line 135, in _handler
    await self._multiplexer.wait()
  File "/usr/local/lib/python3.10/site-packages/snitun/multiplexer/core.py", line 132, in _runner
    async with async_timeout.timeout(PEER_TCP_TIMEOUT):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 120, in __aenter__
    self._do_enter()
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 207, in _do_enter
    self._reschedule()
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 201, in _reschedule
    self._timeout_handler = self._loop.call_at(deadline, self._on_timeout, task)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 736, in call_at
    heapq.heappush(self._scheduled, timer)
RuntimeError: list changed size during iteration
2023-02-13 17:00:59.495 ERROR (MainThread) [root] Uncaught exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 226, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 214, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 128, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1869, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

Can somebody help me to find where is the problem?

tom_l · February 13, 2023, 6:08pm

Start by disabling your third party integrations. It is very likely one of them.

github.com/home-assistant/core

heapq.heappop: list changed size during iteration

opened 03:54PM - 11 Sep 20 UTC

closed 06:45AM - 16 Jun 21 UTC

chilicheech

stale

## The problem  After being up for a few days home assistant stopped responding. Upon looking at the docker logs I saw an error message and no more messages after that for 3 hours. ## Environment  <img width="573" alt="Screen Shot 2020-09-11 at 8 51 43 AM" src="https://user-images.githubusercontent.com/732659/92946911-24a80f00-f40c-11ea-87d4-3a407f565c2c.png"> ## Problem-relevant `configuration.yaml`  ```yaml ``` ## Traceback/Error logs  ```txt hass | 2020-09-11 13:45:59 ERROR (MainThread) [root] Uncaught exception hass | Traceback (most recent call last): hass | File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main hass | return _run_code(code, main_globals, None, hass | File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code hass | exec(code, run_globals) hass | File "/usr/src/homeassistant/homeassistant/__main__.py", line 320, in <module> hass | sys.exit(main()) hass | File "/usr/src/homeassistant/homeassistant/__main__.py", line 312, in main hass | exit_code = runner.run(runtime_conf) hass | File "/usr/src/homeassistant/homeassistant/runner.py", line 133, in run hass | return asyncio.run(setup_and_run_hass(runtime_config)) hass | File "/usr/local/lib/python3.8/asyncio/runners.py", line 43, in run hass | return loop.run_until_complete(main) hass | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete hass | self.run_forever() hass | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 570, in run_forever hass | self._run_once() hass | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1832, in _run_once hass | handle = heapq.heappop(self._scheduled) hass | RuntimeError: list changed size during iteration hass | 2020-09-11 13:45:59 ERROR (SyncWorker_62) [concurrent.futures] exception calling callback for <Future at 0x5ccaf808 state=finished returned list> hass | Traceback (most recent call last): hass | File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 328, in _invoke_callbacks hass | callback(self) hass | File "/usr/local/lib/python3.8/asyncio/futures.py", line 374, in _call_set_state hass | dest_loop.call_soon_threadsafe(_set_state, destination, source) hass | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 764, in call_soon_threadsafe hass | self._check_closed() hass | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed hass | raise RuntimeError('Event loop is closed') hass | RuntimeError: Event loop is closed ``` ## Additional information After restarting the docker container things worked properly. This has never happened before, that I'm aware of.

maurizio53 · February 13, 2023, 6:32pm

You mean custom components?
And how to know which one is causing this? I have many of them like, i think, most people here…

petro · February 13, 2023, 7:08pm

You have to disable them one by one and figure it out. The errors in the logs do not point to the troublesome custom component.

PecosKidd · February 13, 2023, 9:10pm

Or turn them all off and make sure you’re stable at that point, then turn one at a time back on until you experience the issue. Or turn half of them off and see if you’re stable, then half of those, etc… all as a means to narrow down to which component is causing the issue.

tom_l · February 14, 2023, 12:25am

that’s the best way. Half at a time will narrow it down quickly.

And while the folder is still called custom_components for backward compatibility they are called custom integrations now and has been for a very long time.