[Feature request] Allow the user to see the reason of crash/restart

jacoscar · May 29, 2021, 9:52am

Home Assistant is a wonderful platform, but it has started failing in the last 6 months or so, where it would just randomly crash and restart (which takes 20-40 minutes)
It would be useful if there was a way to identify the reason of the restart instead of just logging things like this:

(it finished restarting at 10:22 and it had been unreachable for at least the 20 minutes before that)

2021-05-29 10:01:24 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:01:24 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.ping_eolo is taking over 10 seconds
2021-05-29 10:01:50 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:01:50 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:02:35 WARNING (MainThread) [homeassistant.components.websocket_api.http.connection] [140416179852192] Disconnected: Did not receive auth message within 10 seconds
2021-05-29 10:02:35 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:02:35 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating workday binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:02:47 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:03:00 WARNING (MainThread) [homeassistant.components.websocket_api.http.connection] [140417517316944] Disconnected: Did not receive auth message within 10 seconds
2021-05-29 10:03:00 ERROR (MainThread) [homeassistant.components.mobile_app.notify] Timeout sending notification to https://mobile-apps.home-assistant.io/api/sendPushNotification
2021-05-29 10:03:06 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:03:23 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:03:29 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:03:29 ERROR (MainThread) [homeassistant.components.mobile_app.notify] Timeout sending notification to https://mobile-apps.home-assistant.io/api/sendPushNotification
2021-05-29 10:03:35 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 56, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 18, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 74, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 135, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 131, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/http.py", line 43, in get
    return await WebSocketHandler(request.app["hass"], request).async_handle()
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/http.py", line 141, in async_handle
    await wsock.prepare(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_ws.py", line 135, in prepare
    payload_writer = await super().prepare(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 378, in prepare
    return await self._start(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 386, in _start
 await self._write_headers()
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 458, in _write_headers
    await writer.write_headers(status_line, self._headers)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/http_writer.py", line 119, in write_headers
    self._write(buf)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/http_writer.py", line 67, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
2021-05-29 10:03:59 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:04:01 ERROR (MainThread) [homeassistant.components.mobile_app.notify] Timeout sending notification to https://mobile-apps.home-assistant.io/api/sendPushNotification
2021-05-29 10:04:05 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:04:12 WARNING (MainThread) [homeassistant.components.automation.input_amp_limit] input amp limit: Already running
2021-05-29 10:04:19 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:04:37 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:04:43 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:05:11 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:15:18 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=44 from 2021-05-28 17:39:46)
2021-05-29 10:15:31 WARNING (MainThread) [homeassistant.setup] Setup of utility_meter is taking over 10 seconds.
2021-05-29 10:16:57 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform filter is taking over 10 seconds.
2021-05-29 10:21:02 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:21:02 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating workday binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:21:02 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:21:02 WARNING (MainThread) [homeassistant.components.sensor] Updating tado sensor took longer than the scheduled update interval 0:00:30
2021-05-29 10:21:19 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:00:03
2021-05-29 10:21:20 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 56, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 18, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 74, in ban_middleware
return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 131, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/http.py", line 43, in get
    return await WebSocketHandler(request.app["hass"], request).async_handle()
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/http.py", line 141, in async_handle
    await wsock.prepare(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_ws.py", line 135, in prepare
    payload_writer = await super().prepare(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 378, in prepare
    return await self._start(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 386, in _start
    await self._write_headers()
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_response.py", line 458, in _write_headers
    await writer.write_headers(status_line, self._headers)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/http_writer.py", line 119, in write_headers
    self._write(buf)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/http_writer.py", line 67, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

petro · May 29, 2021, 10:36am

You can’t capture out of memory crashes… because you’re out of memory.

jacoscar · May 29, 2021, 10:50am

I’m fuc**d then

jacoscar · May 29, 2021, 10:55am

What about other random errors like this one?

Logger: homeassistant.setup
Source: setup.py:310
First occurred: 11:39:04 AM (10 occurrences)
Last logged: 11:39:04 AM

Unable to prepare setup for platform utility_meter.sensor: Unable to set up component.

nickrout · May 30, 2021, 4:51am

But you can add more memory.

jacoscar · May 30, 2021, 6:22am

I have 6 GB right now; I would have thought that was enough

nickrout · May 30, 2021, 6:52am

Allocated to that VM?

jacoscar · May 30, 2021, 7:08am

Yes, the laptop has 16GB, 6GB allocated to that VM

nickrout · May 30, 2021, 7:15am

Seems weird then that you are getting an OOM.

nickrout · May 30, 2021, 7:16am

Disable all custom components and third party addons and see if that helps. If it does, add them back one at a time.

jacoscar · May 30, 2021, 8:30am

I have been trying this for 8 months
I don’t have any custom component; adding one component or one integration or entity at the time doesn’t help because of the number of entities I have to add, the mean time to wait for a crash and the average lifespan of a human being

I once had an error relative to utility meter, the error message was saying ‘please open a bug on github’, I did so and then I was told that there must have been something else that was wrong with my configuration