Hi all. I am running HAOS as a VM on a Proxmox server. Everything runs really nicely except every few weeks HA crashes with this screen shot on the console.
The VM has 8GB ram so I don’t think this is a genuine “not enough memory to run HA”, but it might be a memory leak somewhere. Any ideas as to where to look next or what I can do to monitor things to get a lead on the cause ?
There isn’t much in the way of clues in the logs. Here is what the logs say around the time it failed -
2021-12-15 12:49:45 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Executing step delay 0:00:00
2021-12-15 12:49:45 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Test condition template: True
2021-12-15 12:49:45 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Choose at step 6: default: Running automation actions
2021-12-15 12:49:45 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Choose at step 6: default: Executing step call service
2021-12-15 13:00:25 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'action' when rendering '{{ value_json.action }}'
2021-12-15 13:15:57 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'click' when rendering '{{ value_json.click }}'
2021-12-15 13:15:57 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'action' when rendering '{{ value_json.action }}'
2021-12-15 13:26:39 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'action' when rendering '{{ value_json.action }}'
2021-12-15 13:40:31 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'action' when rendering '{{ value_json.action }}'
2021-12-15 14:06:10 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'click' when rendering '{{ value_json.click }}'
2021-12-15 14:06:10 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'action' when rendering '{{ value_json.action }}'
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/core_samba/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/7ad98f9c_zigbee2mqtt/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/a0d7b954_influxdb/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/core_mosquitto/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/core_configurator/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/a0d7b954_esphome/stats request
2021-12-15 14:07:38 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /addons/cebe7a76_hassio_google_drive_backup/stats request
2021-12-15 14:07:38 WARNING (MainThread) [homeassistant.components.hassio] Can't read Supervisor data:
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /ingress/validate_session request
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /ingress/validate_session -
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /ingress/validate_session request
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /ingress/validate_session -
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /ingress/validate_session request
2021-12-15 14:12:02 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /ingress/validate_session -
2021-12-15 14:12:03 INFO (MainThread) [hass_nabucasa.iot] Connection closed
2021-12-15 14:12:03 INFO (MainThread) [homeassistant.components.websocket_api.http.connection] [140412314325344] Connection closed by client
2021-12-15 14:12:03 INFO (MainThread) [homeassistant.components.websocket_api.http.connection] [140412352188320] Connection closed by client
2021-12-15 14:12:09 INFO (MainThread) [hass_nabucasa.iot] Connected
2021-12-15 14:12:27 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for binary_sensor.remote_ui (<class 'homeassistant.components.cloud.binary_sensor.CloudRemoteBinary'>) took 14.146 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+cloud%22
2021-12-15 14:13:11 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sun.sun (<class 'homeassistant.components.sun.Sun'>) took 0.550 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue
2021-12-15 14:13:32 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.kitchen_rear_door_energy_power (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 0.542 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:13:48 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.kitchen_rear_door_energy_today (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 0.918 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:14:04 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.kitchen_rear_door_energy_totalstarttime (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 1.191 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:14:13 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.kitchen_rear_door_energy_voltage (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 1.143 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:14:39 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.tasmota_6420_energy_total (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 0.569 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:14:46 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.tasmota_6420_energy_yesterday (<class 'homeassistant.components.mqtt.sensor.MqttSensor'>) took 0.482 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2021-12-15 14:14:58 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140412688882800] Unexpected error inside websocket API
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/http.py", line 210, in async_handle
msg = await wsock.receive()
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_ws.py", line 438, in receive
await self.pong(msg.data)
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_ws.py", line 293, in pong
await self._writer.pong(message)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 667, in pong
await self._send_frame(message, WSMsgType.PONG)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 650, in _send_frame
self._write(header + message)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 660, in _write
raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
2021-12-15 14:14:59 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 673, in ping
await self._send_frame(message, WSMsgType.PING)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 650, in _send_frame
self._write(header + message)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 660, in _write
raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
2021-12-15 14:14:59 ERROR (MainThread) [snitun.multiplexer.core] Ping fails, no response from peer
2021-12-15 14:14:59 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Restarting
2021-12-15 14:14:59 INFO (MainThread) [hass_nabucasa.iot] Connection closed
2021-12-15 14:14:59 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Running automation actions
2021-12-15 14:14:59 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Executing step call service
2021-12-15 14:15:00 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Test condition template: True
2021-12-15 14:15:00 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Executing step wait for trigger
2021-12-15 14:15:00 INFO (MainThread) [homeassistant.components.automation.new_automation] Cat Room Undercounter LEDs: Initialized trigger
2021-12-15 14:15:01 INFO (MainThread) [snitun.utils.aiohttp_client] AioHTTP snitun client connected to: eu-west-2-0.ui.nabu.casa:443
2021-12-15 14:15:09 INFO (MainThread) [hass_nabucasa.iot] Connected