I have rather regular crashed (roughly weekly) of my instance of Home Assistant.
It has been the case for a few months now. It occurs on all releases 0.1xx and and all type of setup: Generic Install on Linux (debian), hassos, and now Home Assistant Core on Docker.
In every case the scenario looks similar in the log:
2020-06-11 17:10:45 ERROR (MainThread) [homeassistant.components.rest.switch] Timed out while fetching data
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 210, in get_device_state
self._resource, auth=self._auth, headers=self._headers
File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 483, in _request
timeout=real_timeout
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 859, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 986, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 936, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa
File "/usr/local/lib/python3.7/asyncio/base_events.py", line 949, in create_connection
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 473, in sock_connect
return await fut
concurrent.futures._base.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 198, in async_update
await self.get_device_state(self.hass)
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 212, in get_device_state
text = await req.text()
File "/usr/local/lib/python3.7/site-packages/async_timeout/__init__.py", line 45, in __exit__
self._do_exit(exc_type)
File "/usr/local/lib/python3.7/site-packages/async_timeout/__init__.py", line 92, in _do_exit
raise asyncio.TimeoutError
concurrent.futures._base.TimeoutError
2020-06-11 17:10:45 ERROR (MainThread) [homeassistant.components.rest.switch] Timed out while fetching data
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 210, in get_device_state
self._resource, auth=self._auth, headers=self._headers
File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 483, in _request
timeout=real_timeout
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 859, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 986, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 936, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa
File "/usr/local/lib/python3.7/asyncio/base_events.py", line 949, in create_connection
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 473, in sock_connect
return await fut
concurrent.futures._base.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 198, in async_update
await self.get_device_state(self.hass)
File "/usr/src/homeassistant/homeassistant/components/rest/switch.py", line 212, in get_device_state
text = await req.text()
File "/usr/local/lib/python3.7/site-packages/async_timeout/__init__.py", line 45, in __exit__
self._do_exit(exc_type)
File "/usr/local/lib/python3.7/site-packages/async_timeout/__init__.py", line 92, in _do_exit
raise asyncio.TimeoutError
concurrent.futures._base.TimeoutError
2020-06-11 17:10:46 ERROR (MainThread) [homeassistant.helpers.entity] Update for camera.exterieur fails
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.7/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 167, in _new_conn
% (self.host, self.timeout),
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7fa573e46910>, 'Connection to 10.1.0.90 timed out. (connect timeout=5)')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.1.0.90', port=5000): Max retries exceeded with url: /webapi/entry.cgi?_sid=OuGs8iQHiUhoTHGwi1SbeOPhjxwVZii8wLFWa-EuIgowVmsY4uIxkMZ_MugUg48XYQR_SQ8vVDj_dLjjUVmwDo&api=SYNO.SurveillanceStation.Camera&method=List&version=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fa573e46910>, 'Connection to 10.1.0.90 timed out. (connect timeout=5)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 472, in async_device_update
await self.hass.async_add_executor_job(self.update)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/synology/camera.py", line 118, in update
self._surveillance.update()
File "/usr/local/lib/python3.7/site-packages/synology/surveillance_station.py", line 20, in update
cameras = self._api.camera_list()
File "/usr/local/lib/python3.7/site-packages/synology/api.py", line 101, in camera_list
response = self._get_json_with_retry(api['url'], payload)
File "/usr/local/lib/python3.7/site-packages/synology/api.py", line 192, in _get_json_with_retry
return self._get_json(url, payload)
File "/usr/local/lib/python3.7/site-packages/synology/api.py", line 198, in _get_json
response = self._get(url, payload)
File "/usr/local/lib/python3.7/site-packages/synology/api.py", line 185, in _get
verify=self._verify_ssl)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 504, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='10.1.0.90', port=5000): Max retries exceeded with url: /webapi/entry.cgi?_sid=OuGs8iQHiUhoTHGwi1SbeOPhjxwVZii8wLFWa-EuIgowVmsY4uIxkMZ_MugUg48XYQR_SQ8vVDj_dLjjUVmwDo&api=SYNO.SurveillanceStation.Camera&method=List&version=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fa573e46910>, 'Connection to 10.1.0.90 timed out. (connect timeout=5)'))
2020-06-11 17:10:52 INFO (SyncWorker_4) [googleapiclient.discovery] URL being requested: GET https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.15, YeelightPlacard: Bulb closed the connection.
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.11, Yeelight: Bulb closed the connection.
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.158, Strip1: Bulb closed the connection.
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.51, Strip2: Bulb closed the connection.
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.60, Strip3: Bulb closed the connection.
2020-06-11 17:10:52 INFO (SyncWorker_4) [googleapiclient.discovery] URL being requested: GET https://www.googleapis.com/calendar/v3/calendars/s5lbp7v3e6rmjla04fi1oq3vf8%40group.calendar.google.com/events?orderBy=startTime&maxResults=5&singleEvents=true&timeMin=2020-06-11T19%3A10%3A52.105970%2B02%3A00&alt=json
2020-06-11 17:10:52 ERROR (SyncWorker_9) [homeassistant.components.yeelight] Unable to update device 10.1.0.34, Strip4: Bulb closed the connection.2020-06-11 17:10:52 INFO (SyncWorker_13) [googleapiclient.discovery] URL being requested: GET https://www.googleapis.com/calendar/v3/calendars/ima8u8rgbhtdrdlovt356ttik8%40group.calendar.google.com/events?orderBy=startTime&maxResults=5&singleEvents=true&timeMin=2020-06-11T19%3A10%3A52.343140%2B02%3A00&alt=json
2020-06-11 17:11:35 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from unavailable
2020-06-11 17:16:35 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from unavailable
2020-06-11 17:21:35 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from unavailable
2020-06-11 17:26:05 INFO (SyncWorker_13) [oauth2client.transport] Refreshing due to a 401 (attempt 1/2)
2020-06-11 17:26:35 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from unavailable
2020-06-11 17:29:39 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection.140348548613840] Unable to serialize to JSON. Bad data found at $.result[72](state: climate.migo).attributes.temperature=nan(<class 'float'>
Timeout on fetching data… and then a serie of exceptions and eventually, game over.
On every crash it seems that there is something wrong with the Synology Camera integration:
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='10.1.0.90', port=5000): Max retries exceeded with url: /webapi/entry.cgi?_sid=OuGs8iQHiUhoTHGwi1SbeOPhjxwVZii8wLFWa-EuIgowVmsY4uIxkMZ_MugUg48XYQR_SQ8vVDj_dLjjUVmwDo&api=SYNO.SurveillanceStation.Camera&method=List&version=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fa573e46910>, 'Connection to 10.1.0.90 timed out. (connect timeout=5)'))
My LAN is very stable and there is no obvious reason for the connexion between HA and my synology box to fail.
Any hints on understanding what is going on?
Thanks,
GV