Full load and crash - restart

Hi guys. My hass.io is very unstable. A couple times a day it’s crashing with a full CPU-load, http(s) not available, SSH with a lot of lag. When in SSH I can see it’s dealing with a lot of CPU-load until it crashes and restarts.

These are my logs:

httplib_response = conn.getresponse()
File “/usr/local/lib/python3.7/http/client.py”, line 1321, in getresponse
response.begin()
File “/usr/local/lib/python3.7/http/client.py”, line 296, in begin
version, status, reason = self._read_status()
File “/usr/local/lib/python3.7/http/client.py”, line 257, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), “iso-8859-1”)
File “/usr/local/lib/python3.7/socket.py”, line 589, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/requests/adapters.py”, line 449, in send
timeout=timeout
File “/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py”, line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File “/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py”, line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File “/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py”, line 686, in reraise
raise value
File “/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py”, line 600, in urlopen
chunked=chunked)
File “/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py”, line 386, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File “/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py”, line 306, in _raise_timeout
raise ReadTimeoutError(self, url, “Read timed out. (read timeout=%s)” % timeout_value)
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=900)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/hassio/tasks.py”, line 127, in _watchdog_homeassistant_docker
not await self.sys_homeassistant.is_fails()
File “/usr/local/lib/python3.7/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.7/site-packages/hassio/docker/interface.py”, line 391, in _is_fails
docker_container = self.sys_docker.containers.get(self.name)
File “/usr/local/lib/python3.7/site-packages/docker/models/containers.py”, line 862, in get
resp = self.client.api.inspect_container(container_id)
File “/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py”, line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/docker/api/container.py”, line 755, in inspect_container
self._get(self._url(“/containers/{0}/json”, container)), True
File “/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py”, line 46, in inner
return f(self, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/docker/api/client.py”, line 230, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 546, in get
return self.request(‘GET’, url, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 533, in request
resp = self.send(prep, **send_kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 646, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/adapters.py”, line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=900)
19-04-09 00:09:28 WARNING (SyncWorker_8) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_7) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_11) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_18) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_2) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_16) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_19) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_1) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:28 WARNING (SyncWorker_0) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-09 00:09:42 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!

Earlier yesterday:

File “/usr/local/lib/python3.7/site-packages/hassio/tasks.py”, line 97, in _watchdog_homeassistant_docker
if not await self.sys_homeassistant.is_fails() or
File “/usr/local/lib/python3.7/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.7/site-packages/hassio/docker/interface.py”, line 396, in _is_fails
docker_container = self.sys_docker.containers.get(self.name)
File “/usr/local/lib/python3.7/site-packages/docker/models/containers.py”, line 862, in get
resp = self.client.api.inspect_container(container_id)
File “/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py”, line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/docker/api/container.py”, line 755, in inspect_container
self._get(self._url(“/containers/{0}/json”, container)), True
File “/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py”, line 46, in inner
return f(self, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/docker/api/client.py”, line 225, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 546, in get
return self.request(‘GET’, url, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 533, in request
resp = self.send(prep, **send_kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 646, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/adapters.py”, line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=900)
19-04-08 20:08:55 WARNING (SyncWorker_13) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_11) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_18) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_5) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_4) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_0) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_10) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_9) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_8) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:08:55 WARNING (SyncWorker_17) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_7) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_15) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_9) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_11) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_3) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_15) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_18) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_5) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_0) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: localhost
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-04-08 20:09:18 ERROR (MainThread) [hassio.utils] Can’t execute start while a task is in progress
19-04-08 20:09:18 ERROR (MainThread) [hassio.homeassistant] Error on call https://172.30.32.1:8123/api/: Cannot connect to host 172.30.32.1:8123 ssl:None [Connection refused]
19-04-08 20:09:18 WARNING (MainThread) [hassio.tasks] Watchdog miss API response from Home Assistant
19-04-08 20:09:18 INFO (SyncWorker_4) [hassio.docker.interface] Start homeassistant/raspberrypi3-homeassistant
19-04-08 20:09:55 INFO (MainThread) [hassio.homeassistant] Detect a running Home Assistant instance

At this moment I’m running 91.2. But, it was also happening with 91.1.
Hass.io supervisor is on 155.

Don’t know where to start debugging :frowning:

In my personal opinion most strange problems are solved with a new SD card.

Do you still get the error on a new install? If not, probably a component or a spelling error or something. If yes, then it’s probably SD card related.

BR

Yeah, a set of new SD-cards are coming today…
Hope that’s going to help.

Try a reboot?
Is the SD Card full? (df -hl)
Even is the image on an SD Card is corrupted you should be able to use SD Formatter and then install a new image.

Fresh installation yesterday on a brand spanking new SD-card from a different brand.
Old configuration 85% restored. In stead of doing restore I wanted to start over component by component. What’s not ‘on’ right now is access from outside network via Let’s Encrypt/DuckDNS. Trying to only use Remote Control functionality…
So far, so good.

Great!
Hopefully it was the SD card. You rewriting the code anyhow. So hopefully code problems will be gone too.