Cannot start supervisor docker container

Hello everyone. I am having some issues with upgrading my hassio (used the supervised-installer on a nuc running debian). My nuc keeps on freezing and after a reboot it see all my containers stopped (except my portainer of course):


I think I might have a correct supervisor container, each time I start the container I get the following response:

[services.d] starting services
[services.d] done.
20-05-08 07:51:53 INFO (MainThread) [supervisor.bootstrap] Use the old homeassistant repository for machine extraction
20-05-08 07:51:53 INFO (MainThread) [__main__] Initialize Supervisor setup
20-05-08 07:51:53 INFO (MainThread) [supervisor.bootstrap] Setup coresys for machine: qemux86-64
20-05-08 07:51:53 INFO (SyncWorker_0) [supervisor.docker.supervisor] Attach to Supervisor homeassistant/amd64-hassio-supervisor with version 220
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.40/images/sha256:24aa6505d0451445e40565b7311859e28a5dccaddc1ecd12bc04fdf865f89851/json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/src/supervisor/supervisor/__main__.py", line 41, in <module>
    loop.run_until_complete(coresys.core.connect())
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/usr/src/supervisor/supervisor/core.py", line 31, in connect
    await self.sys_supervisor.load()
  File "/usr/src/supervisor/supervisor/supervisor.py", line 42, in load
    await self.instance.cleanup()
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 31, in wrap_api
    return await method(api, *args, **kwargs)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/supervisor/supervisor/docker/interface.py", line 321, in _cleanup
    for image in self.sys_docker.images.list(name=self.image):
  File "/usr/local/lib/python3.7/site-packages/docker/models/images.py", line 364, in list
    return [self.get(r["Id"]) for r in resp]
  File "/usr/local/lib/python3.7/site-packages/docker/models/images.py", line 364, in <listcomp>
    return [self.get(r["Id"]) for r in resp]
  File "/usr/local/lib/python3.7/site-packages/docker/models/images.py", line 316, in get
    return self.prepare_model(self.client.api.inspect_image(name))
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/image.py", line 246, in inspect_image
    self._get(self._url("/images/{0}/json", image)), True
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("readlink /var/lib/docker/overlay2: invalid argument")
20-05-08 07:51:53 ERROR (MainThread) [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f46ed8aa050>
20-05-08 07:51:53 ERROR (MainThread) [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f46ed65d4d0>
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Note that I provided 30GB of disk space for my hassio debian VM and only about 19% of that is in use.
On this post I found someone saying:

In my case… i used portainer just now. Clicked on hassio_supervisor container. Chose Duplicate/edit and recreated with latest image. restarted all containers and i think im up and going again, think i lost hacs though

Is this something that is worth trying? If possible I want to be able to fix this issue without losing any data, all help is appreciated!

1 Like

Having exactly the same issue, hassio 0.109.5 on NUC

Same, HA is not usable. even after system reboot.

@KAjFASH @wannes really weird, what OS are you guys using? I am using debian. Wondering if ubuntu server would work better even tough it’s based on debian…

I’m using quite fresh installation of Ubuntu 18.04.3 LTS. I’ve installed Docker with images of Plex and Transmission to have them as separate containers from HassIO so they are still alive. Then I’ve installed Home Assistant with several addons and now I have few addons running as well as supervisor is also stopped. No other software is here.


Error log from supervisor:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 207, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/local/lib/python3.7/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/src/supervisor/supervisor/__main__.py", line 40, in <module>
    coresys = loop.run_until_complete(bootstrap.initialize_coresys())
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/usr/src/supervisor/supervisor/bootstrap.py", line 50, in initialize_coresys
    coresys = CoreSys()
  File "/usr/src/supervisor/supervisor/coresys.py", line 56, in __init__
    self._docker: DockerAPI = DockerAPI()
  File "/usr/src/supervisor/supervisor/docker/__init__.py", line 34, in __init__
    base_url="unix:/{}".format(str(SOCKET_DOCKER)), version="auto", timeout=900
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 40, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 190, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 215, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
20-05-08 14:55:49 ERROR (MainThread) [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f0ecaaa3b90>
20-05-08 14:55:49 ERROR (MainThread) [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f0eca854750>
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Error log from MQTT addon:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing... 
curl: (7) Failed to connect to supervisor port 80: Host is unreachable
[17:55:49] ERROR: Something went wrong contacting the API
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] hivemq.sh: executing... 
patching file /app/index.html
Hunk 1 FAILED 25/25.
 <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width">
-    <title>MQTT Websocket Client</title>
-
+    <title>MQTT Webclient</title>
+    <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css" integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous">
     <link rel="stylesheet" href="css/normalize.css">
     <link rel="stylesheet" href="css/style.css">
     <link rel="stylesheet" href="css/foundation.css">
     <link rel="stylesheet" href="css/jquery.minicolors.css">
     <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/fancybox/2.1.5/jquery.fancybox.min.css">
 </head>
 <body class="notconnected">
 
 <div id="header">
     <div>
-        <a href="http://www.hivemq.com"><img id="headerlogo" src="assets/logo.png"></a>
-        <span id="headertitle">Websockets Client Showcase</span>
+        <a href="/"><img id="headerlogo" src="assets/logo.png"></a>
+        <span id="headertitle" style="padding: 8px;">MQTT Server and web client - Home Assistant Community Add-ons</span>
     </div>
 </div>
 <div id="content" class="row">
[cont-init.d] hivemq.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 99-message.sh: executing... 
-----------------------------------------------------------
                Oops! Something went wrong.

 We are so sorry, but something went terribly wrong when
 starting or running this add-on.
 
 Be sure to check the log above, line by line, for hints.
-----------------------------------------------------------
[cont-finish.d] 99-message.sh: exited 0.
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.

One more important item - there was internet outage for ~30 minutes during night and in the morning I’ve realized that most of my Node RED flows are not working but I was too busy to check so I’ve found this issue after 10h. Restarts do not help, I’ve tried to recreate container with latest image but no changes. I have full backup created two days ago and manually copied full directory of hass so will try reinstall everything if this issue still persist.

Really annoying, moving to ubuntu also does not seem to help if I hear you, anyone else experiencing this?

Trying to fix it I’ve found this item Supervisor Won't Start . So I’ve run manually

docker pull homeassistant/amd64-hassio-supervisor:latest
docker tag homeassistant/amd64-hassio-supervisor:latest homeassistant/amd64-hassio-supervisor:latest

And now supervisor and several addons run. NodeRed addon is not starting. But supervisor is not accessible from HA UI. Will continue to look how to fix it. Also there is major update https://www.home-assistant.io/blog/2020/05/09/deprecating-home-assistant-supervised-on-generic-linux/ which discussed in lot of threads, seems like it impacts supervisor and could be root of the issue.

I’ve also seen the blog articl, might switch to QCOW2 image

Ubuntu Server.
I ran the install script again (over the existing installation) and everything seems to work as before again.