The developer docs has a hello_world example add-on that was working great up until an hour ago or so. Now the add-on log throws the following error when starting the add-on (and it won’t actually run):
s6-overlay-suexec: fatal: can only run as pid 1
What seems to fix it is disabling the Docker default system init by changing config.yaml to include:
init: false
Since doing this the add-on log looks different with regards to s6 entries in there though.
I experience the same issue on multiple HA OS (v7.5 and 8.0) installs on various architectures (x64 and arm64).
Did something change in the Docker base image perhaps? How do I fix this (other than disabling Docker’s system init)?
I develop an add-on and a user noticed this too. It seems yesterday the s6-overlay version changed. Trying to fix this right now, but I already have "init":false in my config.json file and it’s still not working…
Unfortunately this won’t help. The version of the base image denotes the version of Alpine, Debian or Ubuntu you want (depending on which base image you’re using). However all base images for all available versions were updated yesterday from s6-overlay v2 to v3.
This is the first time we’ve had an issue like this so it warrants some thinking going forward. Since the base images are extremely vanilla inside there really hasn’t been any breaking changes until now. But changing s6 to v3 did create some potential issues.
Please read the linked blog post, it tells you what you need to do. In our testing we found the main issue encountered was that scripts in services.d, cont-init.d and cont-finish.d did not need to have execute permission in s6 v2, is6 corrected that for you if you forgot to add it. s6 v3 does not correct this and if you forget to add execute permission to these scripts it won’t work.
The less common issue is also in the blog post - apparmor. If you do use a custom apparmor profile for your addon please make the changes suggested in there. s6 v3 stores files in new places and needs permission to use them there.
If after making these changes your addon still isn’t working please let me know. If there is another issue I want to know so we can update the doc accordingly.
Also just an FYI for all, if your addon does not list image in its config then your backups should include the previously built image. Already built images should work fine just new ones will require these fixes. So you should able to simply restore from backup until you get a chance to fix things.
If you are an addon developer publishing addons that others use (and not just for personal use) I would strongly recommend publishing images for your addons to a container registry like ghcr.io or docker hub and then using the image option. This way existing versions of your addons will always keep working since the image is already built and available and not built on demand. Otherwise you risk issues like these, where a dependency updated in a way that caused an issue for you and suddenly no one can build a working image for your addon anymore.
Thank you for your extensive reply. I just started playing around trying to create a custom add-on, so I’m very much a rookie.
Could you maybe show what needs to be changed in the hello_world example from the docs to get it working again? It’s a very basic example with only a few lines of code so I would imagine the change required would be simple as well?
You got it already actually, this appears to be all that is needed:
As for the addon log looking different, that’s to be expected. S6 changed a lot in V3 and they log all the things they’re doing. They are mostly backwards compatible with a few exceptions that we’ve tried to note, looks like we may have missed one here.
As an aside you should definitely expect a bunch of addon updates coming up around the ecosystem. Although V3 is mostly backwards compatible they did mark a ton of things as deprecated/legacy that were previously considered best practices in addons. Deprecated stuff doesn’t have to be fixed immediately but does need to be fixed.
EDIT: Put in a PR to adjust the blog post and hello world example. Thanks for catching this @jant90 ! A lot of the HA addons already had init: false so we didn’t catch that this had become mandatory.
Hi, I did everything that was suggested, my addon finally runs but I can’t get the environment variable SUPERVISOR_TOKEN anymore, it has become undefined, therefore I can’t communicate with HA. Any idea on how to fix that? Thanks.
I’m not really sure how that could be, nothing about that was changed. The image has no control over that, supervisor injects that env when it runs the container. If the addon is in a repo I can look but there’s no recent change in that area I’m aware of to point to.
Thanks for your reply, I just figured that the problem was that I was lacking the "hassio_role": "default" in the config file. Weird because it wasn’t necessary before the update… Thanks again!
Hm that’s a new one to me, I didn’t have to make that change with any addons I tested. If the addon is in a GitHub repo I would still like a link so I can see try and figure out why you needed that and if we need to modify the blog post for others. Glad you got it working though!
If we were adding that option for the first time today - absolutely. However keep in mind not everyone uses the HA base images or s6 overlay in their addons and those that don’t still need init: true. So that means that would be another breaking change.
It’s not off the table but it would need to be managed. And tbh it’s probably not be worth it to risk breaking folks. The breaking change here was worth it because the base image was pretty far behind in it’s s6 version and it’s important to keep dependencies current. But that wouldn’t update anything, it would just be a more sensible default today.
Also init: false was actually supposed to be used in s6 v2 as well it just wasn’t required before (like the permissions thing in the run and finish scripts). The example addon and many of the addons in the official addons repo were already using it for that reason. We just missed a couple and unfortunately one of the ones missed was the hello world one.
