[Solved] hassio_audio is in boot loop and spams my syslog with errors

It looks like it doesn’t work on a tinkerboard :frowning:

thank you so much!!! worked for me too!


Core
Version
core-2021.2.3

Host
Operating System
Raspbian GNU/Linux 9 (stretch)
Docker version
19.03.8

Supervisor
Version
supervisor-2021.02.11

Hi,
thank you very much, it worked also for me. (RPI4 4GB Hassio in Docker)

Restored HomeAssistant recently after an SD-Card Burn and was worrying as I saw an increased usage of CPU and load, if it is going to fail again… At least I configured the automatic backup so Im up and runnig again without issues in case the worst case happenes.

Also forgott to change the " " in the dameon.json altough it is obvious because of the color…

I found out about this issue in my installation because of the growing size of the user.log, when I checked it, I found the pulseaudio errors.

I can’t thank you enough… was just pulling out my hair as for a couple of days the LED on my Raspberry was blinking as mad and the cooling fan was turning on every couple of minutes… and I couldn’t find out, what’s going on (except I saw in “iotop” that s6-supervise pulseaudio was writing to disk all the time)…
However, this solution only works partial for me… it stops insane writing to disk, but the CPU usage is still high (in “htop” I can see roughly 14% from service “pulseaudio --system-vvv --disallow-exit --exit-idle-time=-1 --disable shm”).
If I pause Supervisor and stop hassio-audio container, it stops throttling CPU, but after reenabling Supervisor (which reenables hassio-audio) it starts again…
Any hint on how to tackle also this problem?

I have exactly the same issue.

Also I experience increased CPU usage related to the PulseAudio daemon running in hassio_audio.
It fluctuates between 0% and 15% all the time.

I tried to start the pulseaudio daemon in the container with different settings and parameters, but could find a way to reduce the CPU usage.

Perhaps someone else with more knowledge can have a go and try to sort this out.

What I did:

# log on to the hassio_audio container with a Bash shell
container exec -it hassio_audio bash

# changed runtime parameters of the pulseaudio daemon in the "run" file that is kicked off by the s6-supervice process:
/run/s6/services/pulseaudio/run

# original parameters:
exec pulseaudio --system -vvv --disallow-exit --exit-idle-time=-1 --disable-shm

# changed some daemon defaults used by above process when started:
/etc/pulse/daemon.conf

# you can get a full list of parameters using:
pulseaudio --dump-conf

# restarted the daemon within the container by getting the PID and killing it.
# the daemon will be restarted automatically, using the new values in the "run" and "daemon.conf" files.
ps -ef
kill <PID>

# NOTE that any changes are lost when the container is restarted, so it's a quick way to start fresh.
# Also happens when an invalid parameter is provided and the daemon fails to start.

Some helpful sites with info on PulseAudio:
https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Daemon/
https://www.systutorials.com/docs/linux/man/1-pulseaudio/
https://wiki.archlinux.org/index.php/PulseAudio
https://wiki.debian.org/PulseAudio?action=fullsearch&context=180&value=loglevel&titlesearch=Titles#Solving_Problems

You can monitor the hassio_audio container logs realtime as follows:

docker logs -f hassio_audio

I’ve just run into this problem.

I’m trying to find some issue on github about this bug, but I can’t find anything about it not in https://github.com/home-assistant/core , not in https://github.com/home-assistant/supervisor/issues

Is there a ticket where this problem is being fixed?

I’ve fixed it by installing following addon. hassio_audio stays on 0% CPU in docker stats now.
I can’t confirm this approach won’t break your audio as I’m not using any audio device/service connected to the HA.

Could you find anything wrong here:

ze@tinkerboard:~$ cat /etc/docker/daemon.json
{
    "data-root": "/mnt/extdata/docker",
    "log-driver": "journald",
    "seccomp-profile": "/etc/docker/default.json",
    "storage-driver": "overlay2"
}
ze@tinkerboard:~$ head /etc/docker/default.json 
{
	"defaultAction": "SCMP_ACT_TRACE",
	"archMap": [
		{
			"architecture": "SCMP_ARCH_X86_64",
			"subArchitectures": [
				"SCMP_ARCH_X86",
				"SCMP_ARCH_X32"
			]
		},
ze@tinkerboard:~$ sudo docker exec -it hassio_audio ping 8.8.8.8
[sudo] password for ze: 
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=116 time=24.146 ms
64 bytes from 8.8.8.8: seq=1 ttl=116 time=20.841 ms
64 bytes from 8.8.8.8: seq=2 ttl=116 time=21.091 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 20.841/22.026/24.146 ms
ze@tinkerboard:~$ sudo docker exec -it hassio_audio date
Wed Feb 24 21:58:07 WET 2021
ze@tinkerboard:~$ 

@metropt No.
And the patch seems to work because the ping and date commands are working, too…
If you still got problems with hassio_audio it must be something different…

The PulseAudio module-suspend-on-idle module can be loaded in hassio_audio from the command prompt using Docker exec.

docker exec -i hassio_audio pactl load-module module-suspend-on-idle

You may be able to execute the Docker command directly from HA using the Shell Command integration, perhaps together with an automation that’s triggered when e.g. HA starts up (but this won’t cover the case when hassio_audio is restarted afterwards for whatever reason).

I chose a different route, and created a shell script wrapper around the Docker exec command. It listens to Docker Events for when the hassio_audio container is started, and then automatically loads the PulseAudio module. You can launch this shell script in different ways, again from HA should you so wish, BUT there will be a timing problem… the script in it’s current form must already be running when the hassio_audio container is started. I therefore implemented it as a “systemd” service that is started at bootup, after Docker started but before hassio_supervisor (and therefore hassio_audio) is started. Of course the script can be enhanced to check on startup if hassio_audio is already running, and if so then load the suspend module if not already loaded.

The script(s) and description is in github…

1 Like

José, already solved your issue? High CPU, was it?

Only difference I see between my and your config is that you have this extra line in daemon.json:
"data-root": "/mnt/extdata/docker",

But you can ping and I assume your system time in hassio_audio is ok.
Did you try to load the PulseAudio “module-suspend-on-idle” module?

Just to confirm that it actually works, you could do this:

# open two terminal windows, A and B.
# run this in (A) to monitor the logs realtime..
docker logs -f hassio_audio 

# now load the suspend module in (B)
docker exec -it hassio_audio pactl load-module module-suspend-on-idle

# you should see something like this in (A) when the module is loaded
...
I: [pulseaudio] module.c: Loaded "module-suspend-on-idle" (index: #16; argument: "").
I: [pulseaudio] client.c: Freed 0 "pactl"
I: [pulseaudio] protocol-native.c: Connection died.
I: [pulseaudio] module-suspend-on-idle.c: Sink auto_null idle for too long, suspending ...
D: [pulseaudio] sink.c: auto_null: suspend_cause: (none) -> IDLE
D: [pulseaudio] sink.c: auto_null: state: IDLE -> SUSPENDED
D: [pulseaudio] source.c: auto_null.monitor: suspend_cause: (none) -> IDLE
D: [pulseaudio] source.c: auto_null.monitor: state: IDLE -> SUSPENDED
D: [pulseaudio] core.c: Hmm, no streams around, trying to vacuum.
...

# afterwards list the loaded modules to confirm the suspend module is there (nr 16?) 
docker exec -it hassio_audio pactl list modules
...
Module #16
	Name: module-suspend-on-idle
	Argument: 
	Usage counter: n/a
	Properties:
		module.author = "Lennart Poettering"
		module.description = "When a sink/source is idle for too long, suspend it"
		module.version = "14.2"


Below is my daemon.json, same as was posted by Dominik.

cat /etc/docker/daemon.json
{
    "log-driver": "journald",
    "seccomp-profile": "/etc/docker/default.json",
    "storage-driver": "overlay2"
}
2 Likes

To automate this on system startup you can create a service!

sudo nano /etc/pa-suspend-on-idle.sh

#!/bin/bash
sleep 4m
sudo docker exec -i hassio_audio pactl load-module module-suspend-on-idle

sudo nano /etc/systemd/system/pa-suspend-on-idle.service

[Unit]
Description=Disable Pulse Audio

[Service]
Type=simple
ExecStart=/etc/pa-suspend-on-idle.sh

[Install]
WantedBy=multi-user.target

sudo systemctl enable pa-suspend-on-idle

Yep, also created a service. Except my shell script does not load the module every 4 minutes, but only when the container (re)starts.

1 Like

How did you do this? With an Entrypoint?

I could not find a way for Docker to trigger a script when a container is started. Perhaps I missed it? Kubernetes have things like “prestart” and “poststart” hooks, but not for our Containers.
Only other option was listening for Docker Events, but that is not my preference at all.
CPU for the wait is low, zero most of the time, and only now and then a “spike” to 0.6% (in “htop”).

I explained the detail in the github repository.

The looping part of the shell:

function event_loop () {
  while read line; do
    if [[ ${line} == *"Status=start" ]]; then
        # Wait to allow container to start (else may get connection refused error).
        sleep 5
        docker exec -i hassio_audio pactl load-module module-suspend-on-idle
    fi
  done
}
docker events  --filter ${event_filter} --format "${event_format}" | event_loop

2 Likes

Great! Thank you!!
:+1: :partying_face:

Dominik - thanks very much for your very clear and concise instructions. After finding this post it’s all fixed in < 10 mins. You’re a star. Cheers Coo

1 Like

I have tried:

`root@raspberrypi:~# sudo docker exec -i hassio_audio pactl load-module module-suspend-on-idle

Connection failure: Connection refused
pa_context_connect() failed: Connection refused`

I invokes command: docker ps to get the following line:
homeassistant/armv7-hassio-audio:2021.02.1

I am using method Supervised.

How to resolve it?

I see sometimes that the following is consuming high CPU:
ghcr.io/hassio-addons/appdaemon/armv7:0.4.3

Erik
Some comments on what you wrote…

You don’t need to use “sudo” if you are root already. “sudo” is a shortcut to let you execute a command as root (super user do) when you are a “normal” user.
You did not run the command with “-it”. Note the “t” for TTY. This is necessary when you execute docker exec from the command prompt (but you must exclude it when the command is executed from a shell script).

Faultfinding is about a process of elimination. First confirm that you are doing it right, by executing a “generic” command in any of your running containers. If it works, then your command is ok and you don’t have e.g. permission problems. If the same command works in one container and not the other, then there is something wrong with that container. Etc.

Can you try the following? You can run this as “pi”, you don’t need to be root.

# First just execute a generic command in the container, to see if the container itself is ok.
# This should return all the processes running within the container.
docker exec -it hassio_audio ps -ef

# If that worked ok, then list the modules that are currently loaded:
docker exec -it hassio_audio pactl list modules

# If ok then try to load the module again:
docker exec -it hassio_audio pactl load-module module-suspend-on-idle

# If you still get the same error, try to stop the hassio_audio container to reset it. 
# It should be started automatically by the hassio_supervisor after a short period.
docker stop hassio_audio

# If nothing works, can you show the (full) output of the following commands?
docker images
docker ps -a

# You can also try to execute a generic command in other containers, to confirm you do it right. e.g.
docker exec -it hassio_supervisor date