Install package in docker NodeRed

Installation
1

Hi everyone,

After using HA in supervised mode for a few years, I am switching to HA OS. My home automation is mainly managed by NodeRed and uses a CAN bus.
For this, I have to install multiple linux and nodejs packages.

In a first attempt, I added the linux and nodered packages in the Nodered docker configuration (system_packages and npm_packages). The problem is that the docker startup fails many times (error 137) before it starts.

A second solution would be to install the packages in the docker container and then remake an image (so as not to lose everything if the docker restarts) but what about docker updates by HA ?

Moreover, is it possible to change the docker launch command line (idem, without losing the modifications during updates or backup) ?

Thanks for your help,
Regards,

Silvain

2

Not certain what HW you are on, but could you not consider running things in a hypervisor? If you are mostly Node-RED a VM based installation should make many things easier to handle, including updates.

3

Hi,

I use Home assistant OS on Raspberry PI 4, so every docker run in the HAOS hypervisor and I use the NodeRed addon.

4

Unfortunately you will have to follow the path of pain for what you are trying to accomplish.

1 Like
5

For your system packages, are you sure they are part of Alpine (for ARM)

Secondly, if I google on error 137 it points me to the OOM killer, killing processes taking up too much memory. Do you know how much memory ypur packages should be consuming and which Pi4 edition do you have?

6

Hello,

Yes, the packages is part of Alpine, I tested the installation of the packages “by hand” before putting them in the configuration.

Indeed, the error corresponds to the lack of memory, that’s why I look for the docker command line because there can be a memory limitation for each docker.
I will try to monitor the memory consumption of the rpi (host) during the docker launch.

As I said, the docker ends up starting correctly after a lot of reboot (about 20mins)

I was wrong, I’m on a raspberry pi 3.

7

Hello,

I just did the test by looking at the memory consumption of the docker and that of the complete system, during the installation, there are 500Mb taken by the docker and I do not exceed 64% of the total memory consumption of the Rpi, so it is not a general limitation, perhaps a limitation specific to the docker but I do not know where it can be ?

Thanks

8

How did you measure the memory consumption? I’m on a Pi4 (a Pi3 is underpowered for this purpose if you ask me) with 4Gig of memory.

What I get for memory consumption:

CONTAINER ID   NAME                                        CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
ffbd0682915d   homeassistant                               5.98%     645.2MiB / 3.698GiB   17.04%    0B / 0B           114MB / 21.7MB    65
6f5921fb228d   addon_5c53de3b_esphome                      0.14%     33.76MiB / 3.698GiB   0.89%     0B / 0B           27.6MB / 582kB    12
550145a76008   addon_cebe7a76_hassio_google_drive_backup   0.24%     31.78MiB / 3.698GiB   0.84%     1.44GB / 970MB    20.2MB / 864kB    6
7631a42cf22d   addon_a0d7b954_nodered                      2.48%     115.6MiB / 3.698GiB   3.05%     0B / 0B           106MB / 1.69MB    32
6d487e75167a   addon_a0d7b954_vscode                       0.02%     42.9MiB / 3.698GiB    1.13%     458MB / 25kB      68.3MB / 26.4MB   29
2857737cbccd   addon_a0d7b954_ssh                          1.40%     60.93MiB / 3.698GiB   1.61%     0B / 0B           66.3MB / 528kB    22
ddaf96d84bfa   addon_core_deconz                           4.37%     84.25MiB / 3.698GiB   2.22%     489MB / 171MB     223MB / 2.5GB     44
5a7a881cb485   addon_core_samba                            0.00%     13.26MiB / 3.698GiB   0.35%     0B / 0B           27.5MB / 2.25GB   13
cc11dbcb82fb   addon_a0d7b954_nut                          0.00%     7.148MiB / 3.698GiB   0.19%     471MB / 39MB      25.5MB / 442kB    11
e12f94ddd75e   addon_core_mariadb                          0.74%     175.9MiB / 3.698GiB   4.65%     1.36GB / 732MB    12.6GB / 6.42GB   25
bb131718a4df   hassio_multicast                            0.01%     964KiB / 3.698GiB     0.02%     0B / 0B           53.2kB / 102kB    4
c7594cf5f88d   hassio_audio                                0.00%     8.266MiB / 3.698GiB   0.22%     449MB / 0B        21MB / 315kB      13
2b2bbe0c48df   hassio_dns                                  0.17%     16.27MiB / 3.698GiB   0.43%     481MB / 51.9MB    27.9MB / 106kB    15
a17e6f66eeb9   hassio_cli                                  0.00%     3.934MiB / 3.698GiB   0.10%     449MB / 4.07kB    13.4MB / 283kB    9
4450494c23cf   hassio_supervisor                           0.19%     261.7MiB / 3.698GiB   6.91%     2.52GB / 1.89GB   4.14GB / 3.09GB   33
9cfeed894769   hassio_observer                             0.00%     10.25MiB / 3.698GiB   0.27%     452MB / 2.12MB    13.9MB / 106kB    13

Interestingly enough the docker container only reports 32megs of usable free memory using the top command from a shell inside the container.

I’s recommend using the “docker stats” command during start of the container. Also, post the relevant logs.

9

Hello

Thank you all for your answers.

Here are the docker stats after the nodered startup (when it succeeded):

CONTAINER ID   NAME                     CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O        PIDS
6559c9299925   addon_a0d7b954_nodered   0.00%     97.3MiB / 909.6MiB    10.70%    0B / 0B           253MB / 289MB    32
a8d5e41a2844   addon_a0d7b954_ssh       0.00%     3.844MiB / 909.6MiB   0.42%     0B / 0B           236MB / 12.9MB   14
aa75c8f7a178   addon_5c53de3b_esphome   0.01%     7.43MiB / 909.6MiB    0.82%     0B / 0B           81.7MB / 582kB   12
87308527d325   hassio_multicast         0.00%     640KiB / 909.6MiB     0.07%     0B / 0B           1.82MB / 102kB   4
d61c88f35bee   hassio_audio             1.75%     2.918MiB / 909.6MiB   0.32%     13.1MB / 0B       38.5MB / 315kB   13
2982ca885057   hassio_dns               0.25%     9.801MiB / 909.6MiB   1.08%     18MB / 2.79MB     128MB / 106kB    13
98957eee1fbe   hassio_cli               0.00%     1.027MiB / 909.6MiB   0.11%     13.1MB / 10.4kB   22.3MB / 283kB   9
c230750bba53   homeassistant            0.60%     113.2MiB / 909.6MiB   12.44%    0B / 0B           259MB / 531MB    29
d793eaec6a61   hassio_supervisor        0.00%     59.19MiB / 909.6MiB   6.51%     145MB / 177MB     360MB / 10.7MB   27
42f9074d28bb   hassio_observer          0.00%     5.449MiB / 909.6MiB   0.60%     13.8MB / 531kB    55.7MB / 106kB   10

During multiple launches, I never exceed 37.4MiB and 10.02% CPU (just before crash)

Here is the Top inside the docker when crashing :

Mem: 853104K used, 78364K free, 204K shrd, 8260K buff, 204272K cached
CPU:   7% usr   8% sys   0% nic  82% idle   0% io   0% irq   1% sirq
Load average: 1.31 1.17 0.77 3/487 184
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
  151   135 root     S    40036   4%   0   4% apk add g++
  174   166 root     R     1672   0%   1   0% top
  135    43 root     S     4384   0%   0   0% bash /usr/bin/bashio /etc/cont-init.d/customizations.sh
  166     0 root     S     2792   0%   3   0% /bin/bash
   43    42 root     S     1680   0%   0   0% {cont-init} /bin/sh -e /package/admin/s6-overlay-3.1.2.1/etc/s6-rc/scripts/cont-init
   14     1 root     S     1676   0%   0   0% {rc.init} /bin/sh -e /run/s6/basedir/scripts/rc.init top
  183    17 root     S      208   0%   1   0% s6-rc -v2 -bda change
   29    14 root     S      208   0%   2   0% s6-rc -v2 -u -t 0 -- change top
   25     1 root     S      204   0%   0   0% s6-supervise s6rc-fdholder
   24     1 root     S      204   0%   1   0% s6-supervise s6rc-oneshot-runner
   15     1 root     S      204   0%   1   0% s6-supervise s6-linux-init-shutdownd
    1     0 root     S      200   0%   2   0% /package/admin/s6/command/s6-svscan -d4 -- /run/service
   42    33 root     S      196   0%   3   0% /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --
   17    15 root     S      192   0%   3   0% /package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B
   41    29 root     S      188   0%   3   0% /package/admin/s6-2.11.1.2/command/s6-sudoc -e -t 30000 -T 0 -- up 3
#  33    24 root     S      180   0%   3   0% /package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 300

I agree that the rpi3 is not the best for home automation but the price of rpi4 at the moment is quite dissuasive :disappointed_relieved:

10

A long time ago, I moved NR to its own pi3 because it was too slow running on the same pi3 as HA. My HA now runs on a pi4 (bought while price was reasonable), but NR is still on a separate pi3.

I use IOTstack to help install NR and HA (container version).
Obviously it would take some work to split off NR and run it by itself, but its something to think about.

IOTstack also helps with updating HA and other whatever other programs you use it to install.

11

Hello,

Here is the event from Docker :


2022-10-21T15:37:18.751964064Z container exec_create: /bin/sh -c curl --fail http://127.0.0.1:46836 || exit 1 44fc..., supervisor_managed=)
2022-10-21T15:37:18.752249637Z container exec_start: /bin/sh -c curl --fail http://127.0.0.1:46836 || exit 1 44fc..., supervisor_managed=)
2022-10-21T15:37:19.075597893Z container exec_die 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (execID=a24a2bc4975f055c28b77c1d712532e5124a6783fab6510c88b03e94ed65677f, exitCode=1, image=ghc... supervisor_managed=)
2022-10-21T15:37:19.097353134Z container health_status: unhealthy 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (ima...naged=)
2022-10-21T15:37:19.218083288Z container kill 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (image=ghcr..., signal=15, supervisor_managed=)
2022-10-21T15:37:29.308143961Z container kill 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (imag... signal=9, supervisor_managed=)
2022-10-21T15:37:29.523239292Z container die 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (exitCode=137, ima.. supervisor_managed=)
2022-10-21T15:37:29.558953166Z network disconnect ...
2022-10-21T15:37:30.509797431Z container stop 44fc2f..pervisor_managed=)
2022-10-21T15:37:30.838913807Z container destroy 44fc2f0f500fbfab7ecc2942b2904f5f40a0a180c586b767ab856721c45dc429 (ima... pervisor_managed=)

2022-10-21T15:37:31.220601491Z container create 21d7e4e196906

What does it mean that the container is unhealthy ?

12

Hello,

I am making progress in my investigations. If I understand well the functioning of docker, we define a command that it will execute to know if the container is ok. In the NodeRed container, the command is (line 61 of the Dockerfile):

HEALTHCHECK CMD curl --fail http://127.0.0.1:46836 || exit 1

This command will look for the NodeRed home page and if there is a response, then everything is fine.

The problem in my case is that the initial installation takes time and NodeRed launches later than expected or Docker will check if everything is fine too early (with the command) and as it has a negative response, it shuts down the container while the launch is not finished.

So, the question is how to tell docker that everything is ok during the install :thinking:?

13

You can’t right out of the box. Faster storage might help.

14

Hello,
I can change the rpi but I run on an SSD which is relatively fast.

15

The pi4 has usb3, the 3 hasn’t. If your ssd is usb3 as well, your storage will be much faster.

16

I understand that the speed can accelerate the launch of the docker a bit, however, I find that it is not a solution to hope to accelerate the launch enough before docker asks for the health of the docker and the kill if it is not finished.
Especially since the installation depends on the speed of the network or other parameters.

17

Finally, I create a issue :