What's up docker? How to keep your containers up to date

gmarlow · May 6, 2021, 8:53pm

Much better. However WUD is now reporting Adguard has updatable even though I have the latest version.

fmartinou · May 6, 2021, 9:38pm

Are you sure you run the '“latest latest” version?
I mean, according to your screenshot, your running version is dated from 2021/04/30 whereas the latest tag from adguard has been updated today.
Can you try to pull the image?
Docker should download the new version.
And wud should report that there is no more available update (will take up to an hour because wud watches the containers every hour by default)

gmarlow · May 6, 2021, 10:03pm

Yup. Did a docker-compose down and then up with tag:latest (and it didn’t pull a new image).

Did a direct pull, too.

sudo docker pull adguard/adguardhome:latest

latest: Pulling from adguard/adguardhome
Digest: sha256:df033af87e1ea30fb248f0d681525440dcd71e94ea975349c1e02809bce7c526
Status: Image is up to date for adguard/adguardhome:latest

fmartinou · May 6, 2021, 10:19pm

As far I know, docker-compose up/down only pulls when you don’t locally have the tag.
Because you already have a local version of latest,D ocker picks this one.
If you want to pull new versions, you must run

docker pull …
Or
docker-compose pull …

gmarlow · May 6, 2021, 10:21pm

sudo docker pull adguard/adguardhome:latest

latest: Pulling from adguard/adguardhome
Digest: sha256:df033af87e1ea30fb248f0d681525440dcd71e94ea975349c1e02809bce7c526
Status: Image is up to date for adguard/adguardhome:latest

Adguard interface confirms that I’m running the latest version (v0.106.2)

fmartinou · May 6, 2021, 10:26pm

Is that the only container reported wrong?
Can you restart wud?
(it processes all containers at startup) just to make sure to observe the last results

gmarlow · May 6, 2021, 10:28pm

Yes, it’s the only one. I’ve restarted WUD multiple times as well as clicking the “Watch Now” button.

2021-05-06_15-29-09

fmartinou · May 6, 2021, 10:29pm

I’ll try to debug it tomorrow to see if I can explain the issue.

fmartinou · May 7, 2021, 7:34am

Seems to be working for me
The local tag and the tag on the registry point to the same digest:

sha256:b008059a52a2fea8f0959bf90ad849addd4cd10442c36db56d687d29038745cc

Can you check:

That the right image is locally pulled

docker image inspect --format='{{.Id}}'  adguard/adguardhome

# sha256:b008059a52a2fea8f0959bf90ad849addd4cd10442c36db56d687d29038745cc

Check that your running container is using the right image

docker container inspect --format="{{ .Image}}" adguard

# sha256:b008059a52a2fea8f0959bf90ad849addd4cd10442c36db56d687d29038745cc

If the 2nd command returns another digest, then it means that you’re not running the latest version associated to the latest tag (and it means that WUD was right).

Run docker-compose up -d and you should be fine.

I don’t know how you perform your updates but I think the right command sequence should be:

docker pull adguard/adguardhome to get the new version
docker-compose up -d (no need for docker-compose down before) to let Compose automatically recreate the container with the new version)

moralsnipe · May 7, 2021, 2:06pm

So now that I’m using WUD with the MQTT integration, does anyone have a nice lovelace card that can show a list of all containers that have updates?

gmarlow · May 7, 2021, 3:09pm

Weird… This morning it shows that there is NO update available.

mwolter · May 7, 2021, 10:21pm

Hello famartinou, Going through the steps of installing and configuring WUD and TLS appears to be working well. Thanks for doing that!

Working on setting up the labels and not sure what flavor regex it follows. Is there an online tool you can recommend to help with creating a compatible format?

Also, can you provide an example exclude label? For instance, a container I use had a tag created by mistake that is sequentially higher than the most current and needs to be excluded.

One last thing, the example labels here do not have parenthesis for capturing a group of characters but your examples above do. Does the example need to be updated?

fmartinou · May 7, 2021, 10:36pm

You can test your regex using https://regex101.com (JavaScript flavor)

fmartinou · May 8, 2021, 8:54am

One last thing, the example labels here do not have parenthesis for capturing a group of characters but your examples above do. Does the example need to be updated?

Group captures are not required; the examples could be simplified.

mwolter · May 8, 2021, 7:35pm

So far everything has been working well. Takes a bit of time to setup the TLS and labels but hopefully this only needs to be done once.

Couple things that might help others and could be good to add to the documentation.

Include and exclude labels match the string after the colon “:” in the image tag. So if you want to create and test a new regex match, go to regex101 paste the image tag in the “test string” field and enter the regex in its field.

For example: Regex matching for labels can be simplified to:

^\d+\.\d+\.\d+-amd64$$

for the tag

0.8.4-amd64

If you’re interested in accessing a remote docker host via TLS the bash scripts below greatly simplify the process. Portions of the scripts were taken from a containerized script here and customized to work specifically with docker. These scripts create the CA certificate as well as a cert/key pair that can be used on the docker host and the client. All certs are good for 10 years and have been tested with Portainer and What’s up docker.

To use these files, SSH into the docker host, and create a folder to contain the certs as well as the scripts:

mkdir -p ~/certs && cd ~/certs

Make the script to create the certs

nano ~/certs/certs_create.sh
--------
#!/usr/bin/env bash

export CA_KEY=${CA_KEY-"ca-key.pem"}
export CA_CERT=${CA_CERT-"ca.pem"}
export CA_SUBJECT=${CA_SUBJECT:-"test-ca"}
export CA_EXPIRE=${CA_EXPIRE:-"3650"}

export SSL_CONFIG=${SSL_CONFIG:-"openssl.cnf"}
export SSL_KEY=${SSL_KEY:-"key.pem"}
export SSL_CSR=${SSL_CSR:-"key.csr"}
export SSL_CERT=${SSL_CERT:-"cert.pem"}
export SSL_SIZE=${SSL_SIZE:-"4096"}
export SSL_EXPIRE=${SSL_EXPIRE:-"3650"}

export SSL_SUBJECT=${SSL_SUBJECT:-"example.com"}
export SSL_DNS=${SSL_DNS}
export SSL_IP=${SSL_IP}

# Print IP v4 addresses, select only the interface and address, exclude certain interfaces, remove everything except the IP address, replace new line characters \n with comma (,), remove comma from end of string if exists
# export SERVER_IP=${SERVER_IP:-$(ip -o -4 addr show | awk '{print $2" "$4}' | grep -v -e "docker" -e "br-" -e "veth" | sed -n -r -e 's|.* (.*?)\/.*|\1|p' | tr '\n' ',' | sed 's/\,$//')}

# name for docker daemon file
export DAEMON=${DAEMON:-"daemon.json"}

export K8S_NAME=${K8S_NAME:-"omgwtfssl"}
export K8S_NAMESPACE=${K8S_NAMESPACE:-"default"}
export K8S_SAVE_CA_KEY=${K8S_SAVE_CA_KEY}
export K8S_SAVE_CA_CRT=${K8S_SAVE_CA_CRT}
export K8S_SHOW_SECRET=${K8S_SHOW_SECRET}

export OUTPUT=${OUTPUT:-"yaml"}

echo "Generating new config file ${SSL_CONFIG}"

# needed for k8s cert-manager compatibility
echo "
[ v3_ca ]
basicConstraints = critical,CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
" >> /etc/ssl/openssl.cnf

[[ -z $SILENT ]] && echo "--> Certificate Authority"

if [[ -e ./${CA_KEY} ]]; then
    [[ -z $SILENT ]] && echo "====> Using existing CA Key ${CA_KEY}"
else
    [[ -z $SILENT ]] && echo "====> Generating new CA key ${CA_KEY}"
    openssl genrsa -out ${CA_KEY} ${SSL_SIZE} > /dev/null
fi

if [[ -e ./${CA_CERT} ]]; then
    [[ -z $SILENT ]] && echo "====> Using existing CA Certificate ${CA_CERT}"
else
    [[ -z $SILENT ]] && echo "====> Generating new CA Certificate ${CA_CERT}"
    openssl req -x509 -new -nodes -key ${CA_KEY} -days ${CA_EXPIRE} -out ${CA_CERT} -extensions v3_ca -subj "/CN=${CA_SUBJECT}" > /dev/null  || exit 1
fi

cat > ${SSL_CONFIG} <<EOM
[ v3_ca ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
basicConstraints = CA:true
[req]
x509_extensions = v3_ca
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_ca ]
basicConstraints = critical,CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
EOM


ips=0

if [[ -n ${SSL_DNS} || -n ${SSL_IP} ]]; then
    cat >> ${SSL_CONFIG} <<EOM
subjectAltName = @alt_names
[alt_names]
EOM

	IFS=","
	dns=(${SSL_DNS})
	dns+=(${SSL_SUBJECT})
	for i in "${!dns[@]}"; do
	  echo DNS.$((i+1)) = ${dns[$i]} >> ${SSL_CONFIG}
	done

	if [[ -n ${SSL_IP} ]]; then
	    ip=(${SSL_IP})
	    for i in "${!ip[@]}"; do
	      echo IP.$((i+1)) = ${ip[$i]} >> ${SSL_CONFIG}
	    done
	fi
fi

# # Adding server IP addresses as additional addresses to the config
# IFS=","
# if [[ -n ${SERVER_IP} ]]; then
#     ip=(${SERVER_IP})
#     for i in "${!ip[@]}"; do
#         echo IP=${ip[$i]} >> ${SSL_CONFIG}
#     done
# fi


[[ -z $SILENT ]] && echo "--> Certificate Authority"

if [[ -e ./${CA_KEY} ]]; then
    [[ -z $SILENT ]] && echo "====> Using existing CA Key ${CA_KEY}"
else
    [[ -z $SILENT ]] && echo "====> Generating new CA key ${CA_KEY}"
    openssl genrsa -out ${CA_KEY} ${SSL_SIZE} > /dev/null
fi

if [[ -e ./${CA_CERT} ]]; then
    [[ -z $SILENT ]] && echo "====> Using existing CA Certificate ${CA_CERT}"
else
    [[ -z $SILENT ]] && echo "====> Generating new CA Certificate ${CA_CERT}"
    openssl req -x509 -new -nodes -key ${CA_KEY} -days ${CA_EXPIRE} -out ${CA_CERT} -subj "/CN=${CA_SUBJECT}" -config ${SSL_CONFIG} > /dev/null  || exit 1
fi

[[ -z $SILENT ]] && echo "====> Generating new SSL KEY ${SSL_KEY}"
openssl genrsa -out ${SSL_KEY} ${SSL_SIZE} > /dev/null || exit 1

[[ -z $SILENT ]] && echo "====> Generating new SSL CSR ${SSL_CSR}"
openssl req -new -key ${SSL_KEY} -out ${SSL_CSR} -subj "/CN=${SSL_SUBJECT}" -config ${SSL_CONFIG} > /dev/null || exit 1

[[ -z $SILENT ]] && echo "====> Generating new SSL CERT ${SSL_CERT}"
openssl x509 -req -in ${SSL_CSR} -CA ${CA_CERT} -CAkey ${CA_KEY} -CAcreateserial -out ${SSL_CERT} \
    -days ${SSL_EXPIRE} -extensions v3_req -extfile ${SSL_CONFIG} > /dev/null || exit 1


# cleanup unnecessary files
# rm -v key.csr openssl.cnf

# permissions to protect the keys
chmod -v 0400 ca-key.pem key.pem

# create daemon file
echo "
{
    \"hosts\": [\"unix:///var/run/docker.sock\", \"tcp://0.0.0.0:2375\"],
    \"tls\": true,
    \"tlscacert\": \"$(pwd)/${CA_CERT}\",
    \"tlscert\": \"$(pwd)/${SSL_CERT}\",
    \"tlskey\": \"$(pwd)/${SSL_KEY}\",
    \"tlsverify\": true
}    
" > ${DAEMON}


if [[ -z $SILENT ]]; then
echo "====> Complete"
echo "keys can be found in volume mapped to $(pwd)"
echo
fi

Make the script to apply the certs

nano ~/certs/certs_apply.sh
--------
#!/bin/bash
set -ex

mkdir -p /etc/systemd/system/docker.service.d
cat > /etc/systemd/system/docker.service.d/docker.conf <<EOM
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd
EOM

cp daemon.json /etc/docker/

systemctl daemon-reload
systemctl restart docker

Run the following command to create the scripts.

SSL_SUBJECT=dockerhost1.local SSL_IP=192.168.8.12,192.168.8.11 bash ~/certs/certs_create.sh

SSL_SUBJECT: Enter the DNS name of the remote docker host.
SSL_IP: Enter the remote docker host and client IP addresses.
Review the certs_create.sh script for additional environment variables.

Then run the following command to copy the scripts to the proper location on the Docker host. This script will stop and restart the docker service to apply the certs.

sudo bash ~/certs/certs_apply.sh

Then on the client machine where What’s up Docker will run, map a persistent volume to contain the certs.

mkdir -p ~/certs && cd ~/certs

Copy the key.pem, cert.pem and ca.pem files to the client machine in the directory created above.

In the What’s up Docker run command, ensure the volume is mapped

~/certs:/certs

and make sure the, client DNS or IP and port as well as the certificates are passed in as environment variables

WUD_WATCHER_FRIGATE-LXC_HOST="192.168.8.12"
WUD_WATCHER_FRIGATE-LXC_PORT="2375"
WUD_WATCHER_FRIGATE-LXC_CAFILE="/certs/ca.pem"
WUD_WATCHER_FRIGATE-LXC_CERTFILE="/certs/cert.pem"
WUD_WATCHER_FRIGATE-LXC_KEYFILE="/certs/key.pem"

Start the What’s up Docker container and navigate to its web page and verify the environment variables are passed through and it sees the containers on the remote host.

http://192.168.8.11:3000/watchers

finity · May 9, 2021, 6:15pm

I use a custom lovelace card called “auto-entities”.

Here is my card config:

- type: 'custom:auto-entities'
  card: 
    type: entities
  show_empty: false
  state_color: true
  filter:
    include:
      - entity_id: "binary_sensor.wud_image_hub_*"
        state: 'on'
    exclude:
      - entity_id: "binary_sensor.wud_image_hub_homeassistant_*"

I don’t watch for new HA containers since I get notifications for those separately. If you wantr them in just remove the “exclude:” lines.

moralsnipe · May 11, 2021, 3:31pm

Thanks! This was very helpful.

moralsnipe · May 11, 2021, 3:32pm

So I have the MQTT/HA integration turned on. All the devices show up normally. I switched one from container for a service, to another, but the old container still shows up as an entity that I can’t delete.

Should WUD be cleaning up MQTT entities when the container goes away?

fmartinou · June 3, 2021, 9:20am

When a container goes away, WUD removes it from its internal state.
But currently, there is no mqtt message sent to HA to inform that the sensor should be deleted.

According to the Mqtt discovery official doc, WUD should send an empty json message to the discovery topic to do so.

I’ve created an issue on Github to fix it.

fmartinou · June 3, 2021, 12:14pm

Should WUD be cleaning up MQTT entities when the container goes away?

It should be working with version fmartinou/whats-up-docker:develop.
(it will be integrated in the next release of WUD)