What's up docker? How to keep your containers up to date

Depechie · September 2, 2021, 5:33pm

Good suggestion… maybe a final thing though. Adding those single containers in the services of the WUD compose, will actually also create those docker containers. ( so an extra calibre-web is being created )
I don’t seem to see the option to not create and watch a previous created running container? Or is that not how we should use WUD?
( Sorry for being a noob )

fmartinou · September 2, 2021, 5:49pm

I’m not sure to understand

I don’t suggest to run an additional container but to recreate your existing containers with the appropriate options.

Either
you add a wud.watch.digest=true on your containers and you keep working with latest tags

Or
you change the tag to use semver tags (like0.6.12-ls129) and add a wud.tag.include label to filter what tags to consider on the Hub.

For both cases, when you’ll run Docker Compose, it will recreate all the containers containing changes.

If you don’t want to lost data, please check before you have mounted external volumes (but I’m pretty sure you did).

Depechie · September 2, 2021, 6:02pm

Ow I now see the confusion. My other containers are all setup through portainer UI and not in a compose file.
Only WUD is in a compose file.

I will add the extra label for the other containers in portainer

fmartinou · September 2, 2021, 6:26pm

Ok, I get it now

I will add the extra label for the other containers in portainer

Absolutely, that’s the right thing to do

fmartinou · September 2, 2021, 6:28pm

Why is wud the only container defined in Docker Compose?
You should be able to define it through Portainer as the others no?

Maybe none of my business…

Depechie · September 2, 2021, 7:38pm

No that is a good question it’s because of : Portainer v2.6.2 - unable to start containers

KingRichard · September 6, 2021, 1:05pm

@mwolter @fmartinou providing certs to Debian/RPI seems to be very easy with scripts by mwolter ← Special thanks. Now I need to do the same thing for Synology. Any idea on how to do it?

fmartinou · September 6, 2021, 1:30pm

Sorry I don’t.
Can’t you access your Nas with ssh?

I just googled that… I don’t know if it’s relevant…

KingRichard · September 6, 2021, 3:32pm

unsure but using docker-socket-proxy is getting very close to have it running. Currently, my problem is most probably on the certificate error on the server side. Still trying to make it work. @mwolter create a greater script but on for client-side and not from the server-side. Trying to add but somehow it keeps saying my IP is not on the list.

Will share the logs and if using docker-socket-proxy work, it will make very easy install to all machine. So got my finger cross. I’m using this ranrinc/docker-socket-proxy: Secure Containerized Socat Proxy for the Docker Unix Socket (github.com)

KingRichard · September 7, 2021, 5:04am

UPDATE:

For those who would like to implement @mwolter scripts for Synology, I make some adjustments for implementing it. However, it works and you guys can just follow my guide at KRDesigns.com

Again special thanks to @fmartinou for WUD (must-have for docker user) and @mwolter for his remarkable LTS certs scripts and implementations which will work for WUD and Docker Monitor.

Who is this for?
One with multiple machines + docker running which allow you to install single WUD and single Portainer yet you will have the ability to remote and control the rest of your docker.

mwolter · September 12, 2021, 3:13pm

@fmartinou Curious why there are logs from each host for a single container that needs updating?

In this instance ‘container=frigate-lxc_frigate’. Frigate does need to be updated but this container is only on one host ‘frigate-lxc’. Why is there a line from each watcher for this one container?

This is clogging up the logs and makes them difficult to review. There are six containers that need updating. Instead of there being six lines (one for each container), there are 54 (one for each container and host).

07:00:01.616  INFO whats-up-docker/watcher.docker.home-assistant-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.616  INFO whats-up-docker/watcher.docker.timescaledb-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.616  INFO whats-up-docker/watcher.docker.prometheus-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.615  INFO whats-up-docker/watcher.docker.utility-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.615  INFO whats-up-docker/watcher.docker.node-red-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.614  INFO whats-up-docker/watcher.docker.grafana-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.614  INFO whats-up-docker/watcher.docker.frigate-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.613  INFO whats-up-docker/watcher.docker.gateway-pi: Update available (container=frigate-lxc_frigate)
07:00:01.613  INFO whats-up-docker/watcher.docker.pihole-lxc: Update available (container=frigate-lxc_frigate)
07:00:01.612  INFO whats-up-docker/watcher.docker.loki-lxc: Update available (container=frigate-lxc_frigate)

fmartinou · September 12, 2021, 3:36pm

Hi @mwolter

Why is there a line from each watcher for this one container?

Looks like a weird bug .
Does that happen only with logs?
Or triggers are also activated 54 times instead of 6 times?

Can you create an issue on GitHub, please? I’ll take a look.

mwolter · September 12, 2021, 3:37pm

Not sure if it’s affecting triggers. Primarily monitor updates via Prometheus. Sure, I’ll start an issue there.

fmartinou · September 15, 2021, 9:23am

Hi!

I would like to share with you some interesting features you may not know (because added over time).

Any feedbacks are welcome!

1. Authentication
WUD allows, by default, anonymous access.
If you want to expose it publicly, it can be an issue (except if you handle authentication on your end with a Reverse Proxy for example).

Now you can directly configure authentication strategies in WUD (Basic Auth and Openid Connect are currently supported).

# Basic auth example
version: '3'

services:
  whatsupdocker:
    image: fmartinou/whats-up-docker
    ...
    environment:
      - WUD_AUTH_BASIC_JOHN_USER=john
      - WUD_AUTH_BASIC_JOHN_HASH=$$apr1$$8zDVtSAY$$62WBh9DspNbUKMZXYRsjS/
      - WUD_AUTH_BASIC_JANE_USER=jane
      - WUD_AUTH_BASIC_JANE_HASH=$$apr1$$5iyu65pm$$m/6I35fjUT7.1CMnS2w9d1
      - WUD_AUTH_BASIC_BOB_USER=bob
      - WUD_AUTH_BASIC_BOB_HASH=$apr1$$aefKbZEa$$ZSA5Y3zv9vDQOxr283NGx/

# Openid Connect example with Auth0
version: '3'

services:
  whatsupdocker:
    image: fmartinou/whats-up-docker
    ...
    environment:
      - WUD_AUTH_OIDC_AUTH0_CLIENTID=21fdsf065dfsdf05616f
      - WUD_AUTH_OIDC_AUTH0_CLIENTSECRET=rthjrtsd5640grtgfgh654
      - WUD_AUTH_OIDC_AUTH0_DISCOVERY=https://dev-sdfsdf8t.us.auth0.com/.well-known/openid-configuration

2. Release notes links
Let’s say that you know that Maria DB 10.4 changelog is published under
https://mariadb.com/kb/en/mariadb-1064-changelog.

So we can describe the release notes link by a string template https://mariadb.com/kb/en/mariadb-${major}${minor}${patch}-changelog

You can now define this template as a Docker label on your Container.

version: '3'

services:

  mariadb:
    image: mariadb:10.6.4
    labels:
      - wud.link.template=https://mariadb.com/kb/en/mariadb-$${major}$${minor}$${patch}-changelog

And triggers then will render the links

{
    "id":"b46d2c68ef2b3fd87b2687da8d4c6d55cf45945f79748b869c831bbb215ca039",
    "name":"nextcloud_db",
    "watcher":"local",
    "includeTags":"^\d+\.\d+\.\d+$",
    "linkTemplate":"https://mariadb.com/kb/en/mariadb-${major}${minor}${patch}-changelog",
    "image":{
        "id":"sha256:992bce5ed7107642a27af59c2a2ddb7e589a44b639342b9501044511ab00aca0",
        "registry":{
            "url":"https://registry-1.docker.io/v2",
            "name":"hub"
        },
        "name":"library/mariadb",
        "tag":{
            "value":"10.5.9",
            "semver":true
        },
        "digest":{
            "watch":false,
            "repo":"sha256:36288c675a192bd0a8a99cd6ba0780e31df85f0bfd0cbb204837cd108be3d236"
        },
        "architecture":"amd64",
        "os":"linux",
        "created":"2021-04-24T00:32:46.309Z"
    },
    "updateAvailable":true,
    "link":"https://mariadb.com/kb/en/mariadb-1059-changelog",
    "result":{
        "tag":"10.6.4",
        "link":"https://mariadb.com/kb/en/mariadb-1064-changelog"
    }
}

3. Timezone
WUD runs by default with UTC time (CRON, logs…).
If you prefer running it on your local timezone, just set on WUD the regular TZ environment variable

version: '3'

services:
  whatsupdocker:
    image: fmartinou/whats-up-docker
    ...
    environment:
      - TZ=Europe/Paris

4. Watch all containers except few ones
You want to watch for updates almost all your containers except few ones.?
This can be easily achieved with the following combination:

Configure WUD to watch all containers by default (WUD_WATCHER_{watcher_name}_WATCHBYDEFAULT=true)
Exclude the few containers from being watched by adding on them a wud.watch=false label

version: '3'

services:
  mariadb:
    image: mariadb:10.4.5
    ...
    labels:
      wud.watch=false

Depechie · September 27, 2021, 8:46pm

Hey @fmartinou I went from 5.6.0 to 5.7.2 and now some containers are shown as needing an update but the tag is pointing to the architecture instead of the digets/version?
Do we need to change the config for watching?

fmartinou · September 27, 2021, 9:19pm

Hi @Depechie,

Started from version 5.7.1, I changed the update resolution algorithm regarding non-semver tags (like latest).
Now, for them, WUD sorts registry tags alphabetically desc (z->a) and proposes the highest one as a candidate update.

Maybe I will rollback this feature because it surprised lots of users (as you were).

In the meantime, as a quick fix, you can add
wud.tag.include=^latest$ labels on your containers to prevent WUD from considering other tags than latest.

whatdaybob · December 27, 2021, 1:00pm

Hi, anyone solve an issue with number ordering?

v21.1-ls102 is the latest verion of an image but WUD keeps detecting v21.1-ls99 as the latest and I can’t wrap my head around it.

My regex is ^v\d+\.\d+\-ls\d+$$ which captures both but chronologically they are out of order.

~~I assume rolling back to 5.7.0 would solve this?~~

Edit: Just realised 5.7.0 was a while ago so doesn’t support features I am using so can’t test.

Edit2: figured it out, read up on transform tags and got this to work ^v(\d+)\.(\d+)-ls(\d+)$ => $1.$2.$3

fmartinou · December 27, 2021, 5:53pm

Hi @whatdaybob ,

The explanation is that the semver specification implies to compare as numbers major,minor and patch digits only.
Everything after is compared alphabetically so -ls99 is greater than -ls102.

Linux server tags are a hell because they are far far far away from compliant semver versions.
The solution to make it work is the one you found by yourself in Edit 2; applying a transform function first to keep only the semver compliant part.

Edit: To be more clear, your tag v21.1-ls99 is parsed by wud as:

major: 21
minor: 1
patch: 0
prerelease/alpha: ls99

whatdaybob · December 27, 2021, 6:03pm

Ha ha you aren’t joking about the naming conventions! Technically got it working now but noticed a few ui bugs, example is that the link made up using the version numbers show the current numbers and not the updates (threw me through a loop or two). Will get an issue raised when I have chance.

I will ask, using the transform method, does it break the update kind, all my others show minor/major but the ones I transformed all say unknown.

Looking back at my original one, ‘v21.1-ls101‘ I parsed out the 21.1.101 and can confirm that. But ‘v21.1-ls102‘ comes up as unknown.

fmartinou · December 27, 2021, 6:08pm

I wrote about how wud would parse it “by default” (I mean without any transform function).
Once your transform function set up, yeah, you’re right. It’s parsed as you mention it.

For the bugs you’ve detected, please open the related issues on Github.
I’ll do my best