HA on Docker-Compose with Traefik problem

Configuration
1

HI,

I struggling to get Traefik (reverse proxy) to work. I think i read almost every post about it that i could search on google, and tried every possible configuration (i think :slight_smile: ). Whatever i’m doig i get the “Bad Gateway” error when try to enter my HA instance throgh domain. The LetsEncrypt cert creation is OK. Also if i enter the HA with local 192.168…8123 it works. Here is my docker-compose.yaml conf:

version: "3.3"

services:
  home-assistant:
    restart: always
    container_name: homeassistant
    image: homeassistant/raspberrypi4-64-homeassistant:stable
    environment:
      - TZ=Europe/Warsaw
    volumes:
    - /mukeebox:/config
    - /etc/localtime:/etc/localtime:ro
    labels:
    - "traefik.enable=true"
    - "traefik.http.routers.homeassistant.rule=Host(`my_domain`)"
    - "traefik.http.routers.homeassistant.tls=true"
    - "traefik.http.routers.homeassistant.entrypoints=websecure"
    - "traefik.http.routers.homeassistant.tls.certresolver=myresolver"
    - "traefik.http.services.homeassistant.loadbalancer.server.port=8123"
    network_mode: "host"


  reverse-proxy:
    restart: always
    container_name: "traefik"
    image: traefik:v2.0
    ports:
    - "443:443"
    - "8080:8080"
    volumes:
    - "/var/run/docker.sock:/var/run/docker.sock:ro"
    - "./letsencrypt:/letsencrypt"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.tlschallenge=true"
      - "[email protected]"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    extra_hosts:
      - host.docker.internal:172.17.0.1

volumes:
  letsencrypt:
  homeassistant:

Can anyone help ?

2

The logs of the traefik container don’t say anything?

3

this is what i found in log related to it

time="2021-06-24T13:28:40Z" level=debug msg="'502 Bad Gateway' caused by: dial tcp 127.0.0.1:8123: connect: connection refused"
time="2021-06-24T13:28:40Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7\"],\"Cache-Control\":[\"max-age=0\"],\"Sec-Ch-Ua\":[\"\\\" Not;A Brand\\\";v=\\\"99\\\", \\\"Google Chrome\\\";v=\\\"91\\\", \\\"Chromium\\\";v=\\\"91\\\"\"],\"Sec-Ch-Ua-Mobile\":[\"?0\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36\"],\"X-Forwarded-Host\":[\"mydomainl\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"b481345d1f50\"],\"X-Real-Ip\":[\"188.122.0.174\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"mydomainl\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"188.122.0.174:58236\",\"RequestURI\":\"/\",\"TLS\":null}"

but I’m to newbie with this to tell if it means something…

4

127.0.0.1 is wrong, here. That’s from the traefik container perspective, so wrong.
I guess that should be 172.17.0.1 but I don’t know how the traefik container works.

5

According to Using --add-host or extra_hosts with docker-compose - Stack Overflow , you might want to put your extra_host between double-quotes, but not sure if it’s required.

6

Tried:

  • changing extra hosts to
extra_hosts:
      - host.docker.internal:127.0.0.1
  • putting the host.docker.internal:172.17.0.1 as well as 127.0.0.1 inside double-quotes.

No progress :confused:
There is a lot of topics regarding this problem but any solution is working.

Also when i type in host side:
http://127.0.0.1:8123 it connects to HA instance.

7

You have to understand that it’s the traefik container (likely with a 172.17.0.x address) that must communicate with your HA, which is 172.17.0.1 due to host networking.
So 127.0.0.1 is wrong, anywhere.

Try to enter the traefik container and do a ping host.docker.internal. For your setup to work, it should work and point to 172.17.0.1.

That’s assuming your docker network is similar to

docker0   Link encap:Ethernet  HWaddr 02:42:05:xx:xx:xx
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
8

mine is the same as you wrote

docker0: inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255

tried to enter the Traefik console but can’t. I get:

root@mukeebox:/srv# docker exec -it ce1a9d838a6a bash
OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown

I checked in Portainer what is the IP of Traefik container and it states 192.168.96.2, so I assume that’s a problem. Each time i up and down the docker-compose, Traefik gets different IP.

9

Didnt manage to solve. Does anyone have some clue ?

10

Problem is with the

network_mode: host

because when i remove this line and put:

    ports:
      - "8123:8123" 
    expose:
      - 8123

it is working.
Dont know why because everyone on web is using host mode, and its working for them :confused:

Want to use host mode because of the Discovery feature.

11

Could you show the output of docker inspect traefik, please

12

here it is:

[
    {
        "Id": "107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258",
        "Created": "2021-06-25T08:26:28.556596438Z",
        "Path": "/entrypoint.sh",
        "Args": [
            "--log.level=DEBUG",
            "--api.insecure=true",
            "--providers.docker=true",
            "--providers.docker.exposedbydefault=false",
            "--entrypoints.web.address=:80",
            "--entrypoints.websecure.address=:443",
            "--certificatesresolvers.siwik.acme.tlschallenge=true",
            "--certificatesresolvers.siwik.acme.email=wachowski.marekwach@gmail.com",
            "--certificatesresolvers.siwik.acme.storage=/letsencrypt/acme.json"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 3132,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2021-06-25T08:26:30.544012425Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:b697096430455552458fb67ce56db5eae3831c8fd819496003051b4ebc9199a1",
        "ResolvConfPath": "/var/lib/docker/containers/107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258/hostname",
        "HostsPath": "/var/lib/docker/containers/107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258/hosts",
        "LogPath": "/var/lib/docker/containers/107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258/107b1ba28d564cb141d001dab06a3c7a63802d119edc3852011034632d86a258-json.log",
        "Name": "/traefik",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/srv/letsencrypt:/letsencrypt:rw",
                "/var/run/docker.sock:/var/run/docker.sock:ro"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "srv_default",
            "PortBindings": {
                "443/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "443"
                    }
                ],
                "8080/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8080"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": [
                "host.docker.internal:172.17.0.1"
            ],
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/4b98159f561727f5cd7a33b4158335c245645670389e85808b680a9a150d402e-init/diff:/var/lib/docker/overlay2/4456e6c0dfad18c0c9f1ea4aca192a814cab19e8d033db95764d07519bcc9d21/diff:/var/lib/docker/overlay2/733979d0245325fc1efbada29b8f4439205608b0d747aa57458773d9d467aa18/diff:/var/lib/docker/overlay2/69b05423886a8eef0ceb8479c728c0d391d3e7e787d33d8a31ef526652fc81de/diff:/var/lib/docker/overlay2/3c073b3fd75360fe27c24cfadfb2a59fe2d8dde0a9c51e9f6250a939cfd64bbd/diff",
                "MergedDir": "/var/lib/docker/overlay2/4b98159f561727f5cd7a33b4158335c245645670389e85808b680a9a150d402e/merged",
                "UpperDir": "/var/lib/docker/overlay2/4b98159f561727f5cd7a33b4158335c245645670389e85808b680a9a150d402e/diff",
                "WorkDir": "/var/lib/docker/overlay2/4b98159f561727f5cd7a33b4158335c245645670389e85808b680a9a150d402e/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/srv/letsencrypt",
                "Destination": "/letsencrypt",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "107b1ba28d56",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "80/tcp": {},
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "--log.level=DEBUG",
                "--api.insecure=true",
                "--providers.docker=true",
                "--providers.docker.exposedbydefault=false",
                "--entrypoints.web.address=:80",
                "--entrypoints.websecure.address=:443",
                "--certificatesresolvers.siwik.acme.tlschallenge=true",
                "--certificatesresolvers.siwik.acme.email=wachowski.marekwach@gmail.com",
                "--certificatesresolvers.siwik.acme.storage=/letsencrypt/acme.json"
            ],
            "Image": "traefik:v2.0",
            "Volumes": {
                "/letsencrypt": {},
                "/var/run/docker.sock": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "/entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "5587495992fa1d36428f5bd193eba56d7257371235112310589ea3c48a6d5df7",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "srv",
                "com.docker.compose.project.config_files": "docker-compose.yaml",
                "com.docker.compose.project.working_dir": "/srv",
                "com.docker.compose.service": "reverse-proxy",
                "com.docker.compose.version": "1.29.2",
                "org.opencontainers.image.description": "A modern reverse-proxy",
                "org.opencontainers.image.documentation": "https://docs.traefik.io",
                "org.opencontainers.image.title": "Traefik",
                "org.opencontainers.image.url": "https://traefik.io",
                "org.opencontainers.image.vendor": "Containous",
                "org.opencontainers.image.version": "v2.0.7"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "471fff2090a1ff2dd6a44060de14b6b39047e1080b117e932057ccfb4bfd50c9",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "443/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "443"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "443"
                    }
                ],
                "80/tcp": null,
                "8080/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8080"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "8080"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/471fff2090a1",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "srv_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "107b1ba28d56",
                        "reverse-proxy"
                    ],
                    "NetworkID": "d909c5d9dec1364e54771646a575138fbd0146fe6e1e9491959de3c76e56fb28",
                    "EndpointID": "0045078daa025b52b129994ffd0c9159075a069841b8dca7015d7840b6ee04c7",
                    "Gateway": "172.19.0.1",
                    "IPAddress": "172.19.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:13:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
13

The gateway (i.e. host) of traefik is 172.19.0.1 for whatever reason, so that’s the address you should mention in extra_hosts

14

each time i restart docker-compose (down → up) i get different IP and Gateway so i edited the docker-compose.yaml:

added to traefik:

networks
  test:
    ip4_address: 172.19.0.2

and

extra hosts:
  - host.docker.internal: 172.19.0.1

and added the custom network

networks:
  test:
    ipam:
       driver: default
       config:
         - subnet: 172.19.0.0/16

But…

As the HA is in the Host mode i tried to put the Traefik at Host mode as well.
Deleted:

    ports:
      - "8123:8123"
    expose:
      - 8123

and added

    network_mode: "host"

Now it is working :smile: . Dont know if it good configuration from the security perspective though. But assume if I’am behind the router which is taking care of port forwarding - I’m safe.

@koying - thanks for you time and help.