Caddy / reverse proxy, docker, and firewalls. Oh my!

Configuration
1

I previously had a working hass.io configuration, but decided to start over using Docker so I could better understand how everything worked. I’ve had casual flings with Docker before, but haven’t delved too deeply into it. Now I’m running into some issues. So, some background.

Some context:
I’d like to use Caddy (docker container) as a reverse proxy to serve hass and portainer through a dynamic dns on my rPi3. I tried this Caddy docker-compose file:

version: '3.5'

services:
  caddy:
    container_name: caddy
    image: orbsmiv/caddy-rpi
    volumes:
      - ./certs:/root/.caddy
      - ./caddyfile:/etc/Caddyfile
    ports:
      - "80:80"
      - "443:443"

with the following caddyfile:

home.id.duckdns.org {
  proxy / localhost:8123 { websocket transparent }
}

portainer.id.duckdns.org {
  proxy / localhost:9000 { websocket transparent }
}

with the following hass configuration.yaml:

http:
  base_url: https://home.id.duckdns.org
  use_x_forwarded_for: true
  trusted_proxies:
    - 127.0.0.1
    - ::1

I was able to successfully connect to the reverse proxy with this setup, but Caddy couldn’t connect to either portainer or hass. So, I tried using my device’s LAN IP instead:

home.id.duckdns.org {
  proxy / 192.168.0.xxx:8123 { websocket transparent }
}

portainer.id.duckdns.org {
  proxy / 192.168.0.xxx:9000 { websocket transparent }
}

This actually worked okay (I think).

The problem

I’d like to turn on a firewall for my device. I’m using UFW and set up the following config (with some stuff trimmed out for brevity):

Default: deny (incoming), allow (outgoing), deny (routed)

80/tcp ALLOW IN Anywhere
443/tcp ALLOW IN Anywhere

Unfortunately this broke my previous setup. I’m not sure I understand the problem fully, but I believe it’s because I’m using my LAN IP for the reverse proxy, so the firewall is blocking it as an incoming connection on a non-whitelisted port.

So I went down a rabbit trail and started experimenting with networks. I tried the following for my Caddy docker compose config:

version: '3.5'

services:
  caddy:
    container_name: caddy
    image: orbsmiv/caddy-rpi
    volumes:
      - ./certs:/root/.caddy
      - ./caddyfile:/etc/Caddyfile
    ports:
      - "80:80"
      - "443:443"
    networks:
      - internal_bridge

networks:
  internal_bridge:
    driver: bridge
    driver_opts:
      com.docker.network.bridge.name: internal_bridge

Then I tried to whitelist connections on that network in ufw:

sudo ufw allow from any to internal_bridge port 8123

which resulted in the following ufw config:

8123 on internal_bridge    ALLOW IN    Anywhere                  
9000 on internal_bridge    ALLOW IN    Anywhere

This…worked. But I’m not sure I understand what I’m doing, and I don’t want to compromise security.

I also seem to be unable to add my LIFX bulbs. I disabled my firewall and it worked. So what I’m realizing is that I must be doing something wrong with my firewall and docker setup. Can anyone guide me out of this rabbit trail I’ve stumbled onto? I think I’m over-complicating things but I don’t know what I’m doing wrong.