Traefik EXTERNAL ROUTING Bad Address

dekela · January 17, 2024, 8:40pm

I am trying to migrate from ngnix reverse proxy to traefik.
My HA is on a raspberry pi and traaefik is running as a docker container on a different machine (Using Unraid) . I configured external routing in traefik and I am getting Bad Request.

Looking at HA log I see tthe following:
Invalid IP address in X-Forwarded-For: , 85.65.x.x

I have configured:
http:
use_x_forwarded_for: true
trusted_proxies:

Does anyone run the same setup and can help me?

Thanks
Dekel

_dev_null · January 17, 2024, 10:31pm

This doesn’t look good at all, normally the x-forwarded-for header contains a , separated list of proxies you have been through so it looks like the header hasn’t been sanitised correctly

dekela · January 18, 2024, 4:36pm

Agree, I think II even found a bug for something similar:

github.com/home-assistant/core

Proxy IP taken instead of the client one if X-Forwarded-For contains a list

opened 12:58PM - 04 Jan 24 UTC

numo68

### The problem My installation is fronted by a reverse proxy (traefik) that is… further proxied by Cloudflare, with the usual ``` http: use_x_forwarded_for: true trusted_proxies: - 192.168.xx.yy/24 ``` The HTTP request headers arriving at the Home Assistant are ``` Cf-Connecting-Ip: MY-CLIENT-IP X-Forwarded-For: MY-CLIENT-IP, CLOUDFLARE-IP X-Real-Ip: MY-CLIENT-IP ``` If I try to log with invalid credentials, the log however contains ``` 2024-01-04 12:18:55.919 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from CLOUDFLARE-IP (CLOUDFLARE-IP) ``` meaning that while the `X-Forwarded-For` header is trusted, the incorrect component of it is taken. The `X-Forwarded-For` header has no formal specification and Home Assistant is by far not the only software that gets it wrong; the commonly used definition however does allow for a list and sorts the addresses real-client-first (see e.g. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For ). Proposal: - either parse and use the first address of the list - or - better - allow to define the trusted header name itself, instead of a boolean `use_x_forwarded_for`. I am imagining something like `real_ip_from: X-Real-Ip`. Thanks ### What version of Home Assistant Core has the issue? core-2023.12.4 ### What was the last working version of Home Assistant Core? _No response_ ### What type of installation are you running? Home Assistant Container ### Integration causing the issue _No response_ ### Link to integration documentation on our website _No response_ ### Diagnostics information _No response_ ### Example YAML snippet _No response_ ### Anything in the logs that might be useful for us? _No response_ ### Additional information _No response_

dekela · January 18, 2024, 4:38pm

Maybe I can run TCP dump on HA to see the request headers.

_dev_null · January 18, 2024, 5:18pm

Do it, I personally think the header is mangled prior to reaching HA

Or can you point traefik to a web page that simply dumps all the request headers or even simpler run an echo server on port 2500

while true ; do nc -l -p 2500 ; done

dekela · January 22, 2024, 6:26am


~ # while true ; do nc -l -p 2500 ; done

GET /lovelace HTTP/1.1
Host: 192.168.x.x:2500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Referer: https://host.domain/lovelace
Sec-Ch-Ua: "Not_A Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Upgrade-Insecure-Requests: 1
X-Forwarded-For: , 85.65.x.x
X-Forwarded-Host: 192.168.x.x
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 6342128c49cf
X-Real-Ip:

Take a look at the X-Forwarded-For:
I think there’s a bug in Traefik request headers which adds “,”

This is from a ngnix reverse proxy:

GET / HTTP/1.1
Host: host.domain
X-Forwarded-Scheme: https
X-Forwarded-Proto: https
X-Forwarded-For: 85.65.x.x
X-Real-IP: 85.65.x.x
sec-ch-ua: "Not_A Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
sec-ch-ua-mobile: ?0
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
sec-ch-ua-platform: "Windows"
sec-fetch-site: same-origin
sec-fetch-mode: navigate
sec-fetch-dest: empty
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9

_dev_null · January 22, 2024, 7:51am

Yup

Use a different proxy if you can