HA Core + Synology + Kemp Loadbalancer + Cloudfare - 522 Error

Level of Personal Expertise:
Beginner

Using the following applications:

  • DSM 7+
  • HA Core through Synology Docker Container
  • Kemp LoadBalancer configured to use Content Rules to point Cloudfare URL and route traffic to SynlogyIP+HAPort
  • Cloudflare with A Record to domain I would like HA to be.

Configuration.yaml

# HTTP Settings
http:
  use_x_forwarded_for: true
  trusted_proxies: 
    - IPofSynology
    - IPofKempVM
    - NetworkGateway IP
  server_port: xxxx
  login_attempts_threshold: 5
  cors_allowed_origins:
    - IPofSynology
    - IPofKempVM

Misc Notes:
Logs are clean. Bootup takes 5 seconds and nothing in there other than startup activities finishing in ‘done’.

Problem:
If I enter the IP of the SynologyIP:HAPort, it works without a problem over http. However, if I type the homeassistant.mydomain.com > I get a 522.

At the Content Rule level, Kemp gives me an OK for an HTTP GET Test on IP+Port, and I can also run a GET over Postman locally with an 200.

Goal?
I want to manage and I am currently managing all my other applications through a single Virtual Server over 443. HA is the only one struggling right now, and I assume it’s due to the breath of configurations possible on this system.

Does anyone have experience with a similar configuration?

Did you enable websocket?

On the Synology? Or…?

Mind pointing me where I should be looking? Also note that all my other applications are running from the Synology and working.

On you kemp load balancer? Dont know where you run it…

Apologies, but I’m not quite sure I am following.

This

Kemp LoadBalancer configured to use Content Rules to point Cloudfare URL and route traffic to SynlogyIP+HAPort

I get that, I just don’t follow what you mean by enabling web sockets. Unless you’re asking if Kemp has been setup to handle websocket servers.

Yes, if you have not setup websocket on the reverse proxy it will fail to connect.

Look here for some guidance:

Quick update on this. I made an oopsie at Cloudflare and had a typo on my Public IP. That is fixed, and now the following is present on my logs, with error now changing from a 522 to a 520:

File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data stdout

19:22:49 aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'" stdout

19:22:51 2022-04-23 15:22:51 ERROR (MainThread) [aiohttp.server] Error handling request stdout

19:22:51 Traceback (most recent call last): stdout

19:22:51 File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 334, in data_received stdout

19:22:51 messages, upgraded, tail = self._request_parser.feed_data(data) stdout

19:22:51 File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data stdout

19:22:51 aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"

Looks like a similar user asked this on https://community.home-assistant.io/t/ha-with-reverse-proxy-not-working/362278 without much help.

There’s a lot of talk about this error with tons of solutions. Which tells me this error is pretty generic, but all points to problems with configuration, of course…

One of the items that makes sense is that this error is caused by the container receive a request over https when only http is enabled or vice versa. My concerned is that the request is being handled by the Load Balancer first as HTTPS, and then just asking the container to show what is in HTTP (if I understand this correctly). So what is going in here?

I just fixed this. Folks that ever need this, here you go:

You need to force Home Assistant to use HTTPS. How do you do that? You NEED to make a certificate. I use Acme.sh, but feel free to use anything you want. The certificate is honestly just for decoration, since HA apparently doesn’t know how to handle HTTPS requests unless these files exist and the configuration.yaml has the values for fullchain and key.

Kinda silly, but once I understood the problem, the solution was simple.

Additionally, Synology has the ability to setup a Websocket under Login Options > Advance > Reverse Proxy Rules > You DO NOT need to use this. It doesn’t do anything.

Setup your Kemp Content Rules, setup your keys, setup your A record in Cloudfare or your provider of choice, and you’re done.

I hope this is helpful to someone.

… so I have this working now.

What worked for me was:

In Advanced Properties of the main VS do NOT “Add a Port 80 Redirector VS”
In the Sub-VS, under Real Servers, set the “Real Server check method” to ICMP Ping