[Solved] Access via IIS reverse proxy died after upgrade to 0.58

My remote access died after upgrading to Home Assistant 0.58. I does not pass the loading spinner.
I’m using IIS reverse proxy (ARR) with Let’s Encrypt. LAN access still works fine.

The regular procedure of cleaning the browser has no effect. Removing the http password did not help.

Failed Request Tracing from IIS shows a 502.5 error with description ‘The server returned an invalid or unrecognized response’. This is on the /api/websocket resource and after exactly 1 second.

When I downgrade to HA 0.57.3, things are OK again. Therefore, my guess is that one of the following package upgrades is causing this, or exposing a flaw in my ARR setup:

  • homeassistant 0.57.3 -> 0.58.1
  • aiohttp 2.2.5 -> 2.3.2
  • yarl 0.13.0 -> 0.14.0

I’m currently out of ideas. Any ideas where to look next?

2 Likes

Did you cleared the browser’s cache?

Thanks for the response. Cleaned the browser using the regular procedure, used another browser, used another browser on another system, but all to no avail.

My gut feeling is that the yarl or aiohttp component sends a response in a way that the IIS reverse proxy does not accept. But I currently have no idea how to go from there.

Can it be the IIS itself?

I’m having the same issue with IIS since the upgrade - I’ve opened an issue on GitHub which has had no attention so far https://github.com/home-assistant/home-assistant/issues/11082 it’s definately an issue with HASS because if I rollback to an old backup it works fine.

Finally found a solution. It appears IIS ARR does not handle websocket compression properly (see here). You can work your way around this by clearing the Sec-WebSocket-Extensions header on the incomming requests:

  1. First add the server variable to IIS manager: Your site > URL Rewrite > View Server Variables… > Add: HTTP_SEC_WEBSOCKET_EXTENSIONS

  2. Then add the variable to the inbound rule that forwards requests to Home Assistant. This is my rule in web.config:

    <rule name="Forward to Home Assistant" stopProcessing="true">
        <match url="(.*)" />
        <action type="Rewrite" url="http://localhost:8123/{R:0}" logRewrittenUrl="true" />
        <serverVariables>
            <set name="HTTP_SEC_WEBSOCKET_EXTENSIONS" value="" />
        </serverVariables>
    </rule>
    

    You can also add the variable to the inbound rule with IIS-Manager, but it forces you to enter a value. You still end up with editing web.config to clear the value.

This became an issue in 0.58 because aiohttp added support for compression extensions for websockets. This compression made the IIS ARR issue visible. It was the aiohttp 2.3.0 changelog that put me in the right direction.

I’m now running 0.59.2 without any issues :slight_smile:

5 Likes

Aha that’s fixed it. Cheers!

Thank you @grietver!

Don’t forget to add “HTTP_SEC_WEBSOCKET_EXTENSIONS” to the allowed server variables list if it complains with a 500.

1 Like

Hi everyone,

I’m having the same issue over here.
Added HTTP_SEC_WEBSOCKET_EXTENSIONS to server var and my ARR rules:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="Let's Encrypt" enabled="true" stopProcessing="true">
                    <match url="(.well-known/acme-challenge/*)" />
                    <action type="None" />
                </rule>
                <rule name="ReverseProxyInboundRule1" stopProcessing="true">
                    <match url="(.*)" />
                    <action type="Rewrite" url="http://192.168.4.82:8123/{R:0}" />
                    <serverVariables>
                        <set name="HTTP_SEC_WEBSOCKET_EXTENSIONS" value="" />
                    </serverVariables>
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>

On internal LAN, no problem to login, over IIS reverse proxy: not possible to login.
Is there something else I should configure?
Note: my ARR is a different machine than my HASS.

any help is much appreciated!

Kind regards,
Bart

Did you enable WebSocket in IIS?
Did you add HTTP_SEC_WEBSOCKET_EXTENSIONS to the site variables?

2 Likes

That did the trick!

thank you very much.
Setup HASS today. First impression of this community: 10/10! :slight_smile:

kind regards,
Bart

Hi,

I am running into similar problems. I am running IIS 10.0 and hass.io on a Pi. I was hoping the URL-rewrite rules would address my problem. From the output I am connecting on port 80, before I secure the site.

When I add the above script, correcting the local IP address and adding the HTTP_SEC_WEBSOCKET_EXTENSIONS site variables, I got a 404 error.

Any clues greatly appreciated to help me to get this working.

By side stepping IIS and port forwarding directly to hassio. Let’s Encrypt provided the certificate. Turned out to be straight forward and no ducksdns required.

IIS was really a red-herring and does no need to be part of the solution or problem.