Alexa Media Player behind SWAG/Nginx Reverse Proxy

I recently started migrating to a new server running HA in a docker container behind a SWAG/Nginx reverse proxy. I’ve had some issue that took some work to straighten out I’ve done pretty well until I got to reconnecting to the Amazon Alexa world. My first issue was timeout issues from my lambda function while setting up Haaska. After digging around for a while I found I needed to increase the timeout to 3 seconds and that allowed the device discovery time to complete.

Next up was configuring the Alexa Media Player component. I’ve struggled greatly trying to get this one working and it’s been a pretty painful experience to say the least; particularly being I’ve had all of this up and running rock solid on another server for ages without any issues.

Before getting into what is now clearly a reverse proxy related issue, I can confirm that’s the case because I created a port from the HA docker container to the localhost and then setup a port forward on my router to allow outside access I’m able to setup Alexa Media Player without issue when I use https://domain.ext:port as the URL. This seemed to prove, to me at least, that it’s an Nginx proxy issue.

Once setup though it doesn’t actually function because everything else is configured to use my proxied domain that I’ve configured. For example I can tell Alexa to turn on a particular light and that works fine. But if I ask for something that requires Alexa to respond with data from HA it fails. An example would be “Where is Kevin?” causes Alexa to “ding” like it does anytime, but it never feeds back the “Kevin has been at home since…” response which was previously configured and working fine on the old server.

The problem when setting up Alexa Media Player via the proxy domain seems similar to what was happening when Haaska wasn’t getting enough time for a response. I was getting a lot of “a client request body is buffered to a temporary file…” messages in the nginx error log followed by a large number of “epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while reading response header from upstream” errors. Digging around online I solved the buffering issues using some settings in the Home Assistant proxy domain configuration:

    location ~ ^/(api|local|media)/ {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app homeassistant;
        set $upstream_port 8123;
        set $upstream_proto https;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

       ####  Settings added to fix buffering issue  ####
        client_body_buffer_size 4096m;
        client_max_body_size 4096m;
        proxy_max_temp_file_size 4096m;
        proxy_temp_file_write_size 128k;
        proxy_request_buffering off;
    }

But after this it still is failing and generating a slew of the “client prematurely closed connection” errors.

On the HA side it appears to start the process, but then crashes (being the proxy connection dies):

2023-01-09 18:13:55.304 DEBUG (MainThread) [custom_components.alexa_media.helpers] [email protected]: Returning uuid {'uuid': '99bbXXXXXXXXXX479a', 'index': 0}
2023-01-09 18:13:55.416 DEBUG (MainThread) [custom_components.alexa_media.config_flow] Generating OTP from 463860
2023-01-09 18:14:03.146 DEBUG (MainThread) [custom_components.alexa_media.config_flow] Starting proxy for [email protected] - amazon.com
2023-01-09 18:14:03.234 DEBUG (MainThread) [custom_components.alexa_media.config_flow] Found flow_id; adding 192.168.1.1 to known_ips for 300 seconds
2023-01-09 18:14:11.607 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 130, in _call_sslobject_method
    result = func(*args)
  File "/usr/local/lib/python3.10/ssl.py", line 917, in read
    v = self._sslobj.read(len)
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2548)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 33, in read
    return await self._stream.receive(max_bytes=max_bytes)
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 195, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 137, in _call_sslobject_method
    data = await self.transport_stream.receive()
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1265, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 31, in read
    with anyio.fail_after(timeout):
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
    raise TimeoutError
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 112, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 91, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 191, in _receive_event
    data = await self._network_stream.read(
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 30, in read
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc)
httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 81, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/config/custom_components/alexa_media/config_flow.py", line 883, in wrapped
    return await cls.handler(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/authcaptureproxy/auth_capture_proxy.py", line 375, in all_handler
    resp = await getattr(self.session, method)(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout

I’ve spent a lot of time trying to solve this, but unfortunately I seem to have hit the wall and I’m hoping somebody else might have some ideas to help me resolve this issue.

1 Like