500 Errors trying to load Node-Red or ZwaveJS AddON UIs

After updating to Core 2023.9.1 I cannot open the zwavejs UI nor the NodeRed UI any longer.

Sticking to 2023.8.4 works, and if I roll back to 8.4 it works too.

Where would I look to fix this?

Both those addons are fully up to date, so I don’t think it’s simply that. I don’t see any 500 errors in the logs of the addons themselves, nor do I see any Error messages in the HomeAssistant Core logs.

This is the second time I’ve attempted to move up from 2023.8.4 to the .9.x series.

Might need to clear browser cache or hard refresh it. Use the browser dev tools to see if any errors are reported in the console.

No, no javascript errors, only the backend reporting a 500 error. Hard refreshes, private mode, none of those make any difference.

Have also fully rebooted the homeassistant installation, and it is still broke.

Rolling back to 2023.8.4 makes things work again.

The URL you blurred might be important. I don’t know if there’s anything private there, but can you show at least the part that doesn’t appear unique?

Well, maybe there’s nothing else at the URL. I see a similar URL but get 200.

{
	"GET": {
		"scheme": "http",
		"host": "192.168.X.X:8123",
		"filename": "/api/hassio_ingress/iQJ_3qDkiGLiM31r7YFO_Ryev6vCBGfpTDCuETtWxqo/",
		"remote": {
			"Address": "192.168.X.X:8123"
		}
	}
}
{
	"Status": "200OK",
	"Version": "HTTP/1.1",
	"Transferred": "1.12 kB (1.68 kB size)",
	"Referrer Policy": "strict-origin-when-cross-origin",
	"DNS Resolution": "System"
}

Ooh, I saw a different error just once. I got an “unable to create an ingress session” dialog, when clicking on node-red. Perhaps it hadn’t fully started…

I wasn’t able to get that again with a refresh, but I have this:

Zero of my addon WebUI’s work. I also can’t get the SSH web UI one to load.

AHA, found the logs, they’re in the supervisor logs:

23-09-12 18:04:03 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/aiohttp.py", line 139, in sentry_app_handle
    reraise(*_capture_exception(hub))
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/_compat.py", line 60, in reraise
    raise value
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/aiohttp.py", line 129, in sentry_app_handle
    response = await old_handle(self, request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 185, in block_bad_requests
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 201, in system_validation
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 220, in token_validation
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 280, in core_proxy
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/ingress.py", line 141, in handler
    return await self._handle_request(request, addon, path, session_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/ingress.py", line 217, in _handle_request
    async with self.sys_websession.request(
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 558, in _request
    resp = await req.send(conn)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 670, in send
    await writer.write_headers(status_line, self.headers)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 129, in write_headers
    buf = _serialize_headers(status_line, headers)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_writer.pyx", line 132, in aiohttp._http_writer._serialize_headers
    _safe_header(to_str(val))
  File "aiohttp/_http_writer.pyx", line 109, in aiohttp._http_writer.to_str
    raise TypeError("Cannot serialize non-str key {!r}".format(s))
TypeError: Cannot serialize non-str key None

This doesn’t show up in the logs for the older version.

Oh, that’s more helpful. That lead me to https://github.com/home-assistant/supervisor/pull/4551.

Are you using Command Line Authentication? https://github.com/home-assistant/supervisor/issues/4547#issuecomment-1712954536

Yep! I am using CLI authentication!

Since that’s merged, I assume that’ll show up in 2023.9.2 ?

Oop: found this Ingress for Add-ons broken after update to 2023.9.0 · Issue #99811 · home-assistant/core · GitHub, which seems to lead to .4.

The fix seems to be for Supervisor, not HA. The PR I referenced is part of Release 2023.09.1 · home-assistant/supervisor · GitHub.

What version is your Supervisor?

Here’s my supervisor info:

➜  ha supervisor info
addons:
- icon: true
  name: Mosquitto broker
  repository: core
  slug: core_mosquitto
  state: started
  update_available: false
  version: 6.3.1
  version_latest: 6.3.1
- icon: true
  name: Z-Wave JS UI
  repository: a0d7b954
  slug: a0d7b954_zwavejs2mqtt
  state: started
  update_available: false
  version: 1.15.11
  version_latest: 1.15.11
- icon: true
  name: Advanced SSH & Web Terminal
  repository: a0d7b954
  slug: a0d7b954_ssh
  state: started
  update_available: false
  version: 15.0.7
  version_latest: 15.0.7
- icon: true
  name: Node-RED
  repository: a0d7b954
  slug: a0d7b954_nodered
  state: started
  update_available: false
  version: 14.4.5
  version_latest: 14.4.5
- icon: true
  name: Studio Code Server
  repository: a0d7b954
  slug: a0d7b954_vscode
  state: unknown
  update_available: false
  version: 5.10.1
  version_latest: 5.10.1
- icon: true
  name: ESPHome
  repository: 5c53de3b
  slug: 5c53de3b_esphome
  state: started
  update_available: false
  version: 2023.8.3
  version_latest: 2023.8.3
addons_repositories:
- name: Local add-ons
  slug: local
- name: Logan Marchione Home Assistant Add-Ons
  slug: 0d869efa
- name: ESPHome
  slug: 5c53de3b
- name: Home Assistant Community Add-ons
  slug: a0d7b954
- name: Official add-ons
  slug: core
arch: amd64
auto_update: true
channel: stable
debug: false
debug_block: false
diagnostics: true
healthy: true
ip_address: 172.30.32.2
logging: info
supported: true
timezone: America/Chicago
update_available: false
version: 2023.08.3
version_latest: 2023.08.3
wait_boot: 5

I don’t use HAOS, but run it occasionally for testing. I currently have all the latest versions:

image

It is also showing Supervisor 2023.08.3. Checking for updates doesn’t show anything. I don’t have a clue how Supervisor updates are pushed out, but you might just have to wait for a new version.

Yeah the issue I found seems to indicate that I need to wait for Core 2023.9 or opt in to a beta at the moment.

All the stuff works, just not the UI, I might just leave it be for now, until the new supervisor is shipped, or perhaps rollback if I really need the UI.

Thanks for looking into it with me!

Can you access the add-on settings? You could try exposing port 8091 and accessing ZUI directly (host_ip:8091), instead of using ingress.

Oh that’s a good point, I didn’t think about accessing it directly.

Either way, they’ve shipped an update to the thing that resolves it now :smiley: They fixed a small bug in the supervisor regarding user identification, and now it works just fine!