2023.9: New climate entity dialogs, lots of tile features, and template sensors from the UI!

TimoJ · September 7, 2023, 7:45am

I tried to install the 2023.9.0 update (using HAOS Intel PC) but it did not work. It didn’t restart or make any changes. I tried again, same thing. Then I restarted HA and now I get error in Supervisor “failed to set up”. I restored fullback but still the same problem, restored another full backup from last night and still the same. The system was working fine and HA restarted without issues yesterday. So what now? How can I fix this?

Supervisor debug log shows this:

2023-09-07 10:37:23.285 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration ssh_command which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-09-07 10:37:23.285 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration browser_mod which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-09-07 10:37:23.286 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration smartir which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-09-07 10:37:23.286 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration nodered which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-09-07 10:38:23.735 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: hassio
2023-09-07 10:39:23.801 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: hassio
2023-09-07 10:39:24.078 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Supervisor for hassio
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 388, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/hassio/__init__.py", line 651, in async_setup_entry
    await coordinator.async_config_entry_first_refresh()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 251, in async_config_entry_first_refresh
    await self._async_refresh(
  File "/usr/src/homeassistant/homeassistant/components/hassio/__init__.py", line 959, in _async_refresh
    await self.hassio.refresh_updates()
  File "/usr/src/homeassistant/homeassistant/components/hassio/handler.py", line 36, in _wrapper
    data = await funct(*argv, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/hassio/handler.py", line 534, in send_command
    request = await self.websession.request(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 899, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
asyncio.exceptions.CancelledError: Global task timeout
2023-09-07 10:39:24.085 WARNING (MainThread) [homeassistant.bootstrap] Setup timed out for stage 1 - moving forward

Gav_in · September 7, 2023, 7:46am

Cheers…every day a school day

tom_l · September 7, 2023, 7:50am

Actually I remembered incorrectly, you need to add a version number to the manifest file, see: No 'version' key in the manifest file for custom integration 'environment_canada - #7 by tom_l

Mariusthvdb · September 7, 2023, 7:57am

just to decide whether this is a bug or not:

As Buienradar integration only supports a daily forecast, shouldnt the options for Hourly and twice daily be hidden?

Currently it returns the red error, would be nice if these simply wouldn’t be presented.

btw, note the Dagelijks (daily) is translated, and the other 2 aren’t. Might be an indication already for their availability?

Ildar_Gabdullin · September 7, 2023, 8:16am

Preview - may be done by Dev tools - Templates in most cases (exceptions - using secrets and anchors, using “this” variable - but it may be easily sorted out).
And you do not need to restart - just reload templates.

Can you use yaml anchors in UI for repeating code? No.
Can you use secrets? Not sure.
Can you keep a set of 10 sensors in a package along with helpers and automations and easily exclude it from HA ? No.
So, - this is not as advanced as yaml.

itn3rd77 · September 7, 2023, 8:19am

Can anybody confirm that File editor add-on is broken? I get an “500 Internal Server Error” inside UI and a bunch of exceptions in the supervisor log:

23-09-07 10:17:49 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/aiohttp.py", line 139, in sentry_app_handle
    reraise(*_capture_exception(hub))
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/_compat.py", line 60, in reraise
    raise value
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/aiohttp.py", line 129, in sentry_app_handle
    response = await old_handle(self, request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 185, in block_bad_requests
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 201, in system_validation
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 220, in token_validation
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 280, in core_proxy
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/ingress.py", line 141, in handler
    return await self._handle_request(request, addon, path, session_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/api/ingress.py", line 217, in _handle_request
    async with self.sys_websession.request(
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 558, in _request
    resp = await req.send(conn)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 670, in send
    await writer.write_headers(status_line, self.headers)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 129, in write_headers
    buf = _serialize_headers(status_line, headers)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_writer.pyx", line 132, in aiohttp._http_writer._serialize_headers
    _safe_header(to_str(val))
  File "aiohttp/_http_writer.pyx", line 109, in aiohttp._http_writer.to_str
    raise TypeError("Cannot serialize non-str key {!r}".format(s))
TypeError: Cannot serialize non-str key None

Ildar_Gabdullin · September 7, 2023, 8:26am

Templates cannot be created without programming. Maybe in future UI you will have possibilities for operations like “calc a sum for a list of entities” (like it is available for groups).
Ofc, a trend to “simplify” should not cause “remove a functionality “.

Steve61 · September 7, 2023, 8:33am

I just went and tried and yes I can now change the source entity via the GUI. My mind might be going but I am sure that when I set up some utility meters a few months ago - and then changed the naming conventions on source sensors, I had to dig into the core.config file.

I have learned something new today! Thank you.

lassej01 · September 7, 2023, 8:35am

Agree. Had to go back to previous version. Seems like modbus terminates after a few minutes.

nickrout · September 7, 2023, 8:36am

It is python, no compiling needed.

nickrout · September 7, 2023, 8:42am

I can confirm it is NOT broken for me.

tom_l · September 7, 2023, 8:55am

All you have to do is copy the repository to your custom component folder, preserving any sub-folder structure, add any version number to the manifest file (see my post above) then restart HA.

robertwigley · September 7, 2023, 9:07am

Are you on 3.17.3 or 3.1.7? Have you upgraded and it no longer works, or just looking at the breaking changes at this point?

Can you curl http://envoy.local/api/v1/production from the command line?

For me it returns the same data as the integration, so I am assuming that 3.17.3 is still supported as 17 > 9 i.e. it is 17 and not 1 and 7. They cited needing to use web scraping for versions below 3.9 as the reason for withdrawing support, so I suspect that this will still work after upgrading, although I haven’t upgraded yet and if it doesn’t it should be possible to create some RESTful sensors to return the same data as the integration, which I am actually quite tempted to do anyway, as it is then not reliant on the integration.

You can check your firmware version by going to http://envoy.local

mhoogenbosch · September 7, 2023, 9:35am

Right, thanks for clarifying. For me, it works fine. Hopefully it’ll work for you too maybe with a few iterations.

Ildar_Gabdullin · September 7, 2023, 9:42am

This depends on a particular setup and a user’s needs and skills.
See a list of “no” in my post before.
So, a UI-based template sensors is a good ADDITIONAL feature (mainly for beginners), not a full replacement.

arganto · September 7, 2023, 10:17am

andrewjswan · September 7, 2023, 10:27am

Regarding the Weather Forecast Service how do get forecast in the template weather now? If the forecast attribute will not be present, then it turns out that I will not be able to use it in the template weather, and the service cannot be called there. Or will I have to make crutches, get forecast in other entities and then substitute it into the template?

yashijoe · September 7, 2023, 10:29am

Thank you!

lpwevers · September 7, 2023, 11:01am

Hi,

Oh, you’re right. It’s 3.17.3, sorry, my bad. The curl command gives me this output:

curl http://172.16.x.x/api/v1/production

{
  "wattHoursToday": 3118,
  "wattHoursSevenDays": 33183,
  "wattHoursLifetime": 10049638,
  "wattsNow": 825
}

I did the upgrade, but it breaks my Envoy. Gives me something like this:

Now my HA runs as a VM so I made a snapshot before the upgrade. After rolling that back everything went back to normal.

I’ll see about creating the RESTful sensors then, though I’ve never done something like that before. Perhaps first try this out on another installation.

robertwigley · September 7, 2023, 11:08am

Ah, OK, that’s unexpected. Thanks for the update, as it will affect me too. I assumed that as 17 > 9, and it has the API i.e. no web scraping, this would still be supported. I will definitely be creating the RESTful sensors before updating in that case. I’ll post the YAML here for you (and anyone else) to use once I have it.

If any of the Devs involved with the Enphase integration are reading this, is there a particular reason why this firmware version is no longer supported when the API is available?