2024.4: Organize all the things!

The Core 2024.4.1 update did not work well at all for my HA installation.

Almost all pages showed the error message “error while loading page xyz”. Since I didn’t see any clues at all as to the cause of the error and even About, System, People, Developer Tools, Dashboards, Automations & Scenes were affected, I honestly didn’t feel like trying to figure out what the exact cause of the error could be.

Fortunately, my HA is running on Proxmox and I was able to restore everything with last night’s backup.

But: I will have to do the update at some point. I would therefore be very grateful if I could get a hint as to what this extensive error could be due to?

I would be happy to provide logs or other information. However, as my HA is in productive use, I could not leave it in this miserable state. If it is necessary for troubleshooting, I will install the update again.

2 Likes

FWIW, I have used another home automation software product since 2007 and its data is stored in an object-oriented database. It uses nested areas to define spatial hierarchy where each area is assigned a “type” (it’s actually a class but that’s another discussion) from a comprehensive list of predefined types (classes).

I have the flexibility to define the hierarchy any way I want and the resulting structure is understandable (to the software) because it’s not merely based on area names (which are merely text of my choosing) but predefined area types. In other words, I can define an area whose type is floor, attic, boathouse, building, closet, bathroom, lawn, garden, etc (there are dozens of available types). The software employs the hierarchy to generate a default UI (impressive for the time it was first released in the early 2000s).

Its scripting language offers the ability to traverse the hierarchy, select areas by type, etc thereby allowing you to slice and dice it whichever way you need (and the concept of “type” is also available to each individual entity so you can be very precise about selecting what you want and where it’s located).

I had proposed the idea of nested areas with types and it garnered some interest but ultimately “floor” prevailed.

Anyway, just wanted to highlight the extent of the ideas offered in that lengthy discussion (from proponents of the nested areas concept). It’s only of historical interest now because it was ultimately rejected in favor of “floor”.

4 Likes

Yes, one of the reasons it was announced 6 month ago(7 now), So Custom:Card Devs had time to adapt to the Service-Call function, the Native-HA Weather card has worked like that for about the same time, 6 Month :wink:

1 Like

After some testing of labels:

There could be a temptation to “label” all your entities to have a possibility like “show all batteries related to a heating equipment in my south-west bathroom”.
I.e. this could be only for PRESENTATION - to show only THESE particular entities in a table.
It may cause having 100500 labels in the “filter” panel.
Next you may want to rename these labels to be ordered in a specific way (and starting questions like “WTH label_ids are not renamed after I renamed labels”).
You may want to have these labels hierarchically displayed - like “show 5 root labels, each label may have sub-labels etc”.
You may want to have persistent presets like described here - i.e. there will be no need to select a particular label out of 100500 labels, just select a “Bathroom heating batteries” preset.

For now I ended up keep using my “old style labels” which are defined in yaml and may only be used in templates to select entities, not to be used in UI in tables (except “Dev tools → State”).
“New style” labels will be added only when I really will need them to be used in UI tables for filtering; do not think there will be many of them.
As for replacing my “old style labels” by “new style” labels - will probably do it in SOME cases and only when a “yaml labels support” will be added. Why “SOME cases”, not “ALL cases” - because some labels may not be needed in UI (to avoid clutter in labels).

2 Likes

As for “categories”:
I wonder how using “categories” for automations can be a reason “Now I finally can move automations from yaml to UI”.
These are different things. People keep maintaining automations in yaml not only to have them “categorized” (i.e. storing in a file system in some order) - but for using yaml features like “include”, “anchors”, “secrets” etc; not to mention packages.
If you need to use categories for automations - just provide "id"s for them in yaml (which is needed for seeing traces also).

Btw, I have not added any categories yet.
Imho categorizing of objects should be done based on properties of these objects.
But a “category” is not a property.
“Label” is a property - but since you can add many labels to an entity - you cannot group, can only filter.

1 Like

I used the category feature for my 130 automations and 30 or so scripts.

I found that using the GUI convert method to be annoying because it actually duplicates your yaml automation, which you then have to remove by deleting or renaming the old yaml files. You end up with all the new automations with an entity names that are suffixed _2. And a lot of old automation entities that you have to delete one by one.

There is a much easier way.

Do these steps

  1. Make sure all the yaml automations have an id. It can be any string. It just have to be unique. If you had to add any to add, reload automations.

  2. Make sure that all your existing automations are defined as a list. Ie each automation starts with a “-”

  3. Make a safety copy of your existing automations.yaml file unless it is empty

  4. Copy all your yaml automations into the automations.yaml file. Just append with what is already. Do not reload automations at this time!!!

  5. Rename your old automations files to something that does not end with .yaml (or move the file to a location that HA does not include automations from)

5 Reload automations in HA

6 Go to the automations in Settings. Pick any of them and edit yaml (in the 3-dot menu in the upper right corner). Add a space somewhere and delete it again. And save. This makes HA rewrite the automations.yaml file.

7 Now you can organize all the automations like you had then in files and folders by using Category feature maybe combined with labels.

Note that all comments are deleted when you do this. But you can use the description field for each automation to put your notes. It can be multiple lines. You can edit this in the GUI. It is part of the rename automation feature. You leave the name unchanged and you can add or change the description below. In practical this is fine for those little details you know you will forget.

When you do it this way, nothing is duplicated, nothing is renamed. It all works like before except now you can edit the automations via the GUI incl editing each automation as yaml in the GUI

What held many of us from using the GUI was the horror of one long list of automations. The category feature alone has done it for me. I have around 10 categories for my 130 automations. That is plenty to organize them so I can find things quickly. I love that new feature!!

9 Likes

That’s a browser error. Have you cleared your browser cache or tried another browser?

A backup is a backup regardless how you installed Home Assistant.

If you are going to try and create a naming convention that is going to ‘mean’ something then create a hierarchy that generally fits with most of the concepts that are out there in the big wide world already (like address’ have house, street, town, region, code - and that generally fits all cases) - property is pretty well established in ‘structure’ in law and in ‘planning’, and generally works with:

property->building->internal/external->floor->room->sub-area

You can argue that the internal/external distinction can be folded into either building or floor (so you treat house and patio as two different buildings or you treat house as having an extra patio floor) - but that’s about it.

Exactly the current system has; zone - floor - area
Which in my opinion needs to be zone - area - floor - room - section
zone as is now
area = outdoor front, left, right, back, house, parking, shed ,
floor = basement, whatever level and roof
room = kitchen, living etc
section = media, dining table or whatever is in a seperate something in your room

Now I’m need to have a few outdoor floors; like parking floor, back garden floor and side garden floor. That doesn’t sound right and the same for areas in the house, that’s rooms for me.
So I will just name the sensors as before indicating the room and just label them appropriate.

The got the latest update (updated from 2024.3.3) running on a docker container. The new version seems to create a weird issue on my end where the web interface becomes completely inaccessible after a few minutes. It works initially but eventually just stops responding and becomes inaccessible. Same thing when accessing it via the Android/iOS apps.

Unfortunately I don’t see anything helpful in the logs from before that happened – just the usual warnings about hacs and integrations installed through hacs at boot:

2024-04-06 22:42:39.740 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration frigate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-04-06 22:42:39.741 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration keymaster which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-04-06 22:42:39.741 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-04-06 22:42:40.216 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly
2024-04-06 22:42:40.244 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=578 from 2024-04-06 14:16:27.835404)
2024-04-06 22:42:40.458 WARNING (MainThread) [zeroconf] Address not available when binding to ('fdd0:9fbd:bf54:0:42:c0ff:fea8:150', 5353, 0, 0), it is expected to happen on some systems
2024-04-06 22:42:43.583 WARNING (MainThread) [homeassistant.helpers.frame] Detected that custom integration 'hacs' accesses hass.components.frontend. This is deprecated and will stop working in Home Assistant 2024.9, it should be updated to import functions used from frontend directly at custom_components/hacs/frontend.py, line 68: hass.components.frontend.async_register_built_in_panel(, please create a bug report at https://github.com/hacs/integration/issue

Reverting to 2024.3.3 fixes the issue. Is there any specific place / logs I should be looking at for some clues as to what’s happening?

I have the modes configuration in a separate modbus.yaml file, this is what it looks, and it works fine with the previous version of the core:

- name: modbus_hub
  delay: 5
  timeout: 5
  type: tcp
  host: 192.168.1.152
  port: 502
  close_comm_on_error: true
  retry_on_empty: true
  retries: 3
  sensors:
    - name: Smartfox_Total_Energy_Used
      address: 40999
      unit_of_measurement: Wh
      data_type: uint64
      input_type: holding
      slave: 1
      scan_interval: 30
    - name: Smartfox_Daily_Total_Energy_Used
      address: 41011
      unit_of_measurement: Wh
      data_type: uint32
      input_type: holding
      slave: 1
      scan_interval: 15
    - name: SmartFox_Current_Energy_Use
      address: 41017
      unit_of_measurement: W
      data_type: uint32
      input_type: holding
      slave: 1
      scan_interval: 10

All error messages look like they belong to a serial connection to a modbus, but I connect via tcp.

I restored from back-up, and it works again fine.

I’m pretty sure that’s a problem with the new version of Core.

Guys I cant get GPIO working on RPI5 any idea?
Another problem I face with Xiaomi cameras, cant get them to stream video to HA

You’re using multiple deprecated yaml fields. One of which was fully removed close_comm_on_error

1 Like

Happy to try and upgrade again without these, but are you sure they caused all these error messages related to a serial connection?

The config will not load if you have an invalid config.

The deprecation occurred

also that release

It will retry by default on empty if retries is greater than 0, you have it set to 3. So all you need to do is delete them but keep retries: 3

2 Likes

You guys are seriously not suggesting entity-device-naming like above ?
In My “World” entity-names is mend for the System to handle, the shorter the better, without to many “underscore/dashes” etc.

There are Multiple ways of “handling” short-names, And these are also worldwide known “concepts” , or just find “invent” your own, which make sense , in your head, and in reflection to your “property/environment/Devices” etc

And an Area can be whatever you want it to be.
I have 2 Areas in my TV-Room(Which Btw is an Area also ) so that makes it 3 Areas
I have Areas covering 2-3 Rooms
2-3 Areas(For Specific Functions, related to the heating system) in the PanRoom( Which is also an Area )
I have 4 outside Areas ( And i have not decided, or considered whether they should be “grouped” in a Floor ( seems ridicules to me )
I have 3 “Balconies/Verandas” , Yes they are at it’s respective “Floor Level”, But it’s considered Outside Area
Garage, 3 step Down, or wheelchair slide if you prefer, No it’s not a floor(plan) It’s a Garage period.

Cut loose, don’t hang up your self on " Common Concepts, or Established structure " in Law and Planning" , you are blinding your self, and blocking your imaginations . Get out of your enforced paradigm , and use your “hidden” childish imagination :slight_smile: , and logic

1 Like

Great! some of your instructions I’m doing already whenever I do a major change configuration but this is an extensive step by step.
Currently my automation do not start by an “_”
As soon as I perform the upgrade most likely I will probably try the migration and see how it goes. Thank you.

Ok, i dont know how i would have figured this out from the error message, but i tried it and everything works fine now.
Thanks you so much!

Looks like that HA doesn’t fully support that caracter set like probably many others. Why you where expecting to be fixed in this version?

Watch out. He said -, not _.

One is a list, the other isn’t.

1 Like

Good catch! Thank you.

1 Like