Page showing all "unknown_ID" entities, with a dropdown or text field to do bulk replacements

Month of "What the heck?!"
1

Please think of this case:

User has ZHA, with 200 entities. User moves to Zigbee2MQTT, and has to repair all the entities.

User has 500 automations, many of which depend on the Zigbee devices.

Currently, when the user moves between Zigbee platforms the repairing process causes the old entities to disappear when ZHA integration is removed, and the Zigbee2MQTT integration adds entities with their respective new naming scheme.

So for example, let’s take 1 entity as an example, a door sensor:

in ZHA this entity was called: binary_sensor.door_sensor_garden

This was used as a trigger in an automation, which would send a notification to the HA mobile app if the door was opened.

Upon removing ZHA, this entity no longer exists, so the automations engine can’t find it and on the Automation editor it shows “unknown_id” or something like that. I think it might be “unknown entity_id”.

When pairing in Zigbee2Mqtt this sensor shows up with a new entity name, created automatically by Zigbee2MQTT, now it is:

binary_sensor.contact_door_sensor_garden

So one way to get around is to SSH into the system, and open the automations.yaml in nano, then do a find/replace operation to find the old entity_id and replace with the new entity_id. All well and good except when you have 500 automations and 200 devices…

Also, it’s not always easy to find automations that break because entities are no longer found/available. I never received a repair message or error in the above case, I just knew it wasn’t working because it’s a door I use daily and stopped received notifications.

So my WTH request is two-fold:

  1. better detection of broken automation triggers, conditions and actions
  2. a page to make bulk changes to entity_id names
    2.a. in said page, I’d envisage a list of all entity_ids that are “unknown” or not present anymore after removing an integration
    2.b. I would expect to see a text field or dropdown next to each entity_id (one per row, like a table) so that I could select the “replacing” new entity_id or type its name. Better yet, if I could drag and drop!
    2.c. drag and drop interface would present all failed entity_ids as boxes, then I could drag from a list of existing entities into each box, “teaching” home assistant what is the new entity_id that replaces the broken one, and then HA would auto-magically do a find/replace operation in one last click of a button, fixing anything that got broken when the entity_ID changed.
3

The way around this is to rename the entity in z2m. Use the pencil icon next to the device. Name it to match the old entity id and turn on the update home assistant toggle before updating.

image
image2614×948 212 KB

4

I did explain that it’s an example. It could be any other platform. Please re-read my request keeping in mind what I did was to provide an example.

The point of home assistant is to integrate things into one system, so as a moderator I suggest you to please reconsider telling users to go to another system to do these actions.

I disagree with you (respectfully) completely. The correct way to handle these things is on the system that breaks. It’s home assistant that breaks, therefore the solution to fix should be in home assistant itself. Also, if you tell users to keep fixing things in other systems when it’s HA that’s breaking from changes, you’re essentially telling users that HA is unreliable as a platform.

Because of your reply I am now considering moving to Homey - sorry but as a long-standing HA user this platform by now should have a solidified mission which is to integrate other things into one place for local control. So it’s not compatible to tell someone to go to another system.

Also, if I was moving the other way around, from Z2Mqtt to ZHA, your point would be moot, because ZHA is managed in HA.

It definitely is a WTH issue that an automation software can be so prone to automation failures due to entity names changing and there is no easy eay to amend, it requires thousands of clicking and typing. HA at this point should be focusing on providing easier ways of handling errors and breaking changes because it’s no longer an enthusiast platform, it even has a Foundation…

I have Autism and I don’t know how to sugar coat things but I want to say everything I wrote is direct and meant with the best intention possible.

5

I was trying to help you with the issue you raised.

You are now using a 3rd party add-on (z2m). Home Assistant is not responsible for the way that behaves.

6

I was trying to help you with the issue you raised.

Ok, thanks for the good intention, but it didn’t help because you misunderstood me; It is a fact that I said three times by now that Z2M is just an example. I don’t even use Z2M. There is no issue with the Z2M integration.

I’m talking about when entity_ids change which could be due to migrating from one platform to another so when the same device is readded to HA via the new integration/platform the entity_ID is different, for example when moving from one zigbee coordinator to another, or when pairing a sensor that dropped out of the network, sometimes the sensor will get the same name as before with an _2 appended after it. This breaks automations that depend on entity_id. Truthfully HA is awful at tracking these things, it’s a massive pain. I am NOT, I repeat, NOT ATTACKING HA or its developers, it is a massive weakness of the platform though, and this month you guys asked for us to say what’s causing pain so I’m doing that.

All the zigbee devices are the same before the migration, but their entity_id can change just because HA isn’t the best at tracking a device. I chose Zigbee as an example because the sensors have IEEE addresses that are hardcoded. So it’s kinda ridiculous that HA amends entity names when pairing the device to a different integration, when the hardware identifier should easily help HA understand that the device is not new, it was previously added but because it’s now handled via another integration the entity_id is different. HA is terrible at tracking these changes and automations break as a result. I have hundreds of automations and I can’t spend two days going through all of them every couple of months to see what broke, when HA already knows an entity is unavailable or unknown and should tell me. HA is not proactive about showing issues in automations, the user either has to find out by chance or they have to open each automation and see if triggers/actions are available or faulty.

HA should be like a modern car, showing warnings clearly when things break. But automations can break silently with no warning. It’s kinda ridiculous when a Home Automation platform’s core utility - Automating - breaks so easily with no warning due to just an entity_id changing, with no easy way to fix the issue.

At least make some sort of bulk editor to replace old ID with new ID and HA should then automatically amend any automations that relied on the old values with the new values.

To be clear, I am not stupid and you can trust me when I say this is a problem with HA not any other system, and again I used Z2M as just one of many potential examples

This is an issue with HA, absolutely, because HA decides how entity_id names are created, either internally or the integration does. I can already edit an entity_id by accessing the entity’s info page then clicking the gear. But imagine doing that for hundreds of entities.

I’m saying it would be great if there was some sort of spreadsheet-style table to allow bulk editing of entity_id for any ones that are unavailable/unknown to replace old value with new value, and automatically have automations work again.

Do you want me to do a whiteboard session to explain better? I understand that how I’m expressing the issue may not be clear and I’m happy to have a friendly web meeting with you if it helps. I don’t know how involved you are with development of the product but I am open to talking to anyone in your development team to take questions.

The issue is Home Assistant not having a way to amend entity_ids in bulk when a device name is changed by swapping integrations or amending a device in the integration’s app directly. Automations break as a result and rather than showing the user that the automation has an invalid/unknown entity_id on some sort of status page, it just silently fails.

If an automation trigger or action has a device/entity that stops existing in HA then HA should warn the user (it does sometimes, as “repair” warnings, but not every time. And there should be a way to edit/replace entity_id within home assistant itself, because it is HA that utilizes the concept of entity_id.

I actually have a real-life example:

I use ZHA for my zigbee devices. I have a laundry door sensor: binary_sensor.laundry_door

The sensor battery died a couple of weeks ago, I didn’t notice. I opened the door today and the light didn’t come on, so I manually checked, and saw that the sensor was powerless. When I replaced the battery the sensor didn’t reappear online, it dropped out of the network. But the trigger in the automation to turn on the light when I opened the laundry door is still expecting binary_sensor.laundry_door to react. When I went on the ZHA integration and put the coordinator into discovery mode and paired my sensor, HA assigned the entity_id as binary_sensor.laundry_door_2, for no reason! The sensor IEEE address never changed, so HA has no reason to do that, literally. It’s bad logic/programming. Now I had two choices: do I amend just this one automation to use the trigger binary_sensor.laundry_door_2, or do I edit the entity to amend the entity_id back to binary_sensor.laundry_door? The first one fixes just the single automation. The 2nd one requires an extra step of deleting the “unknown” device binary_sensor.laundry_door before I can rename binary_sensor.laundry_door_2 to binary_sensor.laundry_door otherwise HA complains that there is already a binary_sensor.laundry_door entity_id even though it’s invalid because HA decided to take the same device and give it a fresh new binary_sensor.laundry_door_2 entity_id when I paired it back to get it seen by the coordinator after the change of battery.

Now I want you to imagine that you have 50 sensors in the same situation… and I was using just one automation as an example. Imagine having to manually amend 200+ automations.

I am not being a b^^^^, this is a real issue, a real pain in the backside and it’s definite a WTH that I hope can be addressed in the future to make the platform better. I can’t complain about HA, it’s been of great utility to me, but nothing is perfect and this is definitely the platform’s pain point, the way things break with no warning message and no way to easily fix things. Hopefully it can become easier and more intuitive in the future.

Thanks and nothing more to add here.

7

I’d like to remind you:

When someone replies on your post, it does not mean the WTH is invalid. Please take a step back and try to understand what another person is saying before jumping to conclusions.

2 Likes
8

Moderators have a bigger responsibity than users and are meant to be role models. You are correct to ask anyone to try to understand - but as a role model in the community it’s required for Moderators to do exactly the same and it didn’t happen. Can I please ask respectfully again for my post to be read again, I’ve mentioned several times that I used a certain integration as an example, is everyone in this discussion aware what the word example means? I can provide a definition if that helps us all be on the same page.

I know you have passionate followers who will support you no matter what, but please understand that it is hypocrisy to expect something from people lower than your level. Moderators always should display higher standards and despite stating many times Z2M is an example this was ignored.

I have felt like posting the issue only brought frustration and animosity. I want to remind you I am only here posting a detailed report of something that I feel was addressed in enough detail and would be happy to expand further if needed.

I do feel that my report, if taken forward for improvement, would greatly benefit any HA user that has to deal with renaming entities from time to time for whatever reason.

Thanks and have a nice day.

9

I’m sorry, but this is the second time you’ve reacted this way to a WTH which is why I even replied on this thread. Moderators simply moderate, they are not role models. If you can’t handle any feedback from a member of the community, then you should avoid posting. People will discuss things and there will be comments that you do not agree with, and that’s okay. What is not okay is the way you’re reacting to these comments.

10

As a human being, I feel like literally killing myself sometimes. I have Autism, and what you are asking me is literrally impossible. It’s like asking an amputee to grow back their legs. Or someone with Cerebral Palsy to grow a new brain. I have a disability, it makes the way I communicate different. I challenge “normal” people to stop making people like me feel bad over something we can’t control. No, I refuse to be shunned from society and stop posting just because I’m disabled and it affects my communication. Please be inclusive. I said so many times even in capitals that I’m not attacking anyone, that I appreciate the project, and said Thanks many times. Somehow all of you “perfect” human beings simply ignore those things and focus on the parts you don’t understand. Please educate yourself about Autism and other Neurodiversity. You are a Moderator and the powers you possess can affect people in a good way or a bad way, I plead with you to stop making me look like a villain, other users will see this and bully me over it.

I already tried to commit suicide because the Internet is vile, so I want you to know that the constant pressure and discrimination has a REAL effect on people with Neurodiversity and YES you should be a role model about it. Users see how you behave and adopt the same posture.

Thanks for reading, I’m out because I’m literally shaking right now. It’s so hurtful. I only came here to make a report, to try to contribute because I don’t know coding. And every single time my contribution gets shot down.

11

No one is shooting you down. This is what I meant when I said:

16

Well, this escalated quickly, didn’t it.

Hi! I’m Missy, the Community Manager for Home Assistant. First, let me call out that I am also proudly autistic - as someone who was once described as “talks to problems, not people”, I can relate to the struggle of softening speech. I would wager that about 80% of this entire community is on the neurodivergent scale whether they are aware of it or not - it’s one of those correlation things with the tech industry - there are very few “normals” here. :slight_smile: This won’t be a short post but I encourage you to read the whole thing. :wink:

There are some reasons why I asked the mods to lock this thread down before you could respond again. 1) I couldn’t respond immediately. 2) I did not want others to jump in here before I could step in. 3) It has escalated drastically through what is a clear misunderstanding.

I’d like to start with telling you that I agree with your WTH suggestion. You have a very valid point regarding naming entities when they go unavailable and having to rename them. This happens to me all the time and I hate it. It’s not just Zigbee that does it to me, either, it’s Tuya that also does it. Yes, this is definitely within Home Assistant and how its entities work - but it’s also difficult to change.

When posting a WTH suggestion, it’s best to actually describe what you are trying to do, not a hypothetical of someone else. This helps everyone understand the problem you are running into - as you can see, I relate to your real world example you provided. Telling us how to build something or telling us use cases that you simply aren’t a part of clutters up the conversation and can muddle up your entire point. Kind of like what happened here unintentionally.

Now, allow me to address a few things outside of your WTH suggestion, because I think it’s important. I am not going to soften my points here.

  • Your display name is “Asshole Customer”. I really don’t know what else I can say about that, tbh. Like, how do you expect people to treat you with a name like that? :woman_shrugging:
  • Moderators are not employees of Nabu Casa and are not required to suggest only Home Assistant things. Home Assistant is modular with integrations, it’s what makes it pretty great for a lot of people.
  • Moderators help keep the peace around here - all are incredibly knowledgable in Home Assistant, help keep things organized, and know what inappropriate behavior looks like. Not all of them are perfect examples of a community member but rather people who help me get the hard job of moderating a very large community done.
  • We are all about Choice - it’s one of our values. You can choose Homey if you want - no one is here to sell you anything.
  • You have some very condescending messages in your offerings to “help explain” yourself.
  • No one here has been attacking you and there has been no animosity towards you - there’s been simply information shared and a moderator telling you to tone it down.

One person told you how to switch things in Z2M, using the example you provided - this is why it’s best to post a WTH with your exact situation instead of an example use case. Another person said please watch your reactions, and you decided to explain to them how to be a moderator. I noticed that you have only been around these forums for 2 months, I can understand how you would not realize the role of the moderators here. Hopefully I was able to clarify that for you above.

This whole thing here :point_down:t3:

Unnecessary. Yes, you communicate differently - it’s part of what makes us authentic in our responses. But it is able to be reined in with practice, and you do not get to accuse someone else of ableism because they suggested you not post if you are unable to handle discussions around your posts. Yes, I absolutely understand that sometimes the tone that flies out of one’s mouth is that of just to state fact, not insult. However, you do not get to throw your disability around as if it’s the reason people are not helping you - this is simply not the case and I really stress that you take a step back and look at this convo again. Like I stated early in this post - most people here are neurodivergent, and most get by communicating here without issue.

If you truly consider yourself in danger, I encourage you to contact your support circle - as an autistic person I am sure you have one that includes family, friends, and/or medical professionals. If you don’t, please send me a DM, I will happily be an ear for you during a rough time in your life. No one here wants to make things harder on you, I promise.

I’m gonna open this thread back up.

What I want to see in here moving forward are excellent examples of pain points around renaming entities - those who work on developing Home Assistant have the hard task of finding a solution while we can sit in here and talk about what entities this happens to the most for us. :slight_smile: Anything outside of that and I (and mods) will start deleting messages for being off topic. (And just a general note that not all WTH suggestions will be actioned on, but we can still have discussions.)

1 Like