Mqtt discovery: deleting all devices discovered from a given service?

Development
1

I’m developing a service that will send information via MQTT, and which will create devices & sensors using MQTT discovery. The things being tracked in the real world may come and go, and (as I’m envisioning the design right now) I’d like the HA entities to do the same.

I see that I can delete the devices by sending an empty config message.

If my service were guaranteed to never crash, I could simply keep track of everything on that side and send out removal messages when appropriate. But… in the real world, something could happen and I could lose track of state. How do I make sure I clean up “orphan” entities on the HA side?

2

What is your criteria for an entity to be classified as an orphan?

For example, let’s assume a removal message is either not sent or never received. Therefore the entity that should be removed continues to exist. What outward behavior does this entity have that indicates it is orphaned? The challenge is that there’s not much about its behavior that suggests it is orphaned other than its value ceases to change.

MQTT Sensor has an expire_after option where its state becomes unavailable if it fails to be updated after a specified period of time has elapsed. However, in this case, unavailable is ambiguous because even a “non-orphan” entity can potentially become unavailable.

3

tl;dr summary:

My source occasionally has reset events. Those represent the current state. Any entities not listed should not exist at that time.

Long version:

I’ll be more concrete with what I’m doing, which will hopefully help. I’m making a bridge from the local transit system’s streaming bus/train prediction service to Home Assistant, so I can do things like make a light change color when it’s time to leave for the bus.

The transit system API sends updates about “resources” — I’m tracking those for prediction, stop, vehicle, trip, route, line, and alerts types. For each of these, there will be multiple resources at any given time — obviously stops don’t change very often, but vehicle ID depends on what actual busses happen to be on the line that day, and predictions are highly-transient with unique ids.

There are basically two different ways I can go about this that I can think of — the first is to track state in my bridge application, and send highly-interpretted data, while the second is to make the bridge as lightweight as possible and just reflect the resources closely into HA entities, and use templates on the HA side to do interpretation and processing.

I’m trying the second idea first. :slight_smile: The various resources have interlinked data — for example, each prediction has a route_id, stop_id, trip_id, and vehicle_id. I could use those to pull things together into HA devices with the links as attribute data (I’m looking at how my Ecobee thermostats present the weather — they have a “climate” sensor which then has a list of forecast info. A “stop” device could have an “upcoming busses” sensor with the predictions (or a list of prediction ids) as attribute information.) Or, I could just have them all be separate entities and find and ‘stitch’ them with templates and scripting in HA. Not sure yet what will be best.

Anyway: The streaming API sends reset, add, update, and remove events. The crucial thing here is that on first connect, and arbitrarily whenever there is a lot to update at once (first thing in the morning, for example, as our system tragically does not run all night). And when a reset event happens, that’s a clean slate. So, “orphans” would be: any entities created (through previous reset or add events) but which do not exist in this reset event.

4

This sounds like something you will have to manage on the bridge side because it knows which ones should exist (and what’s not in that list is an orphan).

5

But that then requires keeping track of state, rather than just forwarding. Without that, it knows what should exist but doesn’t have any concept of what should not exist. Even if I keep track, there is fragility — what if my service crashes and loses track of what was created?

So it’d be nice to have a way to tell HA “clear all discoveries that match this pattern”.

I don’t mean this to be argumentative; I appreciate your help!

If there’s no good way to do this, it suggests that maybe I should instead limit to creating entities for just stops (which are unlikely to change often!), and put everything else as state attributes under those, with updates via json_attributes_topic. Do you think that makes more sense?

Or, if I want to stick to the first idea, I guess I could have an entity for the bridge state itself, update that on reset, and have a HA automation trigger when that happens, and have that use {integration_entities('mqtt') filtered by something that identifies only the ones created by my bridge — and then call HA’s mqtt publish service (“The call is coming from inside the house!”) to clear them. That seems a bit rube-goldbergian, though.

6

Maybe you could subscribe to the topic you are sending discovery messages to. If they are retained, you’ll receive them and you should be able to remove the orphans based on that. This seems like a good use for the node_id part of the discovery topic and MQTT topic wildcards.

1 Like
7

Oh! That makes sense. Makes the client a little more complicated since it has to listen rather than just talk, but seems like that should work! Thanks!