Mqtt persistent mode

So this isn’t really possible tbh. I’ll try to explain the two challenges you face here, one is feasible but not supported the other is probably impossible. I mean I’m not going to definitively rule it out but extremely unlikely to happen seems like a fair assessment.

The first is that HA does not support importing historical data. An integration can only change the state of an entity right now. There is no way for an integration to change the history of an entity and say “this entity had this state from time X to time Y”. There’s a number of feature requests around this capability, particularly when it comes to historical energy data (here, here and here for example). It makes sense but its not possible currently.

The best an integration could do right now would be to rapidly change states for the queued messages. This ensures all state changes occurred but there’s nothing it can really do with the timestamp of those messages currently.

The second thing you’d need is for an integration to be able to import historical data and trigger state changes as if they occurred at that moment in time. Remember, each integration is totally independent. The energy meter integration has no idea about MQTT or anything like that. The only thing it is doing listening for is a state change event for an entity with id sensor.energy_red. Same with potentially many other things (templates for example).

So for this accumulator to be able to work correctly it would somehow need to receive and account for a backdated state change event. This is a major problem because to properly process a backdated state change event most integrations would need to see everything as it existed at that time. Think about a simple template like this for example:

{{ states('sensor.energy_red') | float + states('sensor.energy_blue') | float }}

If this template was used in a template sensor or a template trigger how would this work if it receives a state change event for sensor.energy_red in the past? It would also have to know the state of sensor.energy_blue at that moment in time as well. If it used the current value of the blue sensor but the past value of the red sensor then the result is nonsense.

Perhaps this second feature request could be somehow limited to just energy meter or integrations that only look at the value of a single entity. But if there’s any way an integration could look at the state of multiple entities there’s really no way to handle a backdated state change event. Recreating the entire state machine at that moment in time is just not feasible.

1 Like

Hi Mike,

given your explanation I do come to agree that is unlikely what I request to be able to be implemented fast and easy on HA :). I mean so far I am sending the current wattage of each phase (red, yellow, blue) to an MQTT sensor, then I create sensor to add them all together and then use a sensor with platform: integration to accumulate the wattage in kWh which in turn I pass that sensor to a utility meter.

Since HA can not “change older entries” in history it is very unlikely that I can achieve what I describe earlier when the client is offline/disconnected as you describing in you latest post.

Maybe the solution to my problem is to have the publishing client (the one that sends the three phase’s value to the broker), to first calculate the total accumulation in kWh and then sent that value to HA (through broker of course) as a sensor that does not have the current wattage but rather the total kWh so far.

In this case even if the messages are lost at some point when HA establishes connection to the broker again it will receive a value of the accumulated wattage of time “now” which I believe is more in line with the description about how HA works.

Thanks

Phanos

1 Like

Not sure why you are mentioning concurrent connections. We are talking here about the persistent session as a remedy against losing data when an mqtt client is disconnected. The persistent session allows receiving the sequence of missing data after reconnection.

Other than that, Mike above very well explained the usage of the persistent session in context of Home Assistant.

Review the thread and find the post by phanos where two MQTT topics are subscribed in separate sessions.

However my response was to your post which introduced an incorrect description of persistence mode’s scope.

I wouldn’t call it “incorrect description” just because I differently named the string used for client identification. But yes, you are right. it’s clientId

Actually he wanted to have separated connections from HA for each entity, which is not supported by HA. However, it should be explicitly said, that it’s not possible to create two or more connections identified by the same clientId. That;s why I don’t understand why you mentioned it in context of my contribution.

Reviewing the thread I’m still hitting your answer which sounds wrong to me:

The documentation link you provided in your first post explains it is a persistent session . The term “session” in this context means from the moment a client connects to the broker to when it disconnects. All of the client’s subscriptions during the persistent session are stored.

The documentation he provided mentions persistent session mode, which obviously doesn’t end with a client disconnection. In fact, after disconnection, the server still queues data subscribed by connection identified with a particular clientID. Any next connection identified by the same clientID is just a continuation of the previous (interrupted) connection. This allows receiving missing data (in the meantime stored on server side)

Possibly because you misread it.