MQTT Sensor : how to define last_reset_value_template

francisp · August 5, 2021, 9:36am

I’m trying to add last_reset to my mqtt sensor, but I always get invalid configuration. 2 examples at the bottom have been commented out with #, if I remove any I get invalid configuration

  - platform: mqtt
    name: "Mains Consumed Energy"
    state_topic: "electricity1/tele/RESULT"
    value_template: >- 
      {% set message = value_json.SerialReceived %}
      {% set payload = message[6:14] %}
      {% set payload_len = (payload | length) %}
      {% set result = namespace(value='') %}
      
      {% for i in range(0, payload_len + 1) | reverse -%}
        {%- if i is divisibleby 2 -%}
          {%- set result.value = result.value + payload[i:i+2] -%}
        {%- endif -%}
      {%- endfor -%}
      
      {{ (result.value|float) / 100 }}
    unit_of_measurement: 'kWh'
    unique_id: "mains_consumed_energy"    
    device_class: energy
    state_class: measurement      
#    last_reset_value_template: {{ '1970-01-01T00:00:00+00:00' }}
#    last_reset_value_template: {{ as_datetime(as_datetime(0|timestamp_utc)|string+‘Z’).isoformat() }}

wmaker · August 5, 2021, 9:30pm

I will be doing the same thing (haven’t yet)…Here is a thread that discusses a couple of ways to do this:

Use homeassistant: customize instead of the mqtt sensor config
Use mqtt sensor config as you have attempted and set up a “fake” last_reset_topic and somehow publish a retained message to that topic with a message containing a reset time

navet · August 5, 2021, 9:34pm

I just added this and it started working.

homeassistant:
  # Customization file
  customize: !include customize.yaml
  customize_glob:
    sensor.*_Energy:
      last_reset: '1970-01-01T00:00:00+00:00'
      device_class: energy
      state_class: measurement

123 · August 5, 2021, 9:40pm

The templates are missing outer quotes.

Mariusthvdb · August 5, 2021, 9:43pm

you can actually use a last_reset_topic, and act upon Frencks request, to stop customizing as much as possible.

the ‘official’ documented ways dont work (in my config, though Frenck says they do…), but we found a way, this is what I use now:

  - platform: mqtt
    state_topic: 'macaddress/energy-solar-sdm630-modbus/f339a2fb/import'
    name: ZP Import
    <<: &energy
      unit_of_measurement: kWh
      value_template: >
        {{value|round(2)}}
      last_reset_topic: 'fake/last_reset'
#      last_reset_value_template: >
#        {{'1970-01-01T00:00:00+00:00'}}
#        {{as_datetime(0|timestamp_utc).isoformat()}}
      device_class: energy
      state_class: measurement

I wrote it in an anchor here, so to merge that with <<: *energy on all my other energy sensors.

all you have to do for this is publish a retained topic, with unquoted payload, like this:

btw your last_reset_value_template is not correct, you should either quote it on 1 line, or use the > multiline indicator:

last_reset_value_template: "{{ as_datetime(as_datetime(0|timestamp_utc)|string+‘Z’).isoformat() }}"

or

last_reset_value_template: >
  {{ as_datetime(as_datetime(0|timestamp_utc)|string+‘Z’).isoformat() }}

you’ll see the error go away, but it wont work (at least thats my experience, but please give it a go) and report back?

ppluti007 · August 5, 2021, 9:47pm

I confirm it works adding the global customization, but i didi it working even managing the sensor platform mqtt using a pow1 with Tasmota:

this is my code (in a package):

sensor:
  - platform: mqtt
    name: "Energia Lavapiatti"
    state_topic: "tele/pow1/SENSOR"
    value_template: "{{ value_json['ENERGY'].Total }}"
    last_reset_topic: "tele/pow1/SENSOR"
    last_reset_value_template: "{{ value_json['ENERGY'].TotalStartTime }}"
    qos: 1
    unit_of_measurement : "kWh"
    device_class: energy
    state_class: measurement

this is how it is

and i can add it to my Energy Panel devices

123 · August 5, 2021, 9:50pm

Question: can the last_reset_topic_template option work without using the last_reset_topic option?

It seems to me it was designed to be used to extract a value from the topic’s payload. However, if used alone, maybe the topic’s payload is not needed?

Asking because there’s no last_reset_topic in the first post.

Mariusthvdb · August 5, 2021, 9:56pm

yes I suppose the same Taras.

I haven’t found a single last_reset_value_template to work at all, but confess only tried them stand alone, without a last_reset_topic to extract data from.

I did, as described above, found a way to use the topic, standalone that is.

This all is of course for the never reset scenario, and doesn’t follow any devices reset, which isn’t supported in my devices to begin with.

the beauty of Pluto’s config is it seems to do just that.

123 · August 5, 2021, 10:14pm

I think the reason why this doesn’t work is because it won’t evaluate the template until a payload is received. However, the purpose of this approach is to avoid having to publish anything to the topic so it’s a stalemate.

      last_reset_topic: 'fake/last_reset'
      last_reset_value_template: >
        {{'1970-01-01T00:00:00+00:00'}}

ppluti007 · August 5, 2021, 10:15pm

My config without the last_reset_topic doesn’t work.
I suppose that it works just like the value_template that is taken from the state_topic… So the last_reset_value_template is taken from the last_reset_topic

Mariusthvdb · August 5, 2021, 10:16pm

not sure if I expressed myself clearly, but using the single last_reset_topic works fine (provided the once in a life time retained publish I screenshotted…)

yes that seems to be the final conclusion, and I think that should be indicated in the docs more clearly.

You can use a last_rest_topic on its own, provided it is of the correct format already. If not and one needs to extract data, use the additional last_reset_value_template

123 · August 5, 2021, 10:23pm

Yes, I understand that particular approach works for the simple reason that it complies with the intended purpose of last_reset_topic.

My comment was targeted at the other approach you appear to have attempted in the past where you hard-coded the Unix Epoch into last_reset_value_template. The assumption is you did that to avoid having to explicitly publish that value. However, it fails because the template is not evaluated until a payload (any payload) is received via last_reset_topic.

Mariusthvdb · August 5, 2021, 10:29pm

correct. I did try that, because Frenck insisted ‘it’ should work. He didnt however say we need the last_rest_topic first…
Ive asked in Discord #devs_energy now, so hope we can get confirmation our conclusion is in fact correct

123 · August 5, 2021, 10:31pm

It sort of does:

Defines a template to extract the last_reset.

It’s also more specific than the description for plain old value_template

Defines a template to extract the value

That’s why I was puzzled by the lack of the last_reset_topic option in the first post. The combination of topic and corresponding value_template is fairly common (json_attributes_topic and json_attributes_template) so seeing last_reset_value_template used alone seemed anomalous to me.

Mariusthvdb · August 5, 2021, 10:35pm

with that knowledge Frencks answer makes sense now. At the time of that post, some of us were in the effort trying to make that stand alone template work, not thinking past our own mind lock…

francisp · August 6, 2021, 2:53am

Seems correct. I noticed how zigbee2mqtt sends it’s discovery messages :

{
  "availability": [
    {
      "topic": "zigbee2mqtt/bridge/state"
    }
  ],
  "device": {
    "identifiers": [
      "zigbee2mqtt_0x04cf8cdf3c77ea45"
    ],
    "manufacturer": "Xiaomi",
    "model": "Mi power plug ZigBee EU (ZNCZ04LM)",
    "name": "mi_outlet",
    "sw_version": "Zigbee2MQTT 1.21.0"
  },
  "device_class": "energy",
  "json_attributes_topic": "zigbee2mqtt/mi_outlet",
  "last_reset_topic": "zigbee2mqtt/mi_outlet",
  "last_reset_value_template": "1970-01-01T00:00:00+00:00",
  "name": "mi_outlet_energy",
  "state_class": "measurement",
  "state_topic": "zigbee2mqtt/mi_outlet",
  "unique_id": "0x04cf8cdf3c77ea45_energy_zigbee2mqtt",
  "unit_of_measurement": "kWh",
  "value_template": "{{ value_json.energy }}"
}

123 · August 6, 2021, 3:19am

Clever!

It uses the same topic for state_topic, json_attributes_topic, and last_reset_topic.

Because state_topic will regularly receive payloads, it will cause last_reset_value_template to be evaluated the moment a payload is received via the shared topic.

The actual content of the payload is irrelevant for last_reset_value_template because it doesn’t use it and is hard-coded with the Unix Epoch.

Applying this simple principle to Marius’ example, we get this:

  - platform: mqtt
    state_topic: 'macaddress/energy-solar-sdm630-modbus/f339a2fb/import'
    name: ZP Import
    unit_of_measurement: kWh
    value_template: >
        {{value|round(2)}}
    last_reset_topic: 'macaddress/energy-solar-sdm630-modbus/f339a2fb/import'
    last_reset_value_template: '1970-01-01T00:00:00+00:00'
    device_class: energy
    state_class: measurement

Your MQTT Sensor becomes this:

  - platform: mqtt
    name: "Mains Consumed Energy"
    state_topic: "electricity1/tele/RESULT"
    value_template: >- 
      {% set message = value_json.SerialReceived %}
      {% set payload = message[6:14] %}
      {% set payload_len = (payload | length) %}
      {% set result = namespace(value='') %}
      
      {% for i in range(0, payload_len + 1) | reverse -%}
        {%- if i is divisibleby 2 -%}
          {%- set result.value = result.value + payload[i:i+2] -%}
        {%- endif -%}
      {%- endfor -%}
      
      {{ (result.value|float) / 100 }}
    unit_of_measurement: 'kWh'
    unique_id: "mains_consumed_energy"    
    device_class: energy
    state_class: measurement
    last_reset_topic: 'electricity1/tele/RESULT'
    last_reset_value_template: '1970-01-01T00:00:00+00:00'

EDIT

Correction. Removed superfluous trailing parentheses in first example.

francisp · August 6, 2021, 3:46am

Yes, no more customization !

My ZMAI90’s like this

  - platform: mqtt
    name: "Mains Consumed Energy"
    state_topic: "electricity1/tele/RESULT"
    value_template: >- 
      {% set message = value_json.SerialReceived %}
      {% set payload = message[6:14] %}
      {% set payload_len = (payload | length) %}
      {% set result = namespace(value='') %}
      
      {% for i in range(0, payload_len + 1) | reverse -%}
        {%- if i is divisibleby 2 -%}
          {%- set result.value = result.value + payload[i:i+2] -%}
        {%- endif -%}
      {%- endfor -%}
      
      {{ (result.value|float) / 100 }}
    unit_of_measurement: 'kWh'
    unique_id: "mains_consumed_energy"    
    device_class: energy
    state_class: measurement
    last_reset_topic: 'electricity1/tele/RESULT'
    last_reset_value_template: '1970-01-01T00:00:00+00:00'

and my global consumption like this :

template:
  - sensor:
      - name: "House electricity energy"
        unit_of_measurement: 'kWh'
        state: >
          {{ (states("sensor.mains_consumed_energy") | float + states("sensor.upstairs_consumed_energy") | float) + (states("sensor.extra_consumed_energy") | float)  | round(2) }}       
        device_class: energy
        state_class: measurement      
        attributes:         
           last_reset: '1970-01-01T00:00:00+00:00'

Mariusthvdb · August 6, 2021, 11:49am

handt thought about that, mostly because I havent rebuilt my ‘legacy’ template sensors to the new template: integration yet. probably could have used

attribute_templates:
  last_reset: '1970-01-01T00:00:00+00:00'

but just didnt think of that. also because state_class: measurement isn’t supported on those legacy template sensors.

however, and this is HUGE news, with 2021.8.2 we can now use the template: in packages, and that is enough reason to spend the rest of my holidays rebuilding the template sensors in my config

so, good new all around!

iSteeb · August 8, 2021, 8:36am

I’m very new to MQTT and frankly almost all of this has gone entirely over my head…

I’m installing a Shelly EM tomorrow and was hoping to use it with MQTT (this was my plan before 2021.8 so it’s a really well timed update for me)!

Not a damn clue how to set it all up… why are there two sensors and more importantly are they both template sensors or what? I get the MQTT sensor but what the heck is the bit that Taras named ZP Import?