Best way to template definitions of MQTT 433MHz temp/humid sensors?

Configuration
,
1

Hi all,
Inside and outside my home I have more than a dozen of cheap 433 MHz temperature/humidity sensors like this:

61IUv3GoflL.AC_SL1500
61IUv3GoflL.AC_SL1500565×1500 32.3 KB

I’m structuring my configuration.yaml much like Frenck and others did, so I’m grouping those RF sensors in a rtl_433 directory containing one yaml file for each sensor. I welcome better solutions.

All those RF sensors all working well, but spanning any configuration patch/innovation/update on every sensor .yaml file is asking for troubles.
So I’m looking for a better way to define them, some alternative to copy/paste 50 yaml lines each.

In my dreams, the only difference could be something like assigning the unique “id” to a “parameter” like ${rtl433_temp_humid_id} (“190” in the example below) and then including a templated file.

My actual definition(s):

  ## Analog sensors: https://www.home-assistant.io/integrations/sensor.mqtt/

  # rtl_433/Nexus-TH/190/2 {"time":"2020-02-04 11:42:39","model":"Nexus-TH","id":190,"channel":2,"battery_ok":1,"temperature_C":20.4,"humidity":54}
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/190/+"
    name: "rtl433_NexusTH_190_rawtemp"
    device_class: temperature         # https://www.home-assistant.io/integrations/sensor/#device-class
    unit_of_measurement: "°C"
    expire_after: 180                 # Defines the number of seconds after the value expires if it’s not updated.
    force_update: true                # Sends update events even if the value hasn’t changed. Useful if you want to have meaningful value graphs in history.
    value_template: "{{ value_json.temperature_C }}"    # https://www.home-assistant.io/docs/mqtt/processing_json/
  - platform: filter                  # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.rtl433_NexusTH_190_rawtemp
    name: "rtl433_NexusTH_190_temperature"
    filters:
      - filter: outlier
        window_size: 4
        radius: 2.0
      - filter: lowpass
        time_constant: 10
        precision: 1
      - filter: time_simple_moving_average
        window_size: 00:20            # HH:MM
        precision: 1

  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/190/+"
    name: "rtl433_NexusTH_190_rawhumid" 
    device_class: humidity
    unit_of_measurement: "%"
    expire_after: 180
    force_update: true
    value_template: "{{ value_json['humidity'] }}" 
  - platform: filter
    entity_id: sensor.rtl433_NexusTH_190_umidity
    name: "rtl433_NexusTH_190_umidity"
    filters:
      - filter: outlier
        window_size: 4
        radius: 2.0
      - filter: lowpass
        time_constant: 10
        precision: 1
      - filter: time_simple_moving_average
        window_size: 00:20            # HH:MM
        precision: 1

BTW I’m open also to more efficient filtering tricks :wink: to smooth the graphing and reduce value changes, as each one generates a DB update.

Any hint?
Thanks,
Piero

2

As that 190 is used only on a few places of your file, I believe you could easily decrease size of your configs if you have multiple sensors in one .yaml file. Have a look at anchors

3

I wish we could use other files like the secrets.yaml file.

Honestly, you can’t use a template to generate a bunch of sensors. But you can use python…

Create your own custom integration. For inputs, take whatever you need. For example

# In configuration.yaml
my_custom_sensor_integration:
  sensor_ids:
    - "190_rawhumid"
    - "190_rawtemp"
    - "random_sensor"

Now have your custom integration generate and maintain each sensor.

That would be a pretty advanced solution.

The next step down would be to generate a python script that will output X sensors.yaml files for you. So any change you make can be propagated to every file without worry.

1 Like
4

@AhmadK
Thanks, I will give a look at anchors.
The solution I’m dreaming of is to define a list of homogeneous sensor(skeleton, parameter1, …, parameterN) in one or more files, so each one uses a common skeleton for its type.
That would be easy to maintain and more foolproof than copy/paste or complex tricks.

@jocnnor
Neat solution, unfortunately I’m quite fluent in Linux/C/bash but not in python, so I can’t help much here.

5

A bash script would work just as well for option #2. I would make this some script you run externally, not automatically from Home Assistant.

6

Yeah, it’s just a pre-(or post?)processor so you can use whatever you’re familiar with.

7

Thanks all, I will start trying anchors and aliases, which are standard and already available.

However, I just spotted a problem in my assumptions that raises another question: the mqtt payload received for each sensor is rtl_433/<protocol>/<sensor_id>/channel, i.e.:

rtl_433/Nexus-TH/132/1 {"time":"2020-02-16 10:48:07","model":"Nexus-TH","id":132,"channel":1,"battery_ok":1,"temperature_C":18.1,"humidity":56}
rtl_433/Nexus-TH/146/2 {"time":"2020-02-16 10:48:17","model":"Nexus-TH","id":146,"channel":2,"battery_ok":1,"temperature_C":17.5,"humidity":55}
rtl_433/Nexus-TH/211/2 {"time":"2020-02-16 10:48:22","model":"Nexus-TH","id":211,"channel":2,"battery_ok":1,"temperature_C":17.3,"humidity":56}
rtl_433/Nexus-TH/137/3 {"time":"2020-02-16 10:48:27","model":"Nexus-TH","id":137,"channel":3,"battery_ok":1,"temperature_C":17.4,"humidity":66}
rtl_433/Nexus-TH/23/3 {"time":"2020-02-16 10:48:37","model":"Nexus-TH","id":23,"channel":3,"battery_ok":1,"temperature_C":17.1,"humidity":58}
rtl_433/Nexus-TH/133/1 {"time":"2020-02-16 10:48:38","model":"Nexus-TH","id":133,"channel":1,"battery_ok":1,"temperature_C":16.9,"humidity":57}
...

This is because I use the following rtl_433 command in a systemctl rtl_433.service:

rtl_433 -F "mqtt://myrouter.local:1883,retain=0,events=rtl_433[/model][/id][/channel]

I thought each sensor [/id] was constant, however I just “discovered” it changes randomly after replacing the sensor batteries :frowning:

In some yaml config file should associate the sensor (almost friendly) name with its [/id], so I can easily update it.

I will try to solve this new challenge with anchors and aliases.
Anyway, hints on the simplest and neatest solution are always welcome :wink:
Piero

8

yeah, that’s the most annoying bit. sometimes they change codes when battery’s charge drops below certain level…

for some of my RFLink/Pilight sensors I have an automation and a python script that allow me to route these new codes to the sensors’ instances without restarting HA (as changes in sensors.yaml require restart). Did it for fun and it works :wink:

9

I’m halfway (or less) to my optimal solution.
This is my first of 12+ sensors definition, with anchors:

  # rtl_433/Nexus-TH/133/2 {"time":"2020-02-04 11:42:39","model":"Nexus-TH","id":133,"channel":2,"battery_ok":1,"temperature_C":20.4,"humidity":54}

  # 137 & anchors
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/137/#"
    name: "Temperature 433-137"         # Use a friendly name
    <<: &rtl_433_temp_sensor            # Define an anchor
      device_class: temperature         # https://www.home-assistant.io/integrations/sensor/#device-class
      unit_of_measurement: "°C"
      expire_after: 180                 # Defines the number of seconds after the value expires if it’s not updated.
      force_update: true                # Sends update events even if the value hasn’t changed. Useful if you want to have meaningful value graphs in history.
      value_template: "{{ value_json.temperature_C }}"    # https://www.home-assistant.io/docs/mqtt/processing_json/
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.temperature_433_137
    <<: &rtl_433_temp_sensor_filter     # Define an anchor
      filters:
        - filter: outlier
          window_size: 4
          radius: 2.0
        - filter: lowpass
          time_constant: 10
          precision: 1
        - filter: time_simple_moving_average
          window_size: 00:20            # HH:MM
          precision: 1
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/137/#"
    name: "Humidity 433-137"            # Use a friendly name
    <<: &rtl_433_humid_sensor           # Define an anchor
      device_class: humidity            # https://www.home-assistant.io/integrations/sensor/#device-class
      unit_of_measurement: "%"
      expire_after: 180                 # Defines the number of seconds after the value expires if it’s not updated.
      force_update: true                # Sends update events even if the value hasn’t changed. Useful if you want to have meaningful value graphs in history.
      value_template: "{{ value_json.humidity }}"    # https://www.home-assistant.io/docs/mqtt/processing_json/
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.humidity_433_137
    <<: &rtl_433_humid_sensor_filter    # Define an anchor
      filters:
        - filter: outlier
          window_size: 4
          radius: 2.0
        - filter: lowpass
          time_constant: 10
          precision: 1
        - filter: time_simple_moving_average
          window_size: 00:20            # HH:MM
          precision: 1

Then I define additional sensors using aliases:

  # 023 & aliases
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/23/#"
    name: "Temperature 433-023"         # Use a friendly name
    <<: *rtl_433_temp_sensor            # Define an anchor
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.temperature_433_023
    <<: *rtl_433_temp_sensor_filter     # Define an anchor
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/23/#"
    name: "Humidity 433-023"            # Use a friendly name
    <<: *rtl_433_humid_sensor           # Define an anchor
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.humidity_433_023
    <<: *rtl_433_humid_sensor_filter    # Define an anchor

  # 132 & aliases
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/132/#"
    name: "Temperature 433-132"         # Use a friendly name
    <<: *rtl_433_temp_sensor            # Define an anchor
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.temperature_433_132
    <<: *rtl_433_temp_sensor_filter     # Define an anchor
  - platform: mqtt
    state_topic: "rtl_433/Nexus-TH/132/#"
    name: "Humidity 433-132"            # Use a friendly name
    <<: *rtl_433_humid_sensor           # Define an anchor
  - platform: filter                    # https://www.home-assistant.io/integrations/filter
    entity_id: sensor.humidity_433_132
    <<: *rtl_433_humid_sensor_filter    # Define an anchor

... and so on ...

That works, and 15 lines are better than 45 lines for each additional sensor.
However it’s still far from my C-style 1-liner dream solution, i.e.:

- sensor_rtl433_th("132")
- sensor_rtl433_th("138")
- sensor_rtl433_th("211")
#... and so on ...

Looking for some simple command-line YAML macro parsers to periodically (cron or automation) auto-build this yaml config crowded of sensors, I found yamp.
It isn’t famous but it implements a keyword that’s magic for a C dev: #define :wink:
It’s written in Python, so it should not be a total alien in the HASS ecosystem.

As I like skeletons/templating for languages as YAML, I will continue to look for simpler solutions.

Then will inexorably come the second half of the problem: as @AhmadK pointed out, cheap 433MHz sensors does not have a MAC address and change their ID on some unpredictable conditions.

10

because they are not network devices. it’s a gateway that produces some IDs for us (and that ID sometimes fluctuates because of interference/low batteries level/god knows what else).

speaking about yamp - let us know if it works for you (note it’s a Python 2.7 code) as I’m hoping to try out this one as it allegedly supports cross-file anchors (but might lack some standard features as per comments).

11

Yes, it was a sad rant :cry:
BTW, bluetooth devices have a unique MAC even if they aren’t strictly network devices :wink: but 433 MHz sensors aren’t so smart.

An alternative could be replacing 433MHz sensors with ESP8266-based WiFi equivalents, plus the usual deep-sleep and other tricks to reduce updates and extend battery life. After all, the temperature does not always change every 60 seconds :wink:
http://www.aliexpress.com/wholesale?SearchText=esp-01s+temperature+humidity

@AhmadK in your automation+python job to reassign those changed IDs, how did you solve the problem of associating one (or more) new ID to the correct “friendly_named” sensor, such as “Kitchen indoor”?

12

as per this discussion it’s not always possible because of high power consumption and difficulty of deep sleep stuff.

I did it quite straightforward. This bit is specific to RFLink that creates sensors with generated names.

# it uses the fact that rflink creates sensor.xxx_update_time
# and updates its value every time new readings (_temp and _hum) arrive
# so we can find out the original sensors' names
# and copy their values into the target sensors (all temperature, humidity and update_time)
# that means the target sensors are updated every time, not only when it's changed!
- alias: sensor_thb_rflink_route_value_to_entity
  trigger:
    platform: state
    entity_id:
      - sensor.dummy_update_time
      # reception
      # lounge
      # kitchen
  action:
    service: python_script.sensor_thb_rflink_route_value_to_entity
    data_template:
      object_id : "{{ trigger.to_state.object_id }}"
      value     : "{{ trigger.to_state.state }}"

and then

arg                     = 'object_id'
object_id               = data.get(arg)
assert object_id, "Please specify %s!" % arg

arg                     = 'value'
value                   = data.get(arg)
assert value, "Please specify %s!" % arg

arg                     = 'force_update'
force_update            = data.get(arg, 'True').lower() == 'true'

update_time_suffix      = '_update_time'

# figure out the base name if the sensor from object_id
base_object_id = object_id.replace(update_time_suffix, '')

known_sensors = {
    'prologue_97b0' : 'rflink_ground_floor_reception',
    'prologue_9100' : 'rflink_ground_floor_lounge',
    'prologue_9440' : 'rflink_ground_floor_kitchen'
}

if base_object_id in known_sensors:
    cmd                     = 'set_state'
    attr_entity_id          = 'entity_id'
    attr_state              = 'state'
    all_siffixes            = {
        'temp': 'temperature',
        'hum': 'humidity'
    }

    for source_suffix, target_suffix in all_siffixes.items():
        source_entity_id = "sensor.{}_{}".format(base_object_id, source_suffix)
        source_object = hass.states.get(source_entity_id)
        assert source_object, "Cannot find %s" % source_entity_id

        source_value = source_object.state

        target_entity_id = "sensor.{}_{}".format(known_sensors[base_object_id], target_suffix)

        # update state only if it's different
        if not force_update:
            target_object = hass.states.get(target_entity_id)
            assert target_object, "Cannot find %s" % target_entity_id
            targer_value = target_object.state

            if source_value == targer_value:
                logger.debug("%s: state (%s) is the same, not updating", target_entity_id, source_value)
                continue

        logger.debug("%s(%s: %s, %s: %s)", cmd, attr_entity_id, target_entity_id, attr_state, source_value)
        hass.services.call('python_script', cmd, {attr_entity_id: target_entity_id, attr_state: source_value} )

    # and finally set state of the target _update_time sensor as we have the source value already and it should be always updated (as it's always different)
    target_entity_id = "sensor.{}{}".format(known_sensors[base_object_id], update_time_suffix)
    logger.debug("%s(%s: %s, %s: %s)", cmd, attr_entity_id, target_entity_id, attr_state, value)
    hass.services.call('python_script', cmd, {attr_entity_id: target_entity_id, attr_state: value} )
else:
    logger.warning("sensor.%s not configured", base_object_id)

With OMG Pilight it’s slightly easier as I only need to add a new ID to a python script

id          = int(float(data.get('id', -1)))
temperature = data.get('temperature')
humidity    = data.get('humidity')
battery     = data.get('battery')

known_entities = {
    39      : 'ground_floor_reception',
    16      : 'ground_floor_lounge',
    68      : 'ground_floor_kitchen'
}

if id in known_entities.keys():
    hass.services.call(
        'python_script',
        'sensor_thb_pilight_set_values',
        {
            'object_id_prefix' : "sensor_pilight_{}_".format(known_entities[id]),
            'temperature'   : temperature,
            'humidity'      : humidity,
            'battery'       : battery
        }
    )
else:
    logger.warning("id %i not configured", id)
13

Thanks @AhmadK, to me that sounds cool and a good learning opportunity!

14

you’re welcome!
that’s still not completed (as you have to make changes in both an automation and a script) but as I don’t use it much I’ve just left it as it is until I have time to re-design it a bit.

15

Hi @PieBru,

I am trying to do the same with similar to your Nexus sensors.
I have this output from RTL_433
grab 2020-06-22 alle 20.55.13

now I believe I should replace here:

rtl_433 -F "mqtt://myrouter.local:1883,retain=0,events=rtl_433[/model][/id][/channel]

“id” with what…“house_code”?

how can I check what is the right name for “House Code” a I believe capitals and space should be replaced by something…

16

@mspinolo Ciao Michele,
[/id] is a keyword, on the given ‘rtl_433’ command you should change only the broker IP address.
Piero