Thanks buddy, You gave me some useful tips here.
I will definitely implement them.
The link you attached would not work for my needs since I don’t want a time triggered notification Nor adding state triggers for each sensor.
My example is using a custom template, once it works, it will report any battery sensor as soon as it becomes unresponsive or unavailable or crosses the low-battery threshold (40%)
It works just fine now but I will be implementing your suggestion to narrow the for-loop scope to the desired domain.
You have been most helpful, not just here, I read a lot of your posts here and they are gold!
@123
I finally found a solution that I am comfortable with.
this is how I set my binary_sensor, without repeating templates and with a bit of clever coding:
binary_sensor:
- platform: template
sensors:
battery_devices_issues:
friendly_name: 'Battery Devices Issues'
device_class: problem
value_template: >-
{% set batteryDevicesStatus = (state_attr('binary_sensor.battery_devices_issues', 'all_battery_devices_status')) %}
{{batteryDevicesStatus.empty_battery_devices|count+batteryDevicesStatus.non_responsive_devices|count>0}}
attribute_templates:
all_battery_devices_status: >-
{% set vars = namespace(
emptyBatteryDevices = '',
nonResponsiveDevices = '',
lowBatteryDevices = '',
goodBatteryDevices = ''
) -%}
{%- for curr_entity in (expand(states.sensor) | selectattr('entity_id') | list) -%}
{%- if curr_entity.entity_id.endswith('_battery_level') -%}
{%- set _batteryDevice=curr_entity.entity_id -%}
{%- set _sensorName= curr_entity.entity_id.split('.')[1].split('_battery_level')[0] -%}
{%- set _batteryLevel = states[_batteryDevice].state -%}
{%- set _temperatureSensor = 'sensor.'~_sensorName~'_temperature' -%}
{%- set _lastChanged = states[_temperatureSensor].last_changed -%}
{%- if _lastChanged!=null -%} {#probably a phone, not a sensor!#}
{%- set _hoursPassed = (((as_timestamp(now()) - as_timestamp(_lastChanged)) | int) % 86400)//3600 %}
{%- if _batteryLevel=='unavailable' -%}
{%- set vars.emptyBatteryDevices = vars.emptyBatteryDevices~"," if vars.emptyBatteryDevices!="" -%}
{%- set vars.emptyBatteryDevices = vars.emptyBatteryDevices~ "'" ~ _sensorName ~ "'" -%}
{%- elif _hoursPassed>=2 -%}
{%- set vars.nonResponsiveDevices = vars.nonResponsiveDevices~"," if vars.nonResponsiveDevices!="" -%}
{%- set vars.nonResponsiveDevices = vars.nonResponsiveDevices~ "'" ~ _sensorName ~ "'" -%}
{%- elif _batteryLevel|int <40 -%}
{%- set vars.lowBatteryDevices = vars.lowBatteryDevices~"," if vars.lowBatteryDevices!="" -%}
{%- set vars.lowBatteryDevices = vars.lowBatteryDevices~ "'" ~ _sensorName ~ "'"-%}
{%- else -%}
{%- set vars.goodBatteryDevices = vars.goodBatteryDevices~"," if vars.goodBatteryDevices!="" -%}
{%- set vars.goodBatteryDevices = vars.goodBatteryDevices~ "'" ~ _sensorName ~ "'" -%}
{%- endif -%}
{%- endif -%}
{%- endif %}
{%- endfor -%}
{%- set batteryDevicesStatusJSON = "{'empty_battery_devices':["~vars.emptyBatteryDevices~"],
'non_responsive_devices':["~vars.nonResponsiveDevices~"],
'low_battery_devices':["~vars.lowBatteryDevices~"],
'good_battery_devices':["~vars.goodBatteryDevices~"],
}" -%}
{{batteryDevicesStatusJSON}}
I am setting a JSON in a custom attribute that I can access from anywhere in home assistant + the sensor accesses it’s own custom attribute to determine it’s state.
Now I am getting notifications as I wanted, no need to update any scripts when I add sensors.
1 Like
If you’re interested, I refactored the template using several techniques that help to reduce its length and complexity.
I have created several battery_level and temperature sensors (as Template Sensors) in order to test the streamlined version of the template. It appears to work as desired but the acid-test will, of course, be your system’s sensors.
battery_devices_issues:
friendly_name: 'Battery Devices Issues'
device_class: problem
value_template: >-
{% set status = (state_attr('binary_sensor.battery_devices_issues', 'status')) %}
{{ status.empty|count + status.dead|count > 0 }}
attribute_templates:
status: >-
{% set ns = namespace(empty = [], dead = [], low = [], good = []) %}
{% for entity in expand(states.sensor) if entity.entity_id.endswith('_battery_level') %}
{% set name = entity.object_id[:-14] %}
{% set level = states(entity.entity_id) %}
{% set changed = states['sensor.'~name~'_temperature'].last_changed %}
{% if changed != null %}
{% if level == 'unavailable' %} {% set ns.empty = ns.empty + [name] %}
{% elif now() - changed >= timedelta(hours=2) %} {% set ns.dead = ns.dead + [name] %}
{% elif level|int < 40 %} {% set ns.low = ns.low + [name] %}
{% else %} {% set ns.good = ns.good + [name] %}
{% endif %}
{% endif %}
{% endfor %}
{{ "{{\"empty\": {}, \"dead\": {}, \"low\": {}, \"good\": {} }}".format(ns.empty, ns.dead, ns.low, ns.good) }}
What has changed
Although I changed the names of many variables, that’s purely cosmetic (feel free to replace them with your original names ). The important changes are:
- The type of the namespace variables is list instead of string. This alone eliminates a lot of “list-building” code.
- The for-loop incorporates the initial if-statement in order to reduce iterations.
- Parsing the sensor’s root-name is simplified by referencing the entity’s
object_idand slicing off the last 14 characters. - Eliminating variables for expressions that are only used once.
- Simplifying the time calculation by using native datetime and timedelta objects.
- Generate the final JSON string using the
formatmethod.
2 Likes
Awesome!
I started coding for home assistant a couple of weeks ago, I am still learning the framework (and jinja).
The HA documentation is lacking, especially in the more complex scenarios, I appreciate your help.
I will test it and report back.
[UPDATE]
Works as expected!
Glad to hear it passed the test.
I made one more simplification. This line contains unnecessary filtering:
{% for entity in (expand(states.sensor)|selectattr('entity_id')|list) if entity.entity_id.endswith('_battery_level') %}
and is reduced to this:
{% for entity in expand(states.sensor) if entity.entity_id.endswith('_battery_level') %}
You should know that the use of states.sensor means this template will be evaluated every time any of your sensors changes state. If battery-powered devices comprise the majority of your sensors then using states.sensor is fine. If they represent the minority then states.sensor casts much too wide a net.
The simplest technique to improve efficiency is to create a group containing all battery-powered entities. The for-loop statement is reduced to:
{% for entity in expand('group.battery_powered') %}
and the template is evaluated only when one of the group’s members changes state (as opposed to any and all sensors in your system).
If manually creating the group seems like an onerous task then one could use an automation, triggered at startup, to automatically create the group and populate it with sensors whose names end with the phrase “battery_level”. The only potential ‘gotcha’ with this technique is if, on startup, Home Assistant creates the Template Binary Sensor before the automation creates the group. 
The sequence of what gets created on startup becomes important here.
Well, about 90% of my sensors are battery powered, I advocate for efficiency most of the time, but at a point where efficiency costs convenience or risks bugs, it simply doesn’t worth it.
I am running Home assistant on a VM on a strong machine, If needed, I can throw more resources at it but right now, it doesn’t go passed 5% CPU so I can probably run it on a much weaker machine.
I am coding a smart-home that works, with minimal intervention of my behalf. I had 4 different smart-home platforms over the passed 17 years (I started with X10). reporting devices and sensors health is a huge part of a functioning system. it deserves the resources and not having to update scripts when adding new sensors is important for future-proofing your smart home setup.
As I explained, it’s entirely possible to create a self-updating group that permits the Template Binary Sensor to monitors only battery-powered sensors; it works with minimal intervention on your behalf. Aside from improved efficiency, it offers the versatility and convenience of a group (of all battery-powered sensors) that can be referenced by other Template Sensors, automations, etc.
The concept of self-updating groups gained more interest when, several versions ago, Home Assistant lost default groups like All Lights, All Switches, etc. The belief was that many users didn’t need the default groups so the development team felt they introduced needless overhead. For the users who did use the default groups, the technique I described manages them without the bother of manual maintenance.
FWIW, my first foray into home automation employed X10 ActiveHome software which I replaced with MisterHouse for awhile. I also experimented with HouseBot (and a few others) but ultimately chose Premise back in 2007. I have been using it ever since; Premise and Home Assistant work together to automate my home. My so-called legacy devices, for which no integrations exist in Home Assistant, continue to be managed by Premise and exposed to Home Assistant via MQTT.
1 Like
Glad to be speaking to a fellow smart-home veteran.
My current set up is a Lutron Homeworks system that I installed 15 years ago, it is hard-wired to every outlet and every light at my house, it still works and the web interface I wrote for it works as a great redundancy when everything else fails (as more complex systems sometimes do).
I wrote Telnet-MQTT-WebAPI bridge for it and brought it with me to my next set ups: alexa, apple homekit, smartthings and now home assistant.
The rest of my smart network is a mesh of zigbee sensors, broadlink remotes, some raspberries and arduino based projects that controls locks, irrigation and a bunch of APIs for every device at my home that supports any kind of connectivity.
My latest and proudest is a smart grill I built with a blow fan and sensors to control the fire temperature.
I am migrating all of the above to home assistant because I want local execution, for security, future proofing, internet connection independence, and faster response times.
How long have you been working with home assistant? You seem to know your way around the framework and templating.
I just confirmed that the proposed group-based technique works fine on startup.
The following automation creates group.battery_powered and populates it with all sensors whose names end with “_battery_level”. It does this on startup and whenever Reload Groups is executed (more triggers can be added).
- alias: 'Create Groups'
trigger:
- platform: homeassistant
event: start
- platform: event
event_type: 'call_service'
event_data:
domain: 'group'
service: 'reload'
action:
- service: group.set
data:
object_id: battery_powered
name: 'Battery Powered'
entities: >
{% set ns = namespace(sensors = []) %}
{% for entity in states.sensor if entity.entity_id.endswith('_battery_level') %}
{% set ns.sensors = ns.sensors + [entity.entity_id] %}
{% endfor %}
{{ ns.sensors }}
The Template Binary Sensor now monitors fewer sensors. The for-loop statement is reduced to:
{% for entity in expand('group.battery_powered') %}
Like so:
battery_devices_issues:
friendly_name: 'Battery Devices Issues'
device_class: problem
value_template: >-
{% set status = (state_attr('binary_sensor.battery_devices_issues', 'status')) %}
{{ status.empty|count + status.dead|count > 0 }}
attribute_templates:
status: >-
{% set ns = namespace(empty = [], dead = [], low = [], good = []) %}
{% for entity in expand('group.battery_powered') %}
{% set name = entity.object_id[:-14] %}
{% set level = states(entity.entity_id) %}
{% set changed = states['sensor.'~name~'_temperature'].last_changed %}
{% if changed != null %}
{% if level == 'unavailable' %} {% set ns.empty = ns.empty + [name] %}
{% elif now() - changed >= timedelta(hours=2) %} {% set ns.dead = ns.dead + [name] %}
{% elif level|int < 40 %} {% set ns.low = ns.low + [name] %}
{% else %} {% set ns.good = ns.good + [name] %}
{% endif %}
{% endif %}
{% endfor %}
{{ "{{\"empty\": {}, \"dead\": {}, \"low\": {}, \"good\": {} }}".format(ns.empty, ns.dead, ns.low, ns.good) }}
I joined this community forum in October 2018. At the time I was experimenting with openHAB (for several months). I decided I preferred something that behaved more like Premise so I began exploring Home Assistant. As I slowly became comfortable with it, I took my time to carefully transfer/convert what I had created in Premise to Home Assistant. I waited until Home Assistant’s UPB integration was finished before transferring control of all lighting. I don’t recall when Home Assistant was no longer in “test” but in “production” in my home but it’s between 1 and 2 years now.
In Premise, I developed several integrations notably for the ELK M1 security panel, Environment Canada weather service, HAI Omnistat2 RC-2000 thermostat, etc. Most of my work is posted on the Cocoontech.com site (best home automation community back in the day; now just a shadow of its former self). The last one I created (early 2018) was an MQTT integration to allow Premise to communicate with other systems. That’s when I started dabbling with openHAB followed by Home Assistant in late 2018.
What I haven’t mentioned is that development of Premise was halted in 2006 and the product became freely available to the public (original price was over US$1000). I jumped on board in 2007. It’s a testament to its advanced architecture that users were able to continue enhancing it even though it remained closed-source.
Anyway, all ancient history now.
I rather think of it as evolution. Some of the dinosaur code I wrote is still running. 
BTW:
Just a suggestion, I won’t be implementing groups, but if I had, I would have used your automatic group technique to create groups for each battery state “dead”, “good”, “low” & “empty”.
That’s inadvisable because the group’s membership is defined exclusively at the moment the group is created. If a sensor’s battery status changes afterwards, it won’t be reflected in the group’s membership. A Template Sensor is better suited for the task because it listens for state-changes (for all entities detected in its template).
I suppose one could use an automation to monitor all battery-powered sensors then update the groups whenever a sensor changes state. However, the automation’s trigger would have to explicitly list each sensor and that fails to meet the objective of avoiding manual maintenance (i.e. if you add a new battery-powered sensor you’ll have to manually update the automation’s trigger).
Are you sure it can’t be done with a template trigger somehow?
Well, I am just learning the framework and already moved on to the next task in a loooooong ToDo list.
However, it feels like Template Trigger is a good hook, you just need to figure out the right template.
can’t you parse the status attribute of our battery_devices_issues for changes?
Review the documentation’s description of what a Template Trigger’s template must do in order to qualify as being a trigger. It’s not merely state-changes (like a State Trigger).
I read it and still thinking it is possible.
working with only a boolean expression is not ideal, but is possible.
The “boolean state” you need to track is:
Is there a change to the battery_devices_issues (our binary_sensor) status attribute.
I am not familiar with the framework as you are, but it sounds like you just need a place to store state and compare changes. (throwing ideas: maybe another attribute inside battery_devices_issues ,maybe another sensor that turns on if there are changes and turns off again, then you can just pick it up in any automation trigger)
I worked on systems far less capable than Home Assistant, (one had only boolean variables to use for states) I still managed to build around the restrictions, you just need to think out of the box and willing to invest long hours.
If the goal is to use a Template Trigger in an automation that update the groups for battery-powered devices, the template can’t be based on “our binary_sensor” because that’s already computing empty/dead/low/good devices which obviates the need for creating groups.
If “our binary_sensor” relies on group.battery_powered, it also cannot be used for the Template Trigger because then it becomes a chicken-egg scenario.
A Template Trigger evaluates a template whenever any of the entities within the template changes state. However, it’s not the entity’s state-change that causes the trigger (it only initiates evaluation) it’s the result of the template’s evaluation that determines if it should trigger or not. That’s what the documentation means when it describes the template evaluating to true/false.
A State Trigger triggers when any of its entities changes state. Period. However, you cannot use a template to dynamically generate the list of entities to be monitored by the State Trigger; you must specify them individually. As mentioned previously, that doesn’t meet the goal of eliminating manual maintenance.
There’s also the Event Trigger. It can be configured to trigger on any state-change for all entities or a specific domain (and other variations). The catch is that it introduces precisely what a group was originally used to avoid, namely avoid constantly monitoring the activities of more entities than is necessary.
No, the goal is having a way to automatically create groups of battery powered devices by their battery status.
- You can use the sensor I wrote to create these groups.
- You can also not use my sensor, only the template I wrote and use it to create\update the groups.
This discussion is stretching for too long, I don’t mind ending it with a disagreement.
The sensor already contains the information that the groups would contain. The groups would duplicate information that already exists. It also strays from the original discussion (which is fine but makes it challenging to follow the conversation’s direction).
This discussion, about using a group, started in this post where I suggested using a group as a means to reduce the number of sensors to be monitored by the Template Binary Sensor. I subsequently supplied the means to do it.
You replied:
I won’t be implementing groups, but if I had, I would have used your automatic group technique to create groups for each battery state “dead”, “good”, “low” & “empty”.
To which I explained it was inadvisable because the technique I proposed only creates them at startup/reload. It would need a trigger that monitors each battery-powered entity’s state … which brings us back to manually maintaining a State Trigger (which is undesirable and negates the advantage of a self-updating group).
You suggested maybe a Template Trigger could be “somehow” employed (except it cannot). Then you proposed the binary_sensor itself could trigger the automation to create the groups but that’s, depending on how you look at it, redundant or circular (redundant because the groups would simply duplicate the binary_sensor’s existing information; circular if the binary_sensor itself relies on a group).
Anyway, I agree the discussion is getting long in the tooth. Good luck.