Numeric_state trigger in automation. False executions

[knst]

I have the following automation configured

- alias: Turn on lights when it gets dark while anybody home
  trigger:
    platform: numeric_state
    entity_id: sensor.kinoteatr_datchik_osveshchennosti_165
    value_template: "{{ states.sensor.kinoteatr_datchik_osveshchennosti_165.state | float }}"
    below: 30
    for: 00:05:00
  condition:
    condition: state
    entity_id: binary_sensor.occupied
    state: 'on'
  action:
    - service: scene.turn_on
      entity_id: scene.kitchen_light_on    

the expected behaviour is turn on the lights when lux sensor changes from value above 30 to value below 30 with 5 min delay.

It works in most cases. But the automation is triggered when lux sensor changes from 0 to anything in range 1-30 that is definitely wrong.

Actually there are 2 automations this one and another that turns lights off when lux sensor gets above 40. Idea is to turn on lights if it is too dark and off when it gets bright enough again.

So I get lights turned on when it is rainy and off when the sun returns also the lights are automatically turned on in the dusk. But in the early morning when sun rises lights are turned on while I do no need it - there is no moment when lux value crosses 30 threshold but automation is still triggered…

Need help with getting it work or any hint on why numeric_state may work like this…

[Mariusthvdb (Marius)]

@knst

value_template: “{{ states.sensor.kinoteatr_datchik_osveshchennosti_165.state | float }}”

you should leave this out, no need for that, the numeric_state uses the entity_id, which you have declared already

you should use the template if you need an attributes: Automation Trigger - Home Assistant

but since your using state here, thats not needed

as to the trigger itself: you might want to experiment creating an extra binary_sensor, with value ‘on’ if the state is below 30 for 5 minutes.

binary_sensor:
  - platform: template
    sensors:
      kinoteatr:
        friendly_name: 'Kinoteatr'
        value_template: >
          {{ states('sensor.kinoteatr_datchik_osveshchennosti_165')|float < 30 }}
        delay_on:
          minutes: 5

and then use that binary_sensor as trigger in the automation.
Rule out further false triggers in the condition part. Maybe a timing setting, or other conditions to narrow down triggering you don’t like

[knst]

actually value_template is attempt to fix the described behaviour. I assumed that for some reason sensor value may come as str rather than float (sensor is registered with fibaro z-wave integration).

dedicated binary_sensor was my next plan, I actually do not like it since it means I need to do this for all places where I have numeric_state and give up using numeric_state at all.

For me it looks like a bug - probably HA forgets previous value of sensor at some conditions an thus change from 0 to 1 triggers below: 30 while it should trigger only if previous value is > 30

[Mariusthvdb (Marius)]

be sure to have a correct config before you start thinking about bugs. You might nor receive an error, but the current config seems a bit flakey tbh.

if you want to try the template, then use the template trigger: https://www.home-assistant.io/docs/automation/trigger/#template-trigger

id try it with
{{ states('sensor.kinoteatr_datchik_osveshchennosti_165') | float }}

to rule out errors for the unitialized sensor

[knst]

Thx. Will try it today.

[pnbruckner (Phil)]

@knst

But the automation is triggered when lux sensor changes from 0 to anything in range 1-30 that is definitely wrong.

Actually, that’s not correct. The way the numeric_state trigger is implemented, it is evaluated whenever one of the referenced entities changes. The first time this happens, if the above/below/for condition(s) is/are met, it will trigger, even though it hasn’t “crossed the boundary.” This is the way it has always worked.

So, in your scenario, if the first time sensor.kinoteatr_datchik_osveshchennosti_165 changes (after HA started) its “numeric” value is less than 30, and stays below 30 for 5 minutes, it will trigger. That is how it has always worked (except that the for option was recently added.)

Once the trigger has “fired”, it won’t fire again until the condition(s) is/are first not met, and then is/are met again. So, again, in your case, the sensor’s numeric value would have to go to 30 or above, and then back below 30, and stay below 30 for 5 minutes, for it to trigger again.

Now there are a few scenarios under which this behavior will change. If the sensor’s state ever “disappears” from the State Machine, or the state of the sensor is ever unavailable or unknown or cannot be parsed as a float, or there is an error when the value_template template is evaluated, it will “forget” if the trigger has previously fired, so the next time the sensor changes, if the conditions are met (and none of the above described scenarios are present), then it will trigger. My guess is something like this is happening. You might want to review the history of the sensor. Also, in some of the above mentioned scenarios (i.e., if the state cannot be parsed as a float, or there is an error while evaluating the value_template) there should be errors in the log. (Unfortunately, there will not be any errors if the sensor’s state is unavailable or unknown, or if the entity disappears temporarily from the State Machine.)

As @Mariusthvdb said, you do not need the value_template in your case. I’d suggest removing it and seeing how it behaves for you. If you still think there’s an issue, let me know and I can help you debug it. I was the last one to significantly “touch” the numeric_state trigger code (to add the for option), so I should be able to get to the bottom of this.

[pnbruckner (Phil)]

BTW, I don’t necessarily agree with the way it currently works. If it were just up to me, I’d change it so either it triggered immediately after HA starts if the condition(s) is/are met (i.e., without waiting for an entity to change state), OR completely do away with this initial triggering if the condition(s) is/are met and only allow it to fire if the condition(s) is/are first not met, and then is/are met (i.e., must cross the boundary.) I really never liked the way it might or might not trigger initially. But I didn’t write it in the first place, and changing this behavior would be a significant breaking change, so I never pursued it.

[knst]

Thank you for the detailed comment.

It is exactly what I suppose to happen - value of the sensor is lost for some reason and next time sensor value changes automation is triggered.
So I added value_template (it behaved quite the same without it) to cast the sensor value to float in case it comes as str, but I just realized that I’ve done it wrong - if the cast fails it will cause the same behaviour as if the value is lost.

The scenario is following

1. HA starts
2. the sensor first value is 0 and automation is triggered (I’ve rebooted HA yesterday when it was already dark)
3. in the morning when the sensor changes from 0 to 1 the automation is triggered

There are no records in log mentioning sensor id and HA has not restarted between #2 and #3.
The same happened yesterday and day before

Will try to add extra logging to numeric_state code.

[pnbruckner (Phil)]

@knst

So I added value_template (it behaved quite the same without it) to cast the sensor value to float in case it comes as str

All states are strings, and all templates return strings. So it does nothing. The code will attempt to convert it to a float anyway.

@knst

if the cast fails it will cause the same behaviour as if the value is lost

Yep.

You could maybe try this:

value_template: "{{ state.state|default(30)|float(30) }}"

If the state doesn’t exist the default will kick in. If the state can’t be converted to a float (e.g., it’s unknown or unavailable), the float default value (i.e., the value in the float filter) will kick in. If the state exists and it can be converted to a float, then it will be used.

[knst]

added logging here

I’d prefer to ensure that the problem is in unknown/unavailable value before adding default or any other workaround…

BTW casting error log has swapped value and entity here
guess it should be

        _LOGGER.warning(
            "Value cannot be processed as a number: %s " "(Offending entity: %s)",
            value, #<---- value should be the first parameter
            entity,

[pnbruckner (Phil)]

Sounds good. Let me know how it goes.

Yep, looks like that mistake was made when that warning message was enhanced to show the entity. See #15021. LOL, looking at that, it appears the mistake was caught, but it wasn’t fixed. Yikes!

[123 (Taras)]

< soapbox >
Hacktoberfest’s goal ought to have been to tackle these ‘fit and finish’ issues prior to any thoughts of a 1.0 release (… whenever that is). For something that is designed to control things, it shouldn’t behave one way at startup and another way thereafter. Worst part is that I suspect the majority of users aren’t aware of this ‘split-personality’ (I know I wasn’t).
</ soapbox >

[knst]

I’ve reproduced the issue and collected log with extra output added to numeric_state

To remind the situation: there is lux sensor connected to HA with fibaro integration and trigger for automation that executes when the sensor value gets below 30 for 5min

The problem looks to be the following

1. HA is rebooted having last reported sensor value equal to 0
2. Sensors do not report any value for a long time (it is night and it had reported 0 before HA reboot and as it is still dark sensor has nothing to report), HA just stores the state before reboot
3. At some point, sensor reports value <30 and automation is executes in 5 min after it

The expected behaviour is ether execute trigger in 5 min after boot (as you described) or not execute it till value gets >30 and back to <30

here is the log.

Oct 5 00:46:39 hass-01 hass[24482]: 2019-10-05 00:46:39 INFO (MainThread) [homeassistant.components.automation] Initialized trigger Turn on lights when it gets dark while anybody home
Oct 5 00:54:01 hass-01 hass[24482]: 2019-10-05 00:54:01 WARNING (MainThread) [homeassistant.helpers.condition] TEMPORARY LOG: Value is 12.0 (entity: <state sensor.kinoteatr_datchik_osveshchennosti_165=12.0; fibaro_id=165, unit_of_measurement=lx, friendly_name=Кинотеатр датчик освещенности, device_class=illuminance @ 2019-10-05T00:54:01.243060+03:00>)
Oct 5 00:54:01 hass-01 hass[24482]: 2019-10-05 00:54:01 WARNING (MainThread) [homeassistant.helpers.condition] TEMPORARY LOG: Value is 12.0 (entity: <state sensor.kinoteatr_datchik_osveshchennosti_165=12.0; fibaro_id=165, unit_of_measurement=lx, friendly_name=Кинотеатр датчик освещенности, device_class=illuminance @ 2019-10-05T00:54:01.243060+03:00>)
Oct 5 00:59:02 hass-01 hass[24482]: 2019-10-05 00:59:02 INFO (MainThread) [homeassistant.components.automation] Executing Turn on lights when it gets dark while anybody home
Oct 5 00:59:02 hass-01 hass[24482]: 2019-10-05 00:59:02 INFO (MainThread) [homeassistant.helpers.script] Script Turn on lights when it gets dark while anybody home: Running script
Oct 5 00:59:02 hass-01 hass[24482]: 2019-10-05 00:59:02 INFO (MainThread) [homeassistant.helpers.script] Script Turn on lights when it gets dark while anybody home: Executing step call service

00:46:39 - trigger is initialized (automation was not executed in 5 min after it)
00:54:01 - I turned on not bright light near the sensor and it reported 12lx
00:59:02 - in 5 min after 12lx value is reported automation was executed

[knst]

@pnbruckner

BTW, I don’t necessarily agree with the way it currently works. If it were just up to me, I’d change it so either it triggered immediately after HA starts if the condition(s) is/are met (i.e., without waiting for an entity to change state), OR completely do away with this initial triggering if the condition(s) is/are met and only allow it to fire if the condition(s) is/are first not met , and then is/are met (i.e., must cross the boundary.) I really never liked the way it might or might not trigger initially. But I didn’t write it in the first place, and changing this behavior would be a significant breaking change, so I never pursued it.

I’d say there should be some parameter for numeric_state to select the behaviuor with default value meaning the current implementation and two other options - ‘fire on startup if value is in range’ and ‘fire only on actual crossing the boundary’

[Mutt (Muttley)]

You only need one of the last two as they are mutually exclusive
Though I’m hard pressed to think of ANY case where you’d need it to fire without crossing as that is implicitly what the automation states.
If you want alternate behaviour then check on start up and fire accordingly.
It does seem non-intuitive the way it is currently.

Edit: Taking this to illogical extremes though : -
If you wanted a pump to cool a nuclear reactor, you’d definately want it running by default if ‘above temp’

[pnbruckner (Phil)]

@Mutt

If you wanted a pump to cool a nuclear reactor

… you wouldn’t be controlling it with open source software.

[pnbruckner (Phil)]

Ignoring for now how maybe it should work, and considering how it does, I agree with @Mutt, you can add a trigger & condition to check at startup and run the action(s) if the sensor’s value meets the original trigger’s condition. In fact, this is what I do with many of my automations. And, since some sensors don’t have a value immediately after startup, I use a delayed startup event. So something like this:

- trigger:
    platform: homeassistant
    event: start
  action:
  - delay: 15
  - event: DELAYED_HOMEASSISTANT_START
- trigger:
  - platform: numeric_state
    entity_id: sensor.kinoteatr_datchik_osveshchennosti_165
    below: 30
    for:
      minutes: 5
  - platform: event
    event_type: DELAYED_HOMEASSISTANT_START
  condition:
  - condition: state
    entity_id: binary_sensor.occupied
    state: 'on'
  - condition: numeric_state
    entity_id: sensor.kinoteatr_datchik_osveshchennosti_165
    below: 30
  action:
    ...

[Mutt (Muttley)]

Hey, given the number of nuclear reactor ‘incidents’ in my recent memory - Are you ‘sure’ they don’t ?

[knst]

My intention was to fire automation only when the sensor value crosses the boundary and ignore HA restarts. I want lights to be turned on when it gets dark and turned off when the sun is bright enough, but not when HA restarts.

Maybe I’m missing something but in your example automation would be triggered twice on start and on next value change…

Actually I solved it with template sensor that is on when lux level reported by sensor is < 30 and trigger fires when template sensor value changes from ‘off’ to ‘on’

- platform: template
  sensors:
    it_is_dark_outside:
      entity_id:
        - sensor.kinoteatr_datchik_osveshchennosti_165
      value_template: "{{ states('sensor.kinoteatr_datchik_osveshchennosti_165') |  float < 30 }}"

- alias: Turn on lights when it gets dark while anybody home
  trigger:
    platform: state
    entity_id: binary_sensor.it_is_dark_outside
    from: 'off'
    to: 'on'
    for: 00:05:00
  condition:
    condition: state
    entity_id: binary_sensor.occupied
    state: 'on'
  action:
    - service: scene.turn_on
      entity_id: scene.kitchen_light_on 

Anyway, now I understand how HA works much better. Thank you.

[Mutt (Muttley)]

An alternative and slightly simpler method : -
Have an input_boolean set initial: ‘on’ and prevent any ‘extraneous’ automations from running if the bit is set.
Then 10secs after hass start, reset the bit.
I have a bathroom fan that always comes on at a restart (from an automation), never found out why.