How Bayes Sensors work, from a Statistics Professor (with working Google Sheets!)

Sadly, each template sensor kinda has it’s own implementation. There are functions to get the entities used in the template and functions to evaluate the template. So you can decide when a template is evaluated in your own component, but you can’t reuse the implementation of say the binary_sensor template implementation.

Basically we now have some sort of circular reasoning where it is both by design that you can’t reuse the binary template sensor and you should reuse the binary template sensor…

I have looked into improving the template code itself, basically adding template functions that trigger only if a certain duration has passed. Sadly, because of the same reasons, it is impossible to implement that, since you have no control when the template is evaluated (which probably is a good thing, but I wish they’d make a choice and not leave us in this gray area).

It might help if people ‘revive’ the pull request? The main code has changed in the meantime, but I think it wouldn’t be too hard to adapt my old code. I really think it is a misunderstanding, but I couldn’t get a reaction out of anyone and gave up after a while.

Very clear explanation! Made me rethink about setting up occupancy.
You say you have 15-20 observations in most of your sensors, but how?
For example, my livingroom has motion, door contact, tv and radio. Thats 4 observations.
How do you get so many observations? I’m very curious which sensors you use to get a stable bayes sensor.

You’re right, as far as “added sensors” I probably only have 4 or 5. But I use a lot of time-of-day sensors. In the pre-COVID times my wife and I had regular hours of the day when we left the house for work (maybe that will come back some day). Same with evening and night times when we were asleep. I also integrate my calendar and the workday sensor.

Time of day sensors are SUPER useful for occupancy and sleep sensors. While tod itself doesn’t provide much information they can help distinguish between an asleep house, an empty house, or a house where someone is quietly reading by the window.

1 Like

Can someone tell me the table cell giving 0.33 as shown in the yaml example? The -in my understanding- appropriate cell in the table returns 0.29

Dear # JudoWill thanks to share, but I have a question for your excel file… For the Probalility formula, I think that have an error, the formula say:

=INDEX(L:L,COUNTA(L:L),1)

In my opinion this formula don’t take into account the last row, any value for this row show the same result. I think that the correct formula should be:

=INDEX(L:L,COUNTA(L:L)+1,1)

What do you think?.

Jose

2 Likes

@josemsubcn Yes, you’re right and it has been confirmed in one of the posts above

Am I the only one who gets confused about the differences between the explanation and the table?

e.g. 4 hours of 10 hours are 0.29 not 0.33 (Cell F16).

@uezezs It seems in his example above he put house is occupied at 12 hours and not 14 hours (as in the spreadsheet). So it’s 4/12=~0.33 whereas 4/14=0.29. I’m guessing it’s a typo/mistake in the yaml saying 10 hours (as it’s 10 hours away not home). Example said 12 hours, but spreadsheet has 14 (to make 10+14=24)

I’d go with the spreadsheet as it seems the most accurate (with the fix mentioned above for the Index).

Another thanks for the original post - I’m still don’t really understand this but the spreadsheet was really helpful together with the explanation. Thanks!

I created a few boolean input selectors and made a bayesian sensor based on these. It helped me better understand the explanation above when I could easily toggle the selectors and see how that affected the probability, instead of waiting for real life sensors to trigger, cool down and so forth.

I’m using most of my sensors (motion + a custom for my media players, to see if anything is playing) and two template sensors with delay_off setup like in one of the entries in this thread. Works great so far!

Hi, can you share how you use time of day sensors ?
Thanks a lot for your clear explanations, that helped a lot !

Oh! Time of day sensors would be really helpful here. I would love to see how to get that working with bayes. Anyone have an example?

Also, how can we handle multiple device_trackers, particularly to ignore them if they are not on the same page ?

Example : I’ve got two gps device_trackers but both can occasionnaly be thrown off… I would like the bayesian sensor to make them “cancel each other” when they disagree, and rely on other signals until they agree. Is that doable ?

Edit:
Well, I found a way to do that but the probabilities cannot be defined by hours true/false. Here is a minimum working example :

  • platform: bayesian
    name: home
    prior: 0.8
    probability_threshold: 0.8
    observations:
    • platform: state
      entity_id: device_tracker.gps1
      to_state: “home”
      prob_given_true: 0.6
      prob_given_false: 0.4
    • platform: state
      entity_id: device_tracker.gps1
      to_state: “not_home”
      prob_given_true: 0.4
      prob_given_false: 0.6
    • platform: state
      entity_id: device_tracker.gps2
      to_state: “home”
      prob_given_true: 0.6
      prob_given_false: 0.4
    • platform: state
      entity_id: device_tracker.gps2
      to_state: “not_home”
      prob_given_true: 0.4
      prob_given_false: 0.6
    • platform: state
      entity_id: light.xxx
      to_state: “on”
      prob_given_true: 0.6
      prob_given_false: 0.4
    • platform: state
      entity_id: light.xxx
      to_state: “off”
      prob_given_true: 0.4
      prob_given_false: 0.6

non-contradictory GPS takes priority over light status:
home with light → 0.93 HOME
home without light → 0.86 HOME
not home with light → 0.73 NOT_HOME
not home without light → 0.54 NOT_HOME

light status takes priority over contradictory GPS:
contradictory gps with light → 0.86 HOME
contradictory gps without light → 0.73 NOT_HOME

I’m going to try adding other observations and see if it holds

You sir are a legend and definitely a great professor, no doubt. That was very clear. I was looking for more information on how this is implemented in HA and your write-up was excellent. I am trying to set up a sensor to estimate whether all of the household is in bed or not, could you please check if the below code makes sense for that.

- platform: bayesian
  prior: 0.32
  name: 'Bedtime'
  probability_threshold: 0.9
  observations:
      - entity_id: group.alarmo_device_tracker
        prob_given_true: 0.99  # If I'm in bed then I have to be home - hence 100% probability
        prob_given_false: 0.7 # I could be home and not in bed 70% of the time during any given day
        platform: 'state'
        to_state: 'home'
      - entity_id: group.alarmo_device_tracker
        prob_given_true: 0.009 # If I'm in bed, then I can't be not home
        prob_given_false: 0.5 # I'm not in bed and not home (i.e. at work or school etc.) 
        platform: 'state'
        to_state: 'not_home'        
      - entity_id: 'sensor.sun'
        prob_given_true: 0.97
        prob_given_false: 0.33
        platform: 'state'
        to_state: 'below_horizon'
      - entity_id: 'sensor.sun'
        prob_given_true: 0.03
        prob_given_false: 0.99
        platform: 'state'
        to_state: 'above_horizon'        
      - entity_id: group.interior_lights
        prob_given_true: 0.99 # I never go to bed with lights on
        prob_given_false: 0.8 # Lights off when I'm not in bed, which is true for most of the DAY, but excluding some lights (e.g.  bathroom, garage, pantry) 
        platform: 'state'
        to_state: 'off'
      - entity_id: group.interior_lights
        prob_given_true: 0.001 # reverse of the above
        prob_given_false: 0.2 # reverse of the above
        platform: 'state'
        to_state: 'on'        
      - entity_id: binary_sensor.master_door_contact
        prob_given_true: 0.98
        prob_given_false: 0.3
        platform: 'state'
        to_state: 'off'
      - entity_id: binary_sensor.master_door_contact
        prob_given_true: 0.02
        prob_given_false: 0.7
        platform: 'state'
        to_state: 'on'        
      - entity_id: 'variable.last_motion'
        prob_given_true: 0.95
        prob_given_false: 0.15
        platform: 'state'
        to_state: 'Bedroom Zooz Sensor motion'
      - entity_id: binary_sensor.abhi_pixel_is_charging
        prob_given_true: 0.9
        prob_given_false: 0.4
        platform: 'state'
        to_state: 'on'
      - entity_id: binary_sensor.abhi_pixel_is_charging
        prob_given_true: 0.1
        prob_given_false: 0.6
        platform: 'state'
        to_state: 'off'        
      - entity_id: alarm_control_panel.alarmo
        prob_given_true: 0.99
        prob_given_false: 0.01
        platform: 'state'
        to_state: 'armed_night'
      - entity_id: alarm_control_panel.alarmo
        prob_given_true: 0.01
        prob_given_false: 0.99
        platform: 'state'
        to_state: 'disarmed'  

I have created a pull request to fix this issue: https://github.com/home-assistant/core/pull/67631

@JudoWill if you felt like reviewing my code and/or reasoning that would be amazing and I would be very greatful

1 Like

These should work

    - platform: "template" 
      prob_given_true: 0.95
      prob_given_false: 0.1
      value_template: >-
        {% if is_state('binary_sensor.device_tracker.gps1', 'home')
           and is_state('binary_sensor.device_tracker.gps1', 'home') %}
           true
        {% elif is_state('binary_sensor.device_tracker.gps1', 'not_home')
           and is_state('binary_sensor.device_tracker.gps1', 'not_home') %}
           false
        {% endif %}

And because you have to give the inverse until my PR is merged

    - platform: "template" 
      prob_given_true: 0.9
      prob_given_false: 0.05
      value_template: >-
        {% if is_state('binary_sensor.device_tracker.gps1', 'home')
           and is_state('binary_sensor.device_tracker.gps1', 'home') %}
           false
        {% elif is_state('binary_sensor.device_tracker.gps1', 'not_home')
           and is_state('binary_sensor.device_tracker.gps1', 'not_home') %}
           true
        {% endif %}

If they disagree they will evaluate to null and so should be ignored by Bayesian.

P.S your probabilities look too conservative - so I’ve tweaked them in my example. Assumes they will both accidentally read ‘home’ when you are away 5% of the time and that they will both read “not_home” when you are home 10% of the time, which is probably still too conservative

Why do the inverse situations in prob_given_false not sum to 1?
13.75/14 = 0.982?

Hey guys,

What do we do if the home occupiers don’t really have a routine?
We work from home and the house MIGHT be empty a couple hours a week

Looks good, but mathematically the inverse probabilities should sum to 1.

      - entity_id: group.alarmo_device_tracker
        prob_given_true: 0.99  # If I'm in bed then I have to be home - hence 100% probability
        prob_given_false: 0.7 # I could be home and not in bed 70% of the time during any given day
        platform: 'state'
        to_state: 'home'
      - entity_id: group.alarmo_device_tracker
        prob_given_true: 0.01 # If I'm in bed, then I can't be not home
        prob_given_false: 0.3 # I'm not in bed and not home (i.e. at work or school etc.) 
        platform: 'state'
        to_state: 'not_home' 

I’d love to see some of yours as further examples.

I’m attempting to do room occupancy based on tod, motion, room power usage and other room occupancy and power usage but feel I’m missing the mark in having it only move a few points rather than the current drastic changes I see.

that is not actually knowing nothing :slight_smile: