I’ve seen dozens of Bayes Binary Sensor explanations. As both a programmer and a statistics professor, they all give me the shivers. There are explanations out these that are wrong both from a math & statistics perspective and from a programming & implementation perspective.
So, here is the definitive explanation. Grab some coffee, and lets get down to brass-stats.
Weaved within the post is how I think about setting the values for these sensors (from a real stats perspective). Knowing how the numbers are actually being used under-the-hood should help with setting them.
(for the impatient): HASS Bayes Sensors FOR REALSIES - Google Sheets
As a concrete example we’ll use a bayes-sensor determine whether my house is occupied. We’ll say the occupied state is TRUE.
Bayes theorem describes a way to update our knowledge of a probability given new information. We always start a bayesian analysis by determining a prior. This is our estimate of the probability before learning about any event. From our example, this is interpreted as “Knowing NOTHING, what do I think the probability of the house being occupied?”. I can estimate this by saying: “I work ~8 hours 5 days a week, have other ~2 hours of non-house activities per day”. So, I estimate that, knowing nothing, my house is empty ~10 hours per day, which means it is occupied 14 hours per day. So, I’d set my prior at ~14/24 ~= 0.58.
Now, for information. Imagine my TV turns on. That should inform my understanding of whether the house is empty. Bayes Rule tells me how to update my prior probability given this new information.
The raw Bayes Rule:
p( house_occupied | TV_on ) = P(house_occupied) * p (TV_on | house_occupied) / p(TV_on)
I would read this equation as: “The probability the house is occupied given the TV is on is equal to the probability the house is occupied (before knowledge) times the probability the TV is on given the house is occupied divided by the probability the TV is on.”
That was a big word-salad but we can break it down.
- p( house_occupied | TV_on ) : Probability that the house is empty now that we know the TV is on. This is called the “posterior probability” (probability post-knowledge).
- p(house_occupied) : Probability the house is occupied before we knew the TV was on (our
prior) - p(TV_on | house_occupied) : Probability the TV is on when the house is occupied. This is the
prob_given_truefrom the config files. This is materially different from “the probability the house is occupied when the TV is on” (how I commonly see it explained). - p(TV_on) : Probability the TV is on (on the whole throughout the day).
The ratio of the last two items are often called the predicate. They represent how “important” this new information is.
Imagine the next two scenarios:
My TV is on only when I’m home: p(TV_on | house_occupied) will be LARGER than p(TV_on).
- If I’m home 12 hours a day, the TV is on for 4 of that. p(TV_on | house_occupied) will be ~4/12
- But p(TV_on) will be 4 hours out of 24 per day. ~= 4/24.
- This means the division of the two will be greater than one. p(TV_on | house_occupied) ~= 4/12 ~= 8/24 while p(TV_on) ~= 4/24. So, my
predicateis ~= 2.0. - And when I multiply it by the
prior, the probability of p( house_occupied | TV_on ) will INCREASE.
I leave my TV on for my dog when I’m away: p(TV_on | house_occupied) will be SMALLER than p(TV_on)
- p(TV_on) ~= 10 (time away, TV on for the dog) + 4 (time I’m home watching TV) / 24 ~= 14/24
- p(TV_on | house_occupied) will still be ~4/12.
- Now, the division of the two is LESS than one. 8/24 / 14/24 ~= 0.57
- This means that when I multiply the
predicateby theprior, the probability will DECREASE.
This makes intuitive sense. If my TV is only on when I’m home, knowing the TV is on increases the probability the house is occupied. If I leave the TV on for my dog, then the TV is actually on more when I’m NOT home. Meaning the TV being on implies that the house is actually empty (except for the dog).
Most of the explanations I’ve seen around are good up to this point. But, they all get the next part wrong. What about the prob_given_false in the docs? Which we haven’t used yet.
The explanation I see floating around is prob_given_false is used when (in our example) the TV is off. This is WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG WRONG from both a statistics and code implementation perspective.
Here’s the leap we need to make that I see everyone making a mistake on. How do we know the probability my TV is on throughout the whole day (the denominator of the Bayes rule)? Well I can re-think about the probability the TV is on as ”The probability the TV is on while I’m home times the probability I’m home plus the probability the TV is on while I’m not home times the probability I’m not home”. In math:
p(TV_on) = p(TV_on | house_occupied)*p(house_occupied) + p(TV_on | house_not_occupied)*p(house_not_occupied)
- p(house_occupied) is the prior
- p(house_not_occupied) is 1-prior
- p(TV_on | house_occupied) is the
prob_given_truefrom the configs - p(TV_on | house_not_occupied) is the
prob_given_falsefrom the configs.
So, we can rewrite the equation as (often split into separate numerator and denominator eqns):
numerator = p(TV_on | house_occupied) * p(house_occupied)
denominator = p(TV_on | house_occupied) * p(house_occupied) + p(TV_on | house_not_occupied) * (1 - p(house_occupied))
probability = numerator / denominator
If we look at the code in HA core/homeassistant/components/bayesian/binary_sensor.py at 83a709b768b88389f0077ca22aa7a445c5babaac · home-assistant/core · GitHub we can see that is exactly how it is implemented.
def update_probability(prior, prob_true, prob_false):
"""Update probability using Bayes' rule."""
numerator = prob_true * prior
denominator = numerator + prob_false * (1 - prior)
probability = numerator / denominator
return probability
And the relevant part of def async_threshold_sensor_state_listener
prior = self.prior
for obs in self.current_obs.values():
prior = update_probability(prior, obs["prob_true"], obs["prob_false"])
self.probability = prior
From this code we can see that EVERY update of an observation includes both a prob_given_true and a prob_given_false.
This is also stated in the docs (Bayesian - Home Assistant):
prob_given_true
(float)(Required)
The probability of the observation occurring, given the event is true.prob_given_false
(float)(Optional)
The probability of the observation occurring, given the event is false can be set as well.
These are the probabilities of the events (TV On) happening GIVEN the true/false states (house occupied). NOT, the probability of the state (house-occupied) given the event (TV on/off). THESE ARE DIFFERENT PROBABILITIES AND ARE NOT INTERCHANGEABLE.
So, I would set my sensor up like this:
prior: 0.58 # home roughly 14/24 hours per day
observations:
- platform: state
entity_id: sensor.tv
prob_given_true: 0.33 #4 hours of TV per the 10 hours I’m home
prob_given_false: 0.017 #15 minutes of TV per 14 hours away, I don’t like zeros here
to_state: on
This means that our sensor will have a new “observation” when the TV is on.
This sensor will give a value of 0.96 when the TV is on.
numerator = p(TV_on | house_occupied) * p(house_occupied)
numerator = 0.33* 0.58 = 0.1914
denominator = p(TV_on | house_occupied) * p(house_occupied) + p(TV_on | house_not_occupied) * (1 - p(house_occupied))
denominator = 0.33* 0.58 +0.017 * (1 - 0.58) = 0.198
probability = numerator / denominator
probability = 0.1914 / 0.198 = 0.96
But, what happens when the TV is off. Well, looking at the code:
def _process_state(self, entity_observation):
"""Add entity to current observations if state conditions are met."""
entity = entity_observation["entity_id"]
should_trigger = condition.state(
self.hass, entity, entity_observation.get("to_state")
)
self._update_current_obs(entity_observation, should_trigger)
def _update_current_obs(self, entity_observation, should_trigger):
"""Update current observation."""
obs_id = entity_observation["id"]
if should_trigger:
prob_true = entity_observation["prob_given_true"]
prob_false = entity_observation.get("prob_given_false", 1 - prob_true)
self.current_obs[obs_id] = {
"prob_true": prob_true,
"prob_false": prob_false,
}
else:
self.current_obs.pop(obs_id, None)
The code only triggers (and adds to the “current_obs”) when the state is in the to_state. When the TV leaves the on state, it will fall out of the current_obs list (that’s the else in the last two lines). This leads to an important consequence, the not “to_state” observations are not seen by the bayesian sensor.
When the TV is off, it will give the prior (because there are no “observations”). So, its value will be 0.41.
If we want the TV being OFF to indicate we are likely to be away, we need to adjust the sensor.
prior: 0.58 # home roughly 14/24 hours per day
observations:
- platform: state
entity_id: sensor.tv
prob_given_true: 0.33 #4 hours of TV per 10 hours I’m home
prob_given_false: 0.017 #15 minutes of TV per 14 hours away, I don’t like zeros here
to_state: on
- platform: state
entity_id: sensor.tv
prob_given_true: 0.66 # 8/12 Hours off while home
prob_given_false: 0.946 # 13.75/14 hours off while I’m away
to_state: off # this is the “TRUE” state now
Now, when the TV is on. It will have p=0.96
When the TV is off. It will have p=0.49
This makes intuitive sense. The TV being on is “rare” (roughly 4 hours per 24) and only happens when I’m home, so the change from the prior is large, 1.66 times. However, the TV is often off (I gotta sleep sometime), it being off doesn’t down-shift the probability as much, only changed by ~9%.
How do we integrate multiple pieces of information. This is actually easy.
We start with the original prior. Then we get new information. We do the Bayes Rule for the new info. The posterior becomes our “new prior”. When we get more new information, we use the “new prior” in our next calculation. That’s what is expressed in the for-loop from async_threshold_sensor_state_listener.
I’ve put these into a Google Spreadsheet that anyone can use. Just duplicate your own from mine. HASS Bayes Sensors FOR REALSIES - Google Sheets
You can use the drag-fill to include more sensors. The “predicate” field indicates how much the sensor will update when the observation is True. The “posterior” column indicates the probability after each observation, if the observation is in the FALSE state, it is not included in the observation list (like it is in HASS).
I hope this helps people understand what is going on under the hood of the Bayes Sensor platform.
