How do I make my sensor know who is home, not just that someone is?

Hardware
1

A couple of months back I posted here about the sensor firing on the Corgi. The thread gave me a lot to try. I aimed the sensor higher — didn’t help, at 2.7 m my short-legged Corgi still walks through the detection cone. A few of you pointed me toward a Bayesian sensor plus Bermuda BLE tracking, and I’ve been wiring those up. It’s genuinely better at telling dog vs. me.

But here’s the wall I hit: even when I get “dog vs. person” right, I still can’t tell which person. The hallway light asks “is someone there” — but what I actually want is “is it me, my partner, or a guest,” because the automation should fire differently for each.

So I’m flipping the question. Instead of fighting the sensor to ignore the dog, what if the system just knew who is moving? Not just presence — identity. If it’s me, what runs? If it’s my partner? If a relative or friend visits?

What actually solves this today? BLE tags kind of count but you’d tag everyone. Cameras + Frigate do faces but that’s a camera in every room. Has anyone here combined room presence with per-person identification in a practical way? What’d you use?

2

I’m not sure it’s possible without extra kit.

I find that a tag on my keyring is very effective at tracking comings and goings, and it’s practical for each member of the household to have a different tag, but most people probably dump their keys somewhere when they’re indoors. Everyone’s in the kitchen!

You might be able to work out some basic logic to eliminate individuals - Dad’s keys are not in the house so it can’t be him - and then use some very complicated Bayesian setup. It’s two o’ clock on a schoolday so it can’t be the kids… But wait! They would be in their rooms. Is the Xbox on? Yup, it’s them. Why aren’t they at school? Send a notification.

Have you tried shouting? Hello! Anybody home? :grin:

3

You can go from this direction but as Red says. It does take extra kit.

Imagine a house owned by a couple that’s a 3*3 grid. (just go with it.). I the house is s main hall and 8 rooms and each room has a door and a motion.

Start by cutting a week into weekdays and weekend and over each of those days cut it into enough time slices to give decent resolution. Then for each day type and day slice you use bayesean math for one bayesean sensor each for a combo of who is expected in the house and each room - so 2 people 9 rooms 18 bayeseans. Each bayesean basically says given all these signals. (motion, TV just turned on, time of day, day part, occupancy signal, the microwave is running etc.) is this person likely to be standing in this room right now with a confidence over X.

Now each of these sensors show the likelihood you’re standing in this room now.

Take the highest of the array for each person.

Now kick back to that motion sensor… When it fires. What’s the bayesean say who did it? When you’re both home it’ll be a little messy but for the most part it’ll resolve more than half of the time and thays before adding BLE.

Add ble into that mix and the accuracy skyrockets because while ble itself is noisy and jumps a lot - the stats smooth everything out. BLE just becomes a very highly weighted signal inthe bayseans array.

If you have math describing the room topology you can also use math (Markov) to weight the adjacent rooms that recently signalled to further filter out impossible travel (you cNt teleport from far bottom left to far top right without travel through connected rooms and physics says thay takes no LESS than x and if you were positive in room z 5 seconds ago these three rooms are still impossible…

(see how this works, you’re stacking a lot of could be and maybe to make damn near impossible to ignore)

Ok you now have a map following you around the house. One more construct.

If you enter a room with one door and the. Close said door. And THEN motion happens… Someone’s in that room permanently until the door opens. This is generally called Wasp-in-a-box. Because your grid. You know the name of the wasp in 60% or better times. You can make it even better still by reverse wasp lockout of any idle room with a door closed. (well he’s not in these take them off the board..)

Stack all these and I’m pretty sure I know where you are on that grid. Template sensor to resolve your person. * object in one of the nine rooms according to that math. I even do a trick where I use the most recent activity and hist stats to weight your preferred rooms at time slice o er the past 10 days…

Complex. Yup.
Takes a lot of setup. No way around it
Accurate. Yup. Very.

This is the basis of how my Room Manager works in ZenOS for my LLM agents.

It works out like this…

1000024785
10000247851080×1313 124 KB

(also she’s right btw. That translated to AI assistant for:‘I read your sensor and it’s reporting a low confidence kitchen score because your phone hasn’t settled ina room but holy moly the kitchen is lit up like a Christmas tree…’)

4

“everyone’s in the kitchen” is way too real. My keys mostly sit on the shoe cabinet by the door, so from the moment I walk in, that tag is basically useless for presence detection.

The “who” question does seem to need something you actually carry or wear. I’ve been wondering lately whether the phone itself could work as an identity sensor.

Thanks for keeping it real — gave me a good laugh.

5

Thanks for taking the time to write all this out — that’s a seriously detailed walkthrough and I appreciate it.

6

It’s possible to track phones around the house, via BLE (so not every phone) and then with phones that would rotate random mac address for bluetooth by default, there will be special setups.

And then, you would put a bunch of ESP devices around your house, which you likely have done already via Bermuda.

You can consult the wiki section of the Bermuda project.

And then BLE tracking still cannot handle guests.

So if you really want to do it you would need camera everywhere to recognize the faces…

7

Nah guest is just motion with no confirmed ble when you have hi quality ble signaling… It can be done. You’d be super surprised what a bayesean can actually do. Camera not required.

8

im not a fan of cameras around the inside of the house but wouldnt facial recognition work best along with frigate

9

They’d work but you said it yourself. Camera in the living spa e isn’t ideal. (I personally won’t have them) ble is much easier imho and a hoarde of esp32 ble proxies all cost less than one camera and don’t need yet another server to run.

Even without bayes filters I’d rather be BLE only

10

You already have the signals: Bermuda RSSI per room, which phones are on wifi, and who is normally in at that hour. The missing piece is the thing that turns those into a name. A Bayesian sensor does that, but you tune priors per person to get there.

I put together an integration for a classifier that takes the same signals and gives back one of your own options plus a confidence, so you can refuse to act when it is guessing:

- action: jev.choice
  response_variable: who
  data:
    state: |
      Bermuda nearest room: A={{ states('sensor.bermuda_a_area') }} B={{ states('sensor.bermuda_b_area') }}
      Phones on wifi: A={{ states('device_tracker.phone_a') }} B={{ states('device_tracker.phone_b') }}
      mmWave living room: {{ states('binary_sensor.mmwave_living') }}
      Now: {{ now().strftime('%H:%M %A') }}
    instructions: Which person is the living room picking up?
    options: [person_a, person_b, guest, nobody]
    option_descriptions:
      person_a: Their phone or BLE tag is in range, or this is their usual hour
      person_b: Their phone or BLE tag is in range
      guest: A person is present but no known device is in range
      nobody: The mmWave is seeing the dog, or nothing at all
- condition: template
  value_template: "{{ who.confidence > 0.8 }}"

It will not invent identity. With no phone and no tag in range it answers guest at a low confidence, which is the correct answer. The gate above is what keeps that out of your automation.

Repo is HA-Jev if it is useful.