mmWave Presence Detection - ESPHome style

TL;DR

The DFRobot SEN0395 (Leapmmw HS2xx3A series) is a fabulous sensor for triggering lighting. It is quick, highly configurable and has been extremely reliable over many months.

Intro

Having recently stumbled upon rumors of a potentially upcoming Aqara mmWave presence sensor, it got me thinking of the potential.

One of the most useful projects deployed to date using Home Assistant is motion based lighting for under-counter LED strips in the kitchen. So successful in fact, that the SO more often than not, will cook entire meals without touching the overhead light switch.

This however came with workarounds. PIR sensors do not detect when a person is static for a period of time and every sensor tested to date is fooled. Including Hue, Zooz, Ecolink, Sonoff, Aqara, etc…

My SO loves to practice mime skills and fool the motion sensors; mmwave to the rescue!

With my growing confidence in ESPHome, from the EmporiaVue2 and the NSPanel, it seemed like a natural fit. Acquiring a DFRobot SEN0395 (aka Leapmmw) 24Ghz mmWave sensor, and with some basic assembly of a single GPIO and UART using a logic level shifter/converter we were in business.


Finessing components into a cheap ABS project box is a skill only those without a 3D printer will appreciate :wink: It works just fine with the plastic cover in place (not shown). The greatest challenge is to come.

While this worked out of the box using a single GPIO, I have to confess that it can be challenging to use as a standalone sensor in high motion environments. It is very sensitive. When they say, “mmwave”, they do mean detection by the millimeter! As a bedroom sensor, it is perfect.

In a high-clutter environment I would sometimes encounter situations where the lights might turn on because the detection range of the mmWave includes reflections. So it will bounce around corners and catch you coming!

Install placement and tuning - take 3

With the enablement of the uart_target_output switch I have parsed four (4) of the eight (8) possible targets. These are viewable on the internal webserver since they are very noisy. This opens up new possibilities in both tuning and automations.

** WARNING ** leaving a browser sitting open on the ESP web_server with all 16 target (dist/SNR) updating realtime may cause your ESP to reboot after a few minutes. Please use the web_server for tuning purposes only. Since the target distand/SNR are internal, Home Assistant will be fine.

num_targets however is not internal. You may wish to set it that way or exclude it from logger;
HA configuration.yaml:

logbook:
  exclude:
    entity_globs:
      - sensor.num_targets*

Tuning by SNR

Traditional tuning procedures require moving to a location where the binary_sensor turns off to determine the “end of detection range/angle” for a particular deployment. The problem with this approach is that “edges are fuzzy” and the latency setting impacts whether or not a sensor will report no detection.

By Using the target SNR you can now better understand the impact of different sensor placement/orientation by observing changes to the SNR (<1 is weak detection, the higher the better).

You can now predict when the sensor will report indicate “no presence” by the fact that the last target goes below 0 SNR then disappears. At this point, increasing latency may increase the potential detection range since the target may “flap” and be detected and dropped again within that time period.

Tuning by distance

Since you can now observe target distance, this aids immensely in the configuration of max target distance. Being able to review what the sensor sees will help dispel any assumption of detection range. This is especially true since I made the assumption that a simple measuring tape would suffice in setting distance. When in fact, the sensor doesn’t report the very edge of the target, but sometimes reflections or “deeper” into a target.

It has also transformed certain lighting scenarios where placement was forced to point towards hallways and may trigger on someone who may passersby. You no longer have to measure or guess what the sensor detects and how far it measures. You assumed it was pinpoint accurate didn’t you :wink: It measures in 15cm increments and that means even +/- 1 increment isn’t pinpoint :stuck_out_tongue:

This approach lets me leave the bathroom door open and the light no longer turns on when someone walks by when using the Hue PIR.

A note about distance: do not assume the mmwave signal will “go-through”, “bounce off” or “be stopped by” walls. Construction materials vary. And while the signals may “bounce off” my walls (around corners) the behavior may be different in your setting.

New Automation Opportunities?

By combining certain SNR & distance scenarios it may be possible to include/exclude combinations. Much testing is needed here and feedback welcome. I will be observing if I can differentiate small pets versus humans based on certain distance/SNR ratios.

Installation

Thanks to @hjmcnew for the new installation approach that greatly simplifies deployment.

Please see https://github.com/hjmcnew/esphome-hs2xx3a-custom-component/tree/release for both options.

Install Option 2:

  • Download the leapmmw_sensor.h file into your esphome configuration directory
  • Include the following in the YAML configuration for your ESP board:
substitutions:
  device_name: leapmmw

  # This will vary based on your board
  uart_tx_pin: TX
  
  # This will vary based on your board
  uart_rx_pin: RX
  
  # This will vary based on your board
  gpio_pin: D0
  
  # (Optional) Path to the leapmmw_sensor.h file relative to your esphome configuration directory.
  header_file: leapmmw_sensor.h

packages:
  remote_package:
    url: https://github.com/hjmcnew/esphome-hs2xx3a-custom-component
    ref: release
    file: packages/leapmmw_sensor.yml

Conclusion

I don’t think we are at the end quite yet…

Bored? See my other projects below or wait for the upcoming “mmWave wars!” Hi-Link vs Leapmmw vs SeeedStudio where we attempt to quantify detection ranges for nano-motion (sleep), micro-motion (sitting), and motion (standing/walking). Because I know you all read the max range and think to deploy the sensor 9 meters away for sleep monitoring :open_mouth:

FAQ

  • Will num_target count the number of people in a room? - No! It counts sources of movement at a smaller level. And a person has many potential movement sources (head, torso, hands, feet) that may be counted as individual targets.
  • Will my fan, cat, HVAC blown drapes get detected? - Yes.

Changelog:

[edit] - with help from @phillip1 there is now a greatly simplified tuning process directly from within HA. Thanks Phillip!

[edit] - updated yaml 5/20/22
[edit] - new features! 6/13/2022

  • hidden sensitivity feature unlocked
  • LED control
  • UART output control

[edit] - new target detection features! 7/26/2022

  • add target parsing and target sensors
  • add query buttons (no parsing at this time)

[edit] - lamba code cleanup by @phillip1 7/27/2022

  • I make spaghetti, he brings the meatballs…
  • all eight targets exposed to web_server

[edit] - want to find the limits of an ESP32? We’ve found them. 8/1/2022

  • reduced targets and tracking output interval as >4 overload an ESP32 web_server

[edit] - parameter synchronization code added 8/10/2022

[edit] - a new install process! 8/11/2022

  • a big thank you to @hjmcnew for packaging up the sprawling configuration into a remote_package

My mmWave Projects

Tracking Radar
mmWave Presence Detection
Low-Latency DFRobot+PIR

Where to buy

Look at Mouser / Digikey for local availability.
This seller has been responsive on Ali

41 Likes

Very interesting sensor ! Thanks for sharing.

When you say they’re too sensitive, do you mean they detected humans they were not supposed to detect (behind walls or corners) or did they false trigger a lot without humans around ?

Never without motion.

Minute, miniscule motion.

Do you have a fan on? Trigger.

Does the air movement you produce from walking by make a coat slightly shift in the breeze, trigger. Lol.

The cool part is that since reflections count. You can measure the distance, raytracing style, of a bounce and consider that for your tuning.

Want your kitchen lights to turn on before you enter the kitchen from around a corner from the sensor? Increase the detection distance :wink:

The problem I had with these type of sensors is that they would see through walls too.

1 Like

Interesting. I have not experienced that. Definitely does not see through my walls. Suspect sensor dB influences that?

The DFRobot seems pretty low power.

Yeah I was looking for a high quality replacement of my now-defunct RCWL 0516 radar sensors. Them triggering on non-human movements (like the fan you mentioned) is annoying though.

The lower frequency ones (like the aforementioned RCWL’s) do. The higher frequency ones less so, the RF just reflects.

1 Like

Oh wow, that will raise questions for the Aqara one then, given it is 60GHz.

I was using RCWL-0516.

The sensor is 23€ on Mouser, I think I’ll just order one and do some tests. Gonna be fun, I love those things :smiley:

I was interested in determining the impact of this higher frequency as well.

Datasheets for the suspected sensor suggested the potential for a very high level of customization.

As for what level of customization is exposed to the consumer may make or break the usefulness.

[edit] - I have seen discussion that the upcoming Aqara uses a sensor from these guys. https://www.acconeer.com/ Sadly looking on digikey the devkit is quite pricey.

1 Like

Yeah, the FP1 looks like it’s going to be quite pricey too…
I hope it’s super super accurate!

Lol. It will be super accurate at triggering on the dust particles floating in the air :wink: /s

1 Like

I don’t know about 24Ghz, but 60Ghz is being used to detect breathing and heart rate. Hopefully the fact that Ripple is open sourced ( GitHub - CTA-Ripple/Ripple-v.0: open radar API standard to enable hardware / software interoperability and accelerate the growth of applications of general purpose consumer radar ) means 60GHz kit should start coming down in price.

I don’t think the sensor is all that expensive. You can buy it today.

But as we all know. The software implementation makes or breaks it. And buying it today means you or me bumbling though it :slight_smile:

1 Like

Great project! I wonder if this would work through glass? Might be good to out inside a TV/AV cabinet pointed at the sofa.

@crlogic cool project… Love it.

Just FYI for those looking for RCWL-0516 replacement. I am also looking for RCWL-0516 replacement. Here is what I end up with. It consume about 70uA.

Here is my 2 months experience with it. Surprising, it is good. It can penetrate barrier like glass. But, it is not much. I am doing Arrival sensor and use this sensor. My goal is to detect whether someone enter my car. I could use 4 reed contact sensor and wire it. This module give me very close to what 4 contact sensors would be functioning. I have a video there when I approach my car. The sensor does not really detect me until I am about 1 to 2 inch from the car.

1 Like

I just tested through double pane, low-E, glass and the answer is; no (disclaimer: you got a ten second test with no attempt to improve).

2 Likes

Thanks - interesting given how sensitive it sounds like it is. I’ll probably still pick one up just to experiment with. Thanks for the writeup!

An update.

Since the first deployment worked so well, I assembled a second SEN0395 to test as a bed occupancy sensor while sleeping. I broke out the multimeter this time and noticed that while it is a 5V sensor, the GPIO are 3.3V. Therefore I was able to dispense with the line level shifter and configure it via UART successfully so that it catches the bed without any surrounding area.

After a single nights test, and with no accompanying PIR, it detected contiguous presence for the duration of the night. And the 45 second cooldown configured worked perfectly in the morning as I got up and walked away.

2 Likes

I have been researching into this mmWave technology and its quite impressive. Was about to buy this mmWave SEN0395 device as it looks to be much better than the RCWL-0516 i have.

I was interested in mobile.andrew.jones’s question. I found a paper that says the EU and US regulations are phasing out the 24GHz band in favour of the 60GHz. This started in 2018 and:

As of Jan. 1, 2022, the 24-GHz UWB will no longer be allowed for industrial use in
both Europe and the United States

This seems the reason for the aqara device using 60GHz.I know this si for industrial use, but i can only assume other uses will follow suit. It seems that 60GHz is slightly better, with “16 times the available bandwidth once the regulations take effect”. So that seem good

I also wonder if 24GHz would interfere with other devices going forward. So, i may wait for a 60GHz mmWave device to be released. Seems the first time playing with esphome and sensors i have hit the bleeding edge tech, so will wait and see what comes out :smiley: . Still tempted to get this 24Ghz device anyway to play.

Source:
https://www.ti.com/lit/wp/spry328/spry328.pdf?ts=1643270730752

1 Like