mmWave Presence Detection - ESPHome style

Does it mean you can identify different areas of movement?

Hi @crlogic

Can you explain a bit what you are trying to do and how you get those values ?

if target = person then I would say match with number of person of the house.

SNR means ?

Yes you can differentiate by distance.

1 Like

That’s the plan. Need to finish testing and use-case writeup.

Negative. Target != person. Target = detectable motion above noise threshold. Aka, wave two hands in different areas == two targets.

Signal to noise ratio. Aka, how “strong” the detection is.

if target is detection motion above a threshold (I just remove noise) but it’s also without motion , I mean staying static ? The noise is to differentiate a static vs motion detection ?

Targets are not link to the number of those antenna patches ? Because patch on top will trigger another area then patch below ?
image

People still “move a little while static” that’s how this works :wink:

[edit] - I think people inherently associate “movement” with “getting up and walking”. But even “standing/sitting still” (aka, static in relation to walking), you sway a little, you breathe, you twitch…

Perhaps a categorization of, “levels of movement” would be helpful. Perhaps;

  • motion = walking
  • micro-motion = standing in place or sitting
  • nano-motion = laying down / sleeping (aka just your breathing)

What? Antenna patches != targets. The firmware supports the identification of up to eight targets.

I have a 100x100deg module coming and it only has two patches. It has the same firmware options.

2 Likes

Pre-sure micro motion is generally used by companies to refer to sleeping, standing, sitting, all of those you’re more or less just breathing.

Darn companies :stuck_out_tongue_winking_eye:

I’ll need to differentiate in the upcoming mmWave sensor showdown!

I am still experimenting with SNR, but one possible use-case is “a close target w/ weak SNR could indicate a cat” type of idea…

1 Like

OP updated including cleaner lambda code from @phillip1

1 Like

@crlogic Would you mind double-checking your posted ESPHome code? I’m compiling it for my D1 Mini but the text sensor part is throwing the following errors:

'text_sensors' is a required option for [text_sensor.custom]

and on the next line:

lambda doesn't contain a return statement, but the lambda is expected to return a value. Please make sure the lambda contains at least one return statement

It would also be worth highlighting (for newbies like me!) the new requirement to include the UART text sensor in the esphome folder as per Custom UART Text Sensor — ESPHome

Good catch. So many changes lately my eye get crossed!

I’ll double check the whole thing later, just added the following;

text_sensor:
  - platform: custom
    lambda: |-
      auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
      App.register_component(my_custom_sensor);
      return {my_custom_sensor};
    text_sensors:
      id: uart_readline
      internal: true
      on_value:
        lambda: |-
          std::string line = id(uart_readline).state;
          if (id(uart_target_output_$shortname).state && line.substr(0, 6) == "$JYRPO")
          {    
              
              auto publishTarget = [](std::string idx, float dist, float snr) {
                  auto sens = App.get_sensors();
                  for(int i = 0; i < sens.size(); i++) {
                    auto name = sens[i]->get_name();
                    auto target = "target_" + to_string(idx);
                    if(name.size() > 10 && name.substr(0, 8) == target) {
                      if(name.substr(9, 3) == "dis") {
                        sens[i]->publish_state(dist);
                      } else if(name.substr(9, 3) == "SNR") {
                        sens[i]->publish_state(snr);
                      }
                    }
                  }
              };
              
              line = line.substr(6);
              std::vector<std::string> v;    
              for(int i = 0; i < line.length(); i++) {
                  if(line[i] == ',') {
                      v.push_back("");
                  } else {
                      v.back() += line[i];
                  }
              }
          
              id(num_targets).publish_state(parse_number<float>(v[0]).value());
              
              publishTarget(v[1], parse_number<float>(v[2]).value(), parse_number<float>(v[4]).value());
              for(int i = parse_number<int>(v[0]).value() +1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
          }
1 Like

Thanks for the fast response! It will now start compiling but then aborts with error 1 after this:

/config/esphome/occupancy.yaml:391:14: error: 'uart_target_output_$shortname' was not declared in this scope
  391 |           if (id(uart_target_output_$shortname).state && line.substr(0, 6) == "$JYRPO")```

Yeah my bad; distracted. Remove this. I have three versions going and watching qualifying!!

1 Like

[A call for testers]

Need some help, the next update is a stretch project for me. I am looking for a small number of testers who are comfortable making breaking changes, potential crashes, and can perform direct serial re-flash if needed.

I have started bumbling my way through a Custom Sensor Component. The goals include;

  • led/distance/latency/sensitivity state queries from the MCU function as expected
    ** these occur on_boot, on_value, and when factory_reset_sensor_config are used

Other changes include:

  • reduced unneeded switches
  • target tracking is now default behavior

Caveats of my first custom component:

  • hard-coded entity names in the .h means you cannot change any id

1 Like

Hey, just thought I’d share my setup:


Quite pleased with the way it’s turned out, my own version of an FP1!

Like others I used a D1 mini as my base. I also included a BH1750 lux sensor so I can automate my lights based on ambient light levels - the body lets in enough light for it to take useful measurements.

One slight issue is that it currently will pick up movement in the hallway through the wall you can see to the right of the device in the picture. Dialling down sensitivity unfortunately makes it less useful at picking up distant motion in the room itself. Any ideas on how to prevent it from looking into the hallway?

3 Likes

@crlogic - I have made some tweaks to your code to better utilise substitutions so that if someone wants to set up more than one of these then all the buttons/sensors/switches are easier to differentiate in HA. Also some minor tweaks to units of measurement:


esphome:
  name: living-room-presence
  includes:
    - uart_read_line_sensor.h

esp8266:
  board: d1_mini #change as needed

# Enable logging
logger:
  logs:
    text_sensor: INFO
    sensor: INFO

substitutions:
  device_name: living_room_presence #change as needed
  device_friendly_name: Living Room Presence #change as needed

# Enable Home Assistant API
api:
  services:
      # Service to send a command directly to the display. Useful for testing
    - service: send_command
      variables:
        cmd: string
      then:
        - uart.write: !lambda
            std::string command = to_string(cmd) +"\r";
            return std::vector<uint8_t>(command.begin(), command.end());

ota:
  password: #snip

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Living Room Presence Hotspot"
    password: #snip

captive_portal:

uart:
  id: uart_bus
  tx_pin: D2
  rx_pin: D1
  baud_rate: 115200

web_server:
  port: 80
  version: 2
  include_internal: true
  ota: false

http_request:
  useragent: esphome/$device_name
  timeout: 2s

binary_sensor:
  - platform: gpio
    name: $device_friendly_name
    id: $device_name
    pin:
      number: D0 #change as needed
      mode: INPUT_PULLDOWN
    on_state:
     - if:
        condition:
          - binary_sensor.is_off: $device_name
        then:
          - sensor.template.publish:
              id: target_1_distance_m
              state: 0.0
          - sensor.template.publish:
              id: target_1_SNR
              state: 0.0
          - sensor.template.publish:
              id: ${device_name}_num_targets
              state: 0.0

sensor:
  - platform: template
    name: ${device_friendly_name} Target 1 Distance
    id: target_1_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 2 Distance
    id: target_2_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 3 Distance
    id: target_3_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 4 Distance
    id: target_4_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 5 Distance
    id: target_5_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 6 Distance
    id: target_6_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 7 Distance
    id: target_7_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 8 Distance
    id: target_8_distance_m
    unit_of_measurement: m
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 1 SNR
    id: target_1_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 2 SNR
    id: target_2_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 3 SNR
    id: target_3_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 4 SNR
    id: target_4_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 5 SNR
    id: target_5_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 6 SNR
    id: target_6_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 7 SNR
    id: target_7_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 8 SNR
    id: target_8_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Targets
    id: ${device_name}_num_targets

  - platform: bh1750 #ignore if not using a bh1750 lux sensor
    name: ${device_friendly_name} Lux
    address: 0x23
    update_interval: 60s

switch:
  - platform: safe_mode
    name: ${device_friendly_name} Safe Mode
    entity_category: diagnostic 

  - platform: template
    name: ${device_friendly_name} mmWave Sensor
    id: ${device_name}_mmwave_sensor
    entity_category: config
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - uart.write: "sensorStart"
    turn_off_action:
      - uart.write: "sensorStop"
    
  - platform: template
    name: ${device_friendly_name} LED
    id: ${device_name}_led
    entity_category: config
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 0"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 1"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} UART Presence Output
    id: ${device_name}_uart_presence_output
    entity_category: diagnostic 
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 1 1"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 1 0"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} UART Target Output
    id: ${device_name}_uart_target_output
    entity_category: diagnostic 
    optimistic: true
    restore_state: true
    assumed_state: false
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 2 1 1 1"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 2 0"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s   

number:
  - platform: template
    name: ${device_friendly_name} Range
    id: ${device_name}_range
    entity_category: config
    min_value: 0.15
    max_value: 9.45
    initial_value: 3.15
    optimistic: true
    step: 0.15
    restore_value: true
    unit_of_measurement: m
    mode: box
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string range = "setRange 0 " + str_sprintf("%.2f", x);
                      return std::vector<unsigned char>(range.begin(), range.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} Latency
    id: ${device_name}_latency
    entity_category: config
    min_value: 1
    max_value: 600
    initial_value: 90
    optimistic: true
    step: 1
    restore_value: true
    unit_of_measurement: s
    mode: box
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string setL = "setLatency 0.1 " + str_sprintf("%.0f", x);
                      return std::vector<unsigned char>(setL.begin(), setL.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor

  - platform: template
    name: ${device_friendly_name} Sensitivity
    id: ${device_name}_sensitivity
    entity_category: config
    min_value: 0
    max_value: 9
    initial_value: 9
    optimistic: true
    step: 1
    restore_value: true
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string mss = "setSensitivity " + to_string((int)x);
                      return std::vector<unsigned char>(mss.begin(), mss.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor

button:
  - platform: restart
    name: ${device_friendly_name} Restart
    entity_category: diagnostic
  
  - platform: template
    name: ${device_friendly_name} Update Sensitivity
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getS = "getSensitivity";
                      return std::vector<unsigned char>(getS.begin(), getS.end());

  - platform: template
    name: ${device_friendly_name} Update Latency
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getL = "getLatency";
                      return std::vector<unsigned char>(getL.begin(), getL.end());

  - platform: template
    name: ${device_friendly_name} Update Range
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getR = "getRange";
                      return std::vector<unsigned char>(getR.begin(), getR.end());

  - platform: template
    name: ${device_friendly_name} Factory Reset
    id: ${device_name}_factory_reset
    entity_category: config
    on_press:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "resetCfg"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor

text_sensor:
  - platform: custom
    lambda: |-
      auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
      App.register_component(my_custom_sensor);
      return {my_custom_sensor};
    text_sensors:
      name: ${device_friendly_name} UART Output
      id: uart_readline
      internal: true
      on_value:
        lambda: |-
          std::string line = id(uart_readline).state;
          if (id(${device_name}_uart_target_output).state && line.substr(0, 6) == "$JYRPO")
          {    
              
              auto publishTarget = [](std::string idx, float dist, float snr) {
                  auto sens = App.get_sensors();
                  for(int i = 0; i < sens.size(); i++) {
                    auto name = sens[i]->get_name();
                    auto target = "target_" + to_string(idx);
                    if(name.size() > 10 && name.substr(0, 8) == target) {
                      if(name.substr(9, 3) == "dis") {
                        sens[i]->publish_state(dist);
                      } else if(name.substr(9, 3) == "SNR") {
                        sens[i]->publish_state(snr);
                      }
                    }
                  }
              };
              
              line = line.substr(6);
              std::vector<std::string> v;    
              for(int i = 0; i < line.length(); i++) {
                  if(line[i] == ',') {
                      v.push_back("");
                  } else {
                      v.back() += line[i];
                  }
              }
          
              id(${device_name}_num_targets).publish_state(parse_number<float>(v[0]).value());
              
              publishTarget(v[1], parse_number<float>(v[2]).value(), parse_number<float>(v[4]).value());
              for(int i = parse_number<int>(v[0]).value() +1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
          }

i2c: #ignore if not using an i2c bus (in my case for the bh1750)
  sda: D6
  scl: D7
  scan: true
  id: bus_a
3 Likes
  • target tracking is now default behavior

Have you noticed device crashing when target tracking is on? I need to test further but seem to be finding that my setup went through a number of unexpected reboots when both target tracking and UART output were on whilst I was debugging range and sensitivity

Sensitivity != distance :wink:

I have recently updated the OP with better tuning descriptions.

That looks very nice! Well done.

I would caution against changing id: as it will be more important in upcoming revisions. name: is fine and can be changed to taste. I will be sure to better highlight this in the future.