mmWave Presence Detection - ESPHome style

crlogic · July 30, 2022, 2:48pm

Good catch. So many changes lately my eye get crossed!

I’ll double check the whole thing later, just added the following;

text_sensor:
  - platform: custom
    lambda: |-
      auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
      App.register_component(my_custom_sensor);
      return {my_custom_sensor};
    text_sensors:
      id: uart_readline
      internal: true
      on_value:
        lambda: |-
          std::string line = id(uart_readline).state;
          if (id(uart_target_output_$shortname).state && line.substr(0, 6) == "$JYRPO")
          {    
              
              auto publishTarget = [](std::string idx, float dist, float snr) {
                  auto sens = App.get_sensors();
                  for(int i = 0; i < sens.size(); i++) {
                    auto name = sens[i]->get_name();
                    auto target = "target_" + to_string(idx);
                    if(name.size() > 10 && name.substr(0, 8) == target) {
                      if(name.substr(9, 3) == "dis") {
                        sens[i]->publish_state(dist);
                      } else if(name.substr(9, 3) == "SNR") {
                        sens[i]->publish_state(snr);
                      }
                    }
                  }
              };
              
              line = line.substr(6);
              std::vector<std::string> v;    
              for(int i = 0; i < line.length(); i++) {
                  if(line[i] == ',') {
                      v.push_back("");
                  } else {
                      v.back() += line[i];
                  }
              }
          
              id(num_targets).publish_state(parse_number<float>(v[0]).value());
              
              publishTarget(v[1], parse_number<float>(v[2]).value(), parse_number<float>(v[4]).value());
              for(int i = parse_number<int>(v[0]).value() +1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
          }

ajoyce · July 30, 2022, 2:56pm

Thanks for the fast response! It will now start compiling but then aborts with error 1 after this:

/config/esphome/occupancy.yaml:391:14: error: 'uart_target_output_$shortname' was not declared in this scope
  391 |           if (id(uart_target_output_$shortname).state && line.substr(0, 6) == "$JYRPO")```

crlogic · July 30, 2022, 3:01pm

Yeah my bad; distracted. Remove this. I have three versions going and watching qualifying!!

crlogic · July 30, 2022, 10:04pm

[A call for testers]

Need some help, the next update is a stretch project for me. I am looking for a small number of testers who are comfortable making breaking changes, potential crashes, and can perform direct serial re-flash if needed.

I have started bumbling my way through a Custom Sensor Component. The goals include;

led/distance/latency/sensitivity state queries from the MCU function as expected
** these occur on_boot, on_value, and when factory_reset_sensor_config are used

Other changes include:

reduced unneeded switches
target tracking is now default behavior

Caveats of my first custom component:

hard-coded entity names in the .h means you cannot change any id

ajoyce · July 31, 2022, 12:07pm

Hey, just thought I’d share my setup:

Quite pleased with the way it’s turned out, my own version of an FP1!

Like others I used a D1 mini as my base. I also included a BH1750 lux sensor so I can automate my lights based on ambient light levels - the body lets in enough light for it to take useful measurements.

One slight issue is that it currently will pick up movement in the hallway through the wall you can see to the right of the device in the picture. Dialling down sensitivity unfortunately makes it less useful at picking up distant motion in the room itself. Any ideas on how to prevent it from looking into the hallway?

ajoyce · July 31, 2022, 12:15pm

@crlogic - I have made some tweaks to your code to better utilise substitutions so that if someone wants to set up more than one of these then all the buttons/sensors/switches are easier to differentiate in HA. Also some minor tweaks to units of measurement:

esphome:
  name: living-room-presence
  includes:
    - uart_read_line_sensor.h

esp8266:
  board: d1_mini #change as needed

# Enable logging
logger:
  logs:
    text_sensor: INFO
    sensor: INFO

substitutions:
  device_name: living_room_presence #change as needed
  device_friendly_name: Living Room Presence #change as needed

# Enable Home Assistant API
api:
  services:
      # Service to send a command directly to the display. Useful for testing
    - service: send_command
      variables:
        cmd: string
      then:
        - uart.write: !lambda
            std::string command = to_string(cmd) +"\r";
            return std::vector<uint8_t>(command.begin(), command.end());

ota:
  password: #snip

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Living Room Presence Hotspot"
    password: #snip

captive_portal:

uart:
  id: uart_bus
  tx_pin: D2
  rx_pin: D1
  baud_rate: 115200

web_server:
  port: 80
  version: 2
  include_internal: true
  ota: false

http_request:
  useragent: esphome/$device_name
  timeout: 2s

binary_sensor:
  - platform: gpio
    name: $device_friendly_name
    id: $device_name
    pin:
      number: D0 #change as needed
      mode: INPUT_PULLDOWN
    on_state:
     - if:
        condition:
          - binary_sensor.is_off: $device_name
        then:
          - sensor.template.publish:
              id: target_1_distance_m
              state: 0.0
          - sensor.template.publish:
              id: target_1_SNR
              state: 0.0
          - sensor.template.publish:
              id: ${device_name}_num_targets
              state: 0.0

sensor:
  - platform: template
    name: ${device_friendly_name} Target 1 Distance
    id: target_1_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 2 Distance
    id: target_2_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 3 Distance
    id: target_3_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 4 Distance
    id: target_4_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 5 Distance
    id: target_5_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 6 Distance
    id: target_6_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 7 Distance
    id: target_7_distance_m
    unit_of_measurement: m
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 8 Distance
    id: target_8_distance_m
    unit_of_measurement: m
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 1 SNR
    id: target_1_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 2 SNR
    id: target_2_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 3 SNR
    id: target_3_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 4 SNR
    id: target_4_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 5 SNR
    id: target_5_SNR
    internal: true

  - platform: template
    name: ${device_friendly_name} Target 6 SNR
    id: target_6_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 7 SNR
    id: target_7_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Target 8 SNR
    id: target_8_SNR
    internal: true
    
  - platform: template
    name: ${device_friendly_name} Targets
    id: ${device_name}_num_targets

  - platform: bh1750 #ignore if not using a bh1750 lux sensor
    name: ${device_friendly_name} Lux
    address: 0x23
    update_interval: 60s

switch:
  - platform: safe_mode
    name: ${device_friendly_name} Safe Mode
    entity_category: diagnostic 

  - platform: template
    name: ${device_friendly_name} mmWave Sensor
    id: ${device_name}_mmwave_sensor
    entity_category: config
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - uart.write: "sensorStart"
    turn_off_action:
      - uart.write: "sensorStop"
    
  - platform: template
    name: ${device_friendly_name} LED
    id: ${device_name}_led
    entity_category: config
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 0"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 1"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} UART Presence Output
    id: ${device_name}_uart_presence_output
    entity_category: diagnostic 
    optimistic: true
    restore_state: true
    assumed_state: true
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 1 1"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 1 0"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} UART Target Output
    id: ${device_name}_uart_target_output
    entity_category: diagnostic 
    optimistic: true
    restore_state: true
    assumed_state: false
    turn_on_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 2 1 1 1"
      - delay: 1s 
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
    turn_off_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "setUartOutput 2 0"
      - delay: 1s      
      - uart.write: "saveConfig"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s   

number:
  - platform: template
    name: ${device_friendly_name} Range
    id: ${device_name}_range
    entity_category: config
    min_value: 0.15
    max_value: 9.45
    initial_value: 3.15
    optimistic: true
    step: 0.15
    restore_value: true
    unit_of_measurement: m
    mode: box
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string range = "setRange 0 " + str_sprintf("%.2f", x);
                      return std::vector<unsigned char>(range.begin(), range.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor
      - delay: 1s
      
  - platform: template
    name: ${device_friendly_name} Latency
    id: ${device_name}_latency
    entity_category: config
    min_value: 1
    max_value: 600
    initial_value: 90
    optimistic: true
    step: 1
    restore_value: true
    unit_of_measurement: s
    mode: box
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string setL = "setLatency 0.1 " + str_sprintf("%.0f", x);
                      return std::vector<unsigned char>(setL.begin(), setL.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor

  - platform: template
    name: ${device_friendly_name} Sensitivity
    id: ${device_name}_sensitivity
    entity_category: config
    min_value: 0
    max_value: 9
    initial_value: 9
    optimistic: true
    step: 1
    restore_value: true
    set_action:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
                      std::string mss = "setSensitivity " + to_string((int)x);
                      return std::vector<unsigned char>(mss.begin(), mss.end());
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 1s
      - switch.turn_on: ${device_name}_mmwave_sensor

button:
  - platform: restart
    name: ${device_friendly_name} Restart
    entity_category: diagnostic
  
  - platform: template
    name: ${device_friendly_name} Update Sensitivity
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getS = "getSensitivity";
                      return std::vector<unsigned char>(getS.begin(), getS.end());

  - platform: template
    name: ${device_friendly_name} Update Latency
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getL = "getLatency";
                      return std::vector<unsigned char>(getL.begin(), getL.end());

  - platform: template
    name: ${device_friendly_name} Update Range
    entity_category: diagnostic
    internal: true
    on_press:
    - uart.write: !lambda
                      std::string getR = "getRange";
                      return std::vector<unsigned char>(getR.begin(), getR.end());

  - platform: template
    name: ${device_friendly_name} Factory Reset
    id: ${device_name}_factory_reset
    entity_category: config
    on_press:
      - switch.turn_off: ${device_name}_mmwave_sensor
      - delay: 1s
      - uart.write: "resetCfg"
      - delay: 3s
      - switch.turn_on: ${device_name}_mmwave_sensor

text_sensor:
  - platform: custom
    lambda: |-
      auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
      App.register_component(my_custom_sensor);
      return {my_custom_sensor};
    text_sensors:
      name: ${device_friendly_name} UART Output
      id: uart_readline
      internal: true
      on_value:
        lambda: |-
          std::string line = id(uart_readline).state;
          if (id(${device_name}_uart_target_output).state && line.substr(0, 6) == "$JYRPO")
          {    
              
              auto publishTarget = [](std::string idx, float dist, float snr) {
                  auto sens = App.get_sensors();
                  for(int i = 0; i < sens.size(); i++) {
                    auto name = sens[i]->get_name();
                    auto target = "target_" + to_string(idx);
                    if(name.size() > 10 && name.substr(0, 8) == target) {
                      if(name.substr(9, 3) == "dis") {
                        sens[i]->publish_state(dist);
                      } else if(name.substr(9, 3) == "SNR") {
                        sens[i]->publish_state(snr);
                      }
                    }
                  }
              };
              
              line = line.substr(6);
              std::vector<std::string> v;    
              for(int i = 0; i < line.length(); i++) {
                  if(line[i] == ',') {
                      v.push_back("");
                  } else {
                      v.back() += line[i];
                  }
              }
          
              id(${device_name}_num_targets).publish_state(parse_number<float>(v[0]).value());
              
              publishTarget(v[1], parse_number<float>(v[2]).value(), parse_number<float>(v[4]).value());
              for(int i = parse_number<int>(v[0]).value() +1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
          }

i2c: #ignore if not using an i2c bus (in my case for the bh1750)
  sda: D6
  scl: D7
  scan: true
  id: bus_a

ajoyce · July 31, 2022, 12:17pm

target tracking is now default behavior

Have you noticed device crashing when target tracking is on? I need to test further but seem to be finding that my setup went through a number of unexpected reboots when both target tracking and UART output were on whilst I was debugging range and sensitivity

crlogic · July 31, 2022, 3:04pm

Sensitivity != distance

I have recently updated the OP with better tuning descriptions.

crlogic · July 31, 2022, 3:08pm

That looks very nice! Well done.

I would caution against changing id: as it will be more important in upcoming revisions. name: is fine and can be changed to taste. I will be sure to better highlight this in the future.

crlogic · July 31, 2022, 3:16pm

Target tracking is definitely chattier and with full debug logging that can occur. I will add an uptime sensor in order to better track this metric moving forward. Especially as I have no ESP8266’s.

The version that defaults to target tracking ON has not been posted. And will not be posted without wider testing. I have noticed that I cannot use longer USB extension cords with target tracking. I just assumed the added power draw + my 16’ cord was pushing it (because with a shorter cable it is ok)

ajoyce · July 31, 2022, 3:32pm

I would caution against changing id:

Noted, I’ll keep an eye out for the next version and switch my ids back!

Sensitivity != distance

I know, trouble is that I need higher sensitivity to pick up micro motion at the other end of my living room. And the detections I’ve had in that particular area in my hallway have had pretty good SNRs too (up to 2.0). I’ll have to keep finessing I guess! Will probably have to be longer interval and lower sensitivity…

I have noticed that I cannot use longer USB extension cords with target tracking

I hadn’t considered that at all! I have got mine hooked up with quite a long cable. It’s not an issue for me because I can only see myself using target tracking for initial setup (for now at least) so I don’t need it to be stable long-term.

Great work with all this by the way!

crlogic · July 31, 2022, 3:40pm

Ideally we have it enabled without impact to stability across difference devices and implementations. I will be sure to keep the tracking on/off switch otherwise since that really isn’t the key feature being worked. It is the led/distance/latency/sensor_on/off state between the MCU & ESP that I am aiming to better keep in sync.

Since distance is going to be the most powerful tuning mechanism. If you can find a suitable sensor placement location that does not overlap livingroom and hallway distance, that would be best. That way you can max out the sensitivity within the configured distance with less worry about hallway motion. The target tracking is very useful here to understand what distance numbers the sensor is seeing.

hjmcnew · July 31, 2022, 6:01pm

@crlogic I was in the process of spinning up a repo on GitHub to create a custom component based on the progress you’ve made but would prefer not to duplicate efforts. I’d be happy to help out with your project. Is there somewhere we can collaborate on the code?

crlogic · July 31, 2022, 6:12pm

The more hands the merrier (i think). DM sent.

crlogic · July 31, 2022, 6:17pm

I have been doing additional checks on this topic and noticed that leaving the web_server page open in Chrome triggered a watchdog reset. The odd time I would see an rx_buf error but didn’t catch all of it.

When I do not leave a browser sitting on the ESP web_server things run well (top and bottom entries below). All restarted manually at roughly the same time.

Middle sensor below had page left open and it would run 3-10 min.

My thoughts for now would be; use the web_server for tuning and not a dashboard.

cyben76 · August 1, 2022, 4:12am

7 left in stock.

MJWMJW · August 1, 2022, 8:55am

I really like your little housing, would you share it?

ajoyce · August 1, 2022, 9:30am

Here you go: Case for SEN0395 (DFRobot mmWave) and Wemos D1 Mini by alasdairj - Thingiverse

KCYeoh · August 1, 2022, 12:09pm

Will this pick up the fan movement?

duceduc · August 2, 2022, 12:57am

Do you mind taking some close up images of your device? I want to see how you bridged the pins to the mmwave.