mmWave Presence Detection - ESPHome style

Target tracking is definitely chattier and with full debug logging that can occur. I will add an uptime sensor in order to better track this metric moving forward. Especially as I have no ESP8266’s.

The version that defaults to target tracking ON has not been posted. And will not be posted without wider testing. I have noticed that I cannot use longer USB extension cords with target tracking. I just assumed the added power draw + my 16’ cord was pushing it (because with a shorter cable it is ok) :wink:

I would caution against changing id:

Noted, I’ll keep an eye out for the next version and switch my ids back!

Sensitivity != distance :wink:

I know, trouble is that I need higher sensitivity to pick up micro motion at the other end of my living room. And the detections I’ve had in that particular area in my hallway have had pretty good SNRs too (up to 2.0). I’ll have to keep finessing I guess! Will probably have to be longer interval and lower sensitivity…

I have noticed that I cannot use longer USB extension cords with target tracking

I hadn’t considered that at all! I have got mine hooked up with quite a long cable. It’s not an issue for me because I can only see myself using target tracking for initial setup (for now at least) so I don’t need it to be stable long-term.

Great work with all this by the way!

Ideally we have it enabled without impact to stability across difference devices and implementations. I will be sure to keep the tracking on/off switch otherwise since that really isn’t the key feature being worked. It is the led/distance/latency/sensor_on/off state between the MCU & ESP that I am aiming to better keep in sync.

Since distance is going to be the most powerful tuning mechanism. If you can find a suitable sensor placement location that does not overlap livingroom and hallway distance, that would be best. That way you can max out the sensitivity within the configured distance with less worry about hallway motion. The target tracking is very useful here to understand what distance numbers the sensor is seeing.

@crlogic I was in the process of spinning up a repo on GitHub to create a custom component based on the progress you’ve made but would prefer not to duplicate efforts. I’d be happy to help out with your project. Is there somewhere we can collaborate on the code?

The more hands the merrier (i think). DM sent.

I have been doing additional checks on this topic and noticed that leaving the web_server page open in Chrome triggered a watchdog reset. The odd time I would see an rx_buf error but didn’t catch all of it.

When I do not leave a browser sitting on the ESP web_server things run well (top and bottom entries below). All restarted manually at roughly the same time.

Middle sensor below had page left open and it would run 3-10 min.

My thoughts for now would be; use the web_server for tuning and not a dashboard.

7 left in stock.

I really like your little housing, would you share it?

Here you go: Case for SEN0395 (DFRobot mmWave) and Wemos D1 Mini by alasdairj - Thingiverse

4 Likes

Will this pick up the fan movement?

Do you mind taking some close up images of your device? I want to see how you bridged the pins to the mmwave.

Will this pick up the fan movement?

Yes, any movement

I’m having trouble compiling I am using the latest code listed on the OP. This is what I get…

Compiling /data/radarr-sensor/.pioenvs/radarr-sensor/lib64d/WiFi/WiFiScan.cpp.o
/config/esphome/radarr-sensor.yaml: In lambda function:
/config/esphome/radarr-sensor.yaml:339:35: error: expected type-specifier before 'UartReadLineSensor'
       auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
                                   ^
/config/esphome/radarr-sensor.yaml:341:31: error: could not convert '{my_custom_sensor}' from '<brace-enclosed initializer list>' to 'std::vector<esphome::text_sensor::TextSensor*>'
       return {my_custom_sensor};
                               ^
Compiling /data/radarr-sensor/.pioenvs/radarr-sensor/lib64d/WiFi/WiFiServer.cpp.o
Compiling /data/radarr-sensor/.pioenvs/radarr-sensor/lib64d/WiFi/WiFiUdp.cpp.o
Compiling /data/radarr-sensor/.pioenvs/radarr-sensor/lib01c/FS/FS.cpp.o
Compiling /data/radarr-sensor/.pioenvs/radarr-sensor/lib01c/FS/vfs_api.cpp.o
*** [/data/radarr-sensor/.pioenvs/radarr-sensor/src/main.cpp.o] Error 1
========================= [FAILED] Took 50.42 seconds =========================

Anyone have any ideas? I’m not editing the code other than changing it to ESP32 and changing the uart pins to 16 and 17

Did you create one of these from the link?
- uart_read_line_sensor.h # https://esphome.io/cookbook/uart_text_sensor.html

First of all Thanks for the quick reply and all the work you’ve done on this.
But to answer your question no I didn’t create one of those. I looked at the link and saw the file name and what the contents of that file should be so I created it and named it as told.

But it says to put it in the ‘configuration’ folder and I’m not sure what folder that is as I have no folder anything like that except in Home Assistant. My esphome folder doesn’t have a configuration folder so I’m not sure what to do. I tried a bunch of different locations but none changed the outcome.

Do you know where I’m supposed to put ’ uart_read_line_sensor.h'?

Yeah it says your because of different installation options, “(Store this file in your configuration directory, for example uart_read_line_sensor.h )”

You should have an “esphome” folder somewhere. If you are using the ESPHome addon for HA then it is your /config/esphome folder.

Great thanks thats what I figured and tried but I’ll make sure and recheck everything I’m sure I missed something.

You can always post a new thread in ESPHome - Home Assistant Community to review your yaml

Releasing a beta version of the code that offers led, distance, latency and sensitivity setting synchronization that earlier versions lacked. I am leaving uart debugging enabled in order to better understand any issues one may have.

Having run this version all week it runs as stable as the OP with the same constraint that leaving a browser sitting open refreshing the web_server will quickly allow an ESP32 to run out of resources and reboot. Therefore the same warning applies; use the web_server for tuning and not a dashboard.

This version is not recommended unless you have a stable working deployment and have the ability to troubleshoot compilation, perform uart debugging and wired re-flashing if necessary.

A custom .h file is required
uart_read_line_sensor_leapmmw.h

#include "esphome.h"
#include <string>

void publishTarget(std::string idx, float dist, float snr) {
  auto get_sensors = App.get_sensors();
  for(int i = 0; i < get_sensors.size(); i++) {
    auto name = get_sensors[i]->get_name();
    auto target = "target_" + to_string(idx);
    if(name.size() > 10 && name.substr(0, 8) == target) {
      if(name.substr(9, 3) == "dis") {
        get_sensors[i]->publish_state(dist);
      } else if(name.substr(9, 3) == "SNR") {
        get_sensors[i]->publish_state(snr);
      }
    }
  }
};
static void clearTargets () {
  for(int i = 1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
}


class leapmmw : public Component, public UARTDevice {
 public:
  leapmmw(UARTComponent *parent) : UARTDevice(parent) {}
  
  void setup() override {
    //
  }

  void publishNumber (std::string sensor, float resp) {
    auto get_numbers = App.get_numbers();
    for(int i = 0; i < get_numbers.size(); i++) {
      auto name = get_numbers[i]->get_name();
      if(name.size() > 6 && name == sensor) {
        get_numbers[i]->publish_state(resp);
      }
    }
  };

  void publishSwitch(std::string sensor, int state) {
    auto sens = App.get_switches();
    for(int i = 0; i < sens.size(); i++) {
      auto name = sens[i]->get_name();
      if(name.size() > 2 && name == sensor) {
          sens[i]->publish_state(state);
      }
    }
  };

  void getmmwConf(std::string mmwparam) {
    write_array(std::vector<unsigned char>(mmwparam.begin(), mmwparam.end()));
  }

  int readline(int readch, char *buffer, int len)
  {
    static int pos = 0;
    int rpos;

    if (readch > 0) {
      switch (readch) {
        case '\n': // Ignore new-lines
          break;
        case '\r': // Return on CR
          rpos = pos;
          pos = 0;  // Reset position index ready for next time
          return rpos;
        default:
          if (pos < len-1) {
            buffer[pos++] = readch;
            buffer[pos] = 0;
          }
      }
    }
    // No end of line has been found, so return -1.
    return -1;
  }
  std::string getline;

  void loop() override {
    const int max_line_length = 40;
    static char buffer[max_line_length];

    while (available()) {
      if (id(mmwave_sensor).state == 0 && id(num_targets).state > 0) {
        id(num_targets).publish_state(0);
        clearTargets();
      }
      if(readline(read(), buffer, max_line_length) >= 4) {
        std::string line = buffer;

        if (line.substr(0, 18) == "leapMMW:/>$JYBSS,0" && id(num_targets).state > 0) {
          id(num_targets).publish_state(0);
          clearTargets();
        }
        if (line.substr(0, 6) == "$JYRPO") {
          std::string vline = line.substr(6);
          std::vector<std::string> v;    
          for(int i = 0; i < vline.length(); i++) {
              if(vline[i] == ',') {
                  v.push_back("");
              } else {
                  v.back() += vline[i];
              }
          }
          id(num_targets).publish_state(parse_number<float>(v[0]).value());
          publishTarget(v[1], parse_number<float>(v[2]).value(), parse_number<float>(v[4]).value());
          for(int i = parse_number<int>(v[0]).value() +1 ; i < 9; i++) publishTarget(to_string(i), 0, 0);
        }
        if (line.substr(0, 6) == "$JYRPO" && id(mmwave_sensor).state == 0) {
          publishSwitch("mmwave_sensor", 1);
        }

        // compare last line
        if (line.substr(0, 8) == "Response") {
          ESP_LOGD("custom", "Found Response - line is: %s", line.c_str());
          ESP_LOGD("custom", "Found Response - lastline is: %s", getline.c_str());
          
          // leapMMW:/>getSensitivity
          if (getline.substr(0, 24) == "leapMMW:/>getSensitivity") {
            std::string getSensitivity = line.substr(9, 1);
            if (getSensitivity.empty()) {
              ESP_LOGD("custom", "Did not find a value for getSensitivity");
            } else {
              ESP_LOGD("custom", "The value of getSensitivity is: %f", parse_number<float>(getSensitivity).value());
              publishNumber("sensitivity", parse_number<float>(getSensitivity).value());
            }
          }

          // leapMMW:/>getRange
          if (getline.substr(0, 18) == "leapMMW:/>getRange") {
            std::string getRange = line.substr(15, 5);
            if (getRange.empty()) {
              ESP_LOGD("custom", "Did not find a value for getRange");
            } else {
              ESP_LOGD("custom", "The value of getRange is: %f", parse_number<float>(getRange).value());
              publishNumber("distance", parse_number<float>(getRange).value());
            }
          }

          // leapMMW:/>getLatency
          if (getline.substr(0, 20) == "leapMMW:/>getLatency") {
            std::string getLatency = line.substr(15, 3);
            if (getLatency.empty()) {
              ESP_LOGD("custom", "Did not find a value for getLatency");
            } else {
              ESP_LOGD("custom", "The value of getLatency is: %f", parse_number<float>(getLatency).value());
              publishNumber("latency", parse_number<float>(getLatency).value());
            }
          }

          // leapMMW:/>getLedMode
          if (getline.substr(0, 20) == "leapMMW:/>getLedMode") {
            std::string getLedMode = line.substr(11, 1);
            if (getLedMode.empty()) {
              ESP_LOGD("custom", "Did not find a value for getLedMode");
            } else {
              int led_state = parse_number<int>(getLedMode).value();
              ESP_LOGD("custom", "The value of getLedMode is: %i", led_state);
              int setled_state;
              if (led_state == 1) {
                setled_state = 0;
              } else if (led_state == 0) {
                setled_state = 1;
              }
              publishSwitch("led", setled_state);
            }
          }
        }
        if (line.substr(0, 4) == "Done") {
          ESP_LOGD("custom", "Found Done - line is: %s", line.c_str());
          ESP_LOGD("custom", "Found Done - lastline is: %s", getline.c_str());
          // leapMMW:/>sensorStop
          if (getline.substr(0, 20) == "leapMMW:/>sensorStop") {
            ESP_LOGD("custom", "sensorStop completed successfully");
            publishSwitch("mmwave_sensor", 0);
          }

          // leapMMW:/>sensorStart
          if (getline.substr(0, 21) == "leapMMW:/>sensorStart") {
            ESP_LOGD("custom", "sensorStart completed successfully");
            publishSwitch("mmwave_sensor", 1);
          }
        }
        getline = buffer; 
      }
    }
  }
};

Beta YAML

esphome:
  name: tinypico-mmwave-extra
  platform: ESP32
  board: esp32dev
  includes:
    - uart_read_line_sensor_leapmmw.h 
  on_boot:
    priority: -100
    then:
      - script.execute: get_mmwave_params # still needed with getmmwConf()

# Enable logging
logger:
  logs:
    sensor: INFO # DEBUG level with uart_target_output = overload!
    text_sensor: INFO

# Enable Home Assistant API
api:
  reboot_timeout: 6h
  services:
      # Service to send a command directly to the display. Useful for testing
    - service: send_command
      variables:
        cmd: string
      then:
        - uart.write: !lambda
            std::string command = to_string(cmd) +"\r";
            return std::vector<uint8_t>(command.begin(), command.end());

ota:
  password: !secret ota

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: !secret ap_ssid
    password: !secret ap_password

substitutions:
  device_name: mmwave-extra

web_server:
  port: 80
  version: 2
  include_internal: true
  ota: false

http_request:
  useragent: esphome/$device_name
  timeout: 2s

captive_portal:

uart:
  id: uart_bus
  tx_pin: GPIO19
  rx_pin: GPIO23
  baud_rate: 115200
  debug:
    direction: BOTH
    after:
      delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_string(direction, bytes);

binary_sensor:
  - platform: gpio
    name: mmwave_presence_detection
    id: mmwave_presence_detection
    pin:
      number: GPIO18
      mode: INPUT_PULLDOWN
    on_state:
      - if:
          condition:
            binary_sensor.is_off: mmwave_presence_detection
          then:
            - sensor.template.publish:
                id: num_targets
                state: 0
            - lambda: |-
                return clearTargets();
    
sensor:      
  - platform: uptime
    name: uptime_sensor
    id: uptime_sensor
    update_interval: 60s
    internal: true
    on_raw_value:
      then:
        - text_sensor.template.publish:
            id: uptime_human_readable
            state: !lambda |-
                      int seconds = round(id(uptime_sensor).raw_state);
                      int days = seconds / (24 * 3600);
                      seconds = seconds % (24 * 3600);
                      int hours = seconds / 3600;
                      seconds = seconds % 3600;
                      int minutes = seconds /  60;
                      seconds = seconds % 60;
                      return (
                        (days ? to_string(days)+":" : "00:") +
                        (hours ? to_string(hours)+":" : "00:") +
                        (minutes ? to_string(minutes)+":" : "00:") +
                        (to_string(seconds))
                      ).c_str();
              
  - platform: template
    name: target_1_distance_m
    id: target_1_distance_m # do not change
    internal: true
    
  - platform: template
    name: target_2_distance_m
    id: target_2_distance_m # do not change
    internal: true
    
  - platform: template
    name: target_3_distance_m
    id: target_3_distance_m # do not change
    internal: true
    
  - platform: template
    name: target_4_distance_m
    id: target_4_distance_m # do not change
    internal: true
    
  # - platform: template
  #   name: target_5_distance_m
  #   id: target_5_distance_m # do not change
  #   internal: true
    
  # - platform: template
  #   name: target_6_distance_m
  #   id: target_6_distance_m # do not change
  #   internal: true
    
  # - platform: template
  #   name: target_7_distance_m
  #   id: target_7_distance_m # do not change
  #   internal: true
    
  # - platform: template
  #   name: target_8_distance_m
  #   id: target_8_distance_m # do not change
  #   internal: true

  - platform: template
    name: target_1_SNR
    id: target_1_SNR # do not change
    internal: true

  - platform: template
    name: target_2_SNR
    id: target_2_SNR # do not change
    internal: true
    
  - platform: template
    name: target_3_SNR
    id: target_3_SNR # do not change
    internal: true
    
  - platform: template
    name: target_4_SNR
    id: target_4_SNR # do not change
    internal: true

  # - platform: template
  #   name: target_5_SNR
  #   id: target_5_SNR # do not change
  #   internal: true

  # - platform: template
  #   name: target_6_SNR
  #   id: target_6_SNR # do not change
  #   internal: true
    
  # - platform: template
  #   name: target_7_SNR
  #   id: target_7_SNR # do not change
  #   internal: true
    
  # - platform: template
  #   name: target_8_SNR
  #   id: target_8_SNR # do not change
  #   internal: true

  - platform: template
    name: num_targets
    id: num_targets # do not change

  - platform: custom
    lambda: |-
      auto s = new leapmmw(id(uart_bus));
      App.register_component(s);
      return {};
    sensors:
      
switch:
  - platform: safe_mode
    name: use_safe_mode

  - platform: template
    name: mmwave_sensor
    id: mmwave_sensor # do not change
    entity_category: config
    optimistic: true
    # assumed_state: true
    turn_on_action:
      - uart.write: "setUartOutput 1 0"
      - delay: 1s
      - uart.write: "setUartOutput 2 1 1 2"
      - delay: 1s
      - uart.write: "saveConfig"
      - delay: 3s 
      - uart.write: "sensorStart"
      # - delay: 3s
      # - script.execute: get_mmwave_params # not needed with getmmwConf
    turn_off_action:
      - uart.write: "sensorStop"

  - platform: template
    name: led
    id: led  # do not change
    entity_category: config
    optimistic: true
    turn_on_action:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 0"
      - delay: 1s
      - lambda: |-
          leapmmw(id(uart_bus)).getmmwConf("getLedMode 1");
      - delay: 1s 
      - switch.turn_on: mmwave_sensor
    turn_off_action:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: "setLedMode 1 1"
      - delay: 1s
      - lambda: |-
          leapmmw(id(uart_bus)).getmmwConf("getLedMode 1");
      - delay: 1s
      - switch.turn_on: mmwave_sensor 

number:
  - platform: template
    name: distance
    id: distance # do not change
    entity_category: config
    min_value: 0.15
    max_value: 9.45
    initial_value: 6
    optimistic: true
    step: 0.15
    unit_of_measurement: M
    mode: box
    # lambda: |-
    #   return leapmmw(id(uart_bus)).getmmwConf("getRange");
    set_action:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
          std::string range = "setRange 0 " + str_sprintf("%.2f", x);
          return std::vector<unsigned char>(range.begin(), range.end());
      - delay: 1s
      - lambda: |-
          leapmmw(id(uart_bus)).getmmwConf("getRange");
      - delay: 1s
      - switch.turn_on: mmwave_sensor 
      
  - platform: template
    name: latency
    id: latency # do not change
    entity_category: config
    min_value: 1
    max_value: 600
    initial_value: 15
    optimistic: true
    # lambda: |-
    #   return leapmmw(id(uart_bus)).getmmwConf("getLatency");
    step: 1
    unit_of_measurement: s
    mode: box
    set_action:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
          std::string setL = "setLatency 0.1 " + str_sprintf("%.0f", x);
          return std::vector<unsigned char>(setL.begin(), setL.end());
      - delay: 1s
      - lambda: |-
          leapmmw(id(uart_bus)).getmmwConf("getLatency");
      - delay: 1s
      - switch.turn_on: mmwave_sensor

  - platform: template
    name: sensitivity
    id: sensitivity # do not change
    entity_category: config
    min_value: 0
    max_value: 9
    initial_value: 7
    optimistic: true
    # lambda: |-
    #   return leapmmw(id(uart_bus)).getmmwConf("getSensitivity");
    step: 1
    set_action:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: !lambda
          std::string mss = "setSensitivity " + to_string((int)x);
          return std::vector<unsigned char>(mss.begin(), mss.end());
      - delay: 1s
      - lambda: |-
          leapmmw(id(uart_bus)).getmmwConf("getSensitivity");
      - delay: 1s
      - switch.turn_on: mmwave_sensor

button:
  - platform: restart
    name: Restart_ESP_$device_name
    entity_category: diagnostic
    on_press:
      - uart.write:
          id: uart_bus
          data: "resetSystem 0"

  - platform: template
    name: factory_reset_MCU_$device_name
    id: factory_reset_MCU
    entity_category: diagnostic
    on_press:
      - switch.turn_off: mmwave_sensor
      - delay: 1s
      - uart.write: "resetCfg"
      - delay: 3s
      - switch.turn_on: mmwave_sensor 
      
script:
  - id: get_mmwave_params
    then: 
      - uart.write: "getLedMode 1"
      - delay: 1s
      - uart.write: "getRange"
      - delay: 1s
      - uart.write: "getLatency"
      - delay: 1s
      - uart.write: "getSensitivity"
2 Likes

Great work. I’ve tried to implement the code, but there seems the part with the text_sensor missing in the yaml.

Where can I find the text_sensor part?