TL;DR - I took a tracking radar sensor + an ESP32 and made an individually addressable light strip follow me as I walk by. am making a Nerf CWIS for my desk! (it’s a work in progress…)
Tracking servo’s deployed!
Introduction
This project is a confession of what happens when ambition exceeds skills. Having successfully deployed ESPHome projects based on DFRobot and (other) RFBeam mmWave sensors for presence detection, it was time to tackle real-time radar targeting!
Having been slightly disappointed with the RFBeam K-LD2 that provides GPIO-based detection of approach/receeding/left/right target parameters, it was time to upgrade to the RFBeam K-LD7 radar sensor module that provides target distance, angle and speed!
This is what they sell it for; how boring!
With excited goals of creating a Nerf CIWS to keep those pesky furry felines off my work desk, I dove straight into a ESPHome brick wall.
my hopes - an HA controlled sentry…
reality - what do I do with this again…!?
…yes I really did Dremel off the wifi antenna…
K-LD7 ESPHome Support
ESPHome has very wide support for sensor modules and when there is no native support, custom UART device fills in the gaps. Having been successful in all projects to date with no C++ skills, how hard could it be to copy & paste my way to success?
After months of poking away at it, the HA Community came to the rescue. Before we got there, it had been a struggle.
Python Support
As it turns out, there is excellent Python support for the K-LD7 which introduced me to using bytearrays and hexadecimal data. That library can be stripped down and works very well in CircuitPython on an ESP32; even at very high baud rates over wifi.
A New Hope
Having shelved the ESPHome side of the project for a while, help soon came from @phillip1 in developing the C++ .h necessary for reading and parsing, not only target data, but configuration parameters as well.
How does it get targeting data?
A vastly oversimplified explanation is;
-
an analogue modulating mmWave transmitter emits a radiation pattern
-
two I/Q receivers sample ADC data
-
the onboard MCU performs FFT magic…
-
candidate targets above threshold are generated
-
a single track is generated for speed, angle and distance for the most dominant target
Sensor Placement
It is important to understand the relation between sensor orientation and the angle output. The K-LD7 uses a 2D array which means that depending on placement, angle is either degrees “left and right”, or “up and down”. But not both. Keep your eye open for 3D tracking mmWave sensor modules (this year?).
Capturing Target Data
All this magic is performed locally on the K-LD7 with a long list of parameters that can be used for target discrimination. The datasheet goes into detail of all hex values that the sensor accepts and reports. More on this later.
The challenge is getting them into ESPHome at a default eight times a second (~125ms per target reporting interval). When you have an unsupported sensor using a custom device, custom parsing code must be written in C++.
Target Speed to Refresh Rate Relationship
There is an inversely proportional relationship between target speed resolution and sensor refresh rate. Depending on the intended installation, this is an important parameter to consider as it includes a power-consumption impact.
ESPHome Custom UART Device
From the YAML perspective, it seems so innocent. Yet there is lot that goes into making these four sensors available. Please note that since that target data can potentially update up to eight times a second, they are marked internal: true in order to preserve the default History configuration of Home Assistant. This is easily changed if the use-case warrants. In the example provided, HA does not need the target data and is only used for configuration and OTA.
sensor:
- platform: custom
lambda: |-
auto s = new kld7(id(uart_bus0));
App.register_component(s);
return {s->distance_sensor, s->speed_sensor, s->angle_sensor, s->db_sensor};
sensors:
- name: distance_sensor
id: distance_sensor
internal: true
unit_of_measurement: cm
- name: speed_sensor
id: speed_sensor
internal: true
unit_of_measurement: kph
- name: angle_sensor
id: angle_sensor
internal: true
unit_of_measurement: deg
- name: db_sensor
id: db_sensor
internal: true
unit_of_measurement: dB
Radar Parameters
But wait, there’s more UART data we can use! The radar is highly configurable. So much so that you can get yourself into situations where nothing can possibly be detected! Study the datasheet parameter chart carefully for best results.
In order to tune these parameters, each setting is configurable in HA and the sensor even synchronizes these settings on every boot. There is also a factory_reset option in order to start over should you get into a situation where zero targets are detected due to self-cancelling parameter settings.
What do we do with it?
Anything you want! As a example, I have included the code that takes the targeting data and uses WLED UDP Realtime Control to light up an LED based on distance data. This very simple example is but one of many possibilities that are made available when you can track an object in 2D space.
Custom Device C++
A thank you goes out to @phillip1 for the support as I demonstrated all the traits of a C++ noob. I have learned a lot and wanted to highlight some of code. While the ESPHome documentation is great to get you started, knowing your types, conversions and general programming principles helps a lot.
Sensor Application Response
Every sensor command issued by a yaml uart.write returns a dedicated hexadecimal response. Validating a few specific response is performed by the following;
bool doesHeaderMatch(std::vector<uint8_t> bytes, std::string header, int payloadLength) {
if (bytes[0] != header[0] || bytes[1] != header[1] || bytes[2] != header[2] || bytes[3] != header[3]) {
return false;
}
// ESP_LOGD("custom", "header matched: ", header);
return bytes[4] == payloadLength && bytes[5] == 0 && bytes[6] == 0 && bytes[7] == 0;
}
Should you wish to add responses in the datasheet that we have yet to, a matching struct/union is needed with the appropriate bytes sizes from the datasheet.
typedef struct {
uint16_t distance;
int16_t speed;
int16_t angle;
uint16_t db;
} TDAT;
typedef union {
TDAT tdat;
uint8_t bytes[sizeof(TDAT)];
} TDATUnion;
Hex Parsing and Publishing
The combination struct/union make publishing of the data quite concise;
TDATUnion tdatUnion;
std::copy(bytes.begin() + 8, bytes.end(), tdatUnion.bytes);
// ESP_LOGD("custom", "Sending TDAT");
distance_sensor->publish_state(tdatUnion.tdat.distance);
speed_sensor->publish_state(tdatUnion.tdat.speed / 100);
angle_sensor->publish_state(tdatUnion.tdat.angle / 100);
db_sensor->publish_state(tdatUnion.tdat.db / 100);
Smart Servos
This needs a post on its own - but the Lynxmotion LSS work very well w/ ESPHome and save you from having to write your own PWM PID / rotary encoder things…
What’s left?
Semi-auto Nerf is next…
Todo
- a few of the configuration parameters use non-linear index-to-result parameters. ESPHome
selectcould make this more human readable but isn’t quite baked yet as it.
The Code
Appendix
Caveats and Lessons Learned
- adding too much debugging will slow down target tracking publishing
- ESPHome UART
debugmakes the sensor read unreliable
FAQ
- How do I wire the sensor?
** Pinouts are in the datasheet - I used your pins and it didn’t work
** yup
- Wifi doesn’t work!
** replace myEthernetsection with an appropriateWifisection in the YAML - How does this C++ deployment compare to the CircuitPython one?
** they both work very well, ESPHome is just easier to OTA and control w/ HA - Do I need an Ethernet ESP32
** No - What about this_other_random_mmWave_sensor?
** buy it and tell us! - Where is the turret build?
**
- Were is the WLED usermod for the K-LD7?
** please go submit one!
More Project Ideas
- calculate relative coordinates using distance and angle to plot robot vacuum movement
- creepy doll/picture with servo controlled eyes that follows you as you go by?
- outdoor servo-controlled spotlight that lights up and tracks (perhaps 3D is better for this)
- a Lovelace card that plots a live track at x interval…
My mmWave Projects
Tracking Radar
mmWave Presence Detection
Low-Latency DFRobot+PIR
mmWave Wars: one sensor (module) to rule them all
SO I changed