How to read and parse UART HEX data?

CircuitPython is great too.

My challenge was dropping off wifi after a day or two. Which may have changed in the past year. I guess I’ll find out!

Massaged the python library into CircuitPython on an ESP32-S2 sending data to WLED via UDP Sync. Maybe one day I’ll know enough C++ to get it working in ESPHome.

Pretty basic POC using radar distance tracking: Radar tracking - Album on Imgur

2 Likes

Dunno if it would help but have you tinkered with some of the other debug options?

Stuff like this…


uart:
  - id: uart_bus2
    tx_pin: GPIO17
    rx_pin: GPIO16
    debug:
      direction: BOTH
      dummy_receiver: true
      after:
        bytes: 4
      sequence:     
        - lambda: UARTDebug::log_int(direction, bytes, ',');

Thanks for the suggestion. The sensor in question does not use line/carriage returns of any sort. After further testing I can confirm that fixed size byte arrays are indeed what are used. Got it working with MicroPython and have yet to loop back to decipher the hex data in cpp. Thx!

1 Like

I’m having kind of a similar issue, as discussed in https://community.home-assistant.io/t/intercept-echo-filter-modbus-communications-from-1-uart-hub-to-another-vs-pool-pump-solar-speed-override-project/340383. Hope I’m not hijacking.

At this point I’m trying to understand how to actually consume the data that is coming in over UART/Modbus. It’s being written into the log just fine but exactly how do I get to it in a usable form? Been reading up on C++ a bit to try to understand stuff on the documentation pages but I think I need a little more advice. Thanks!

I’d presume that the external component publishes something to be used. Haven’t checked the git repo. Are there examples?

I’ll try and take a look when I can, perhaps it could help me as well. I’ve been playing with Arduino Serial examples. Suspect theses are two different issues as I am guessing the component you have found does the parsing.

I’ve gotten to the point where I can send hex data from my PC to the ESPHome node via RS485 and have that data logged by ESPHome as raw hex as well as ASCII characters. I can also send either predefined raw hex bytes via UART/RS485 or predefined hex commands (with automatically generated CRC bytes) via Modbus/RS485. So I just need to be able to read and filter incoming hex so I can pass it out again over the second physical UART/RS485 interface, substituting my custom commands for the filtered-out commands.

OK, some things are slowly coming to me, so I’m going to try to get them down here before I lose them.
It now seems to me that your data resides in the “bytes” variable in that lambda line that’s logging the data. The “sequence” action publishes “bytes” to the log . The logging helper that you left uncommented is showing the data a string with some characters escaped, hence the backslashes. So, I’m thinking that if you use that lambda to assign a value of “bytes” to your text sensor, your text sensor’s state would be the string of data you received. That could then be manipulated as necessary.

Your lambda might be something like:

 id(text_sensor.uart_hex).state = bytes;

or:

 id(text_sensor.uart_hex).publish_state(bytes);

Maybe?

1 Like

I’m a newbie to this stuff, but I sensed when getting help on something a little similar, that this is the level/complexity where advanced users kind of recommend moving from a ESPHome config solution to a custom component solution in C++.

You could look over this and the GitHub project if you’re interested.

I would head over to Discord and see what the guru’s reckon.

Thanks for the reference. That repo has a lot to go through!

I believe it’s this block which is the main logic which decodes the UART message and publishes part of it as a sensor.

Have to admit I have a hard time following it too;)

Sorry about image. Lazy on my mobile.

Yes that looks accurate. While I don’t know much C++ (aside from basic concepts and formatting). I can read through that well enough. The struggle is typically types and converting between them. Especially hex strings to vectors!

Not sure if you’re still working on this… maybe you’d rather me help you with some other c++ when I’m a bit bored on a weekday.

But… I don’t think you even need to bother with a custom component since it’s fairly simple. 15 lines or so of your lambda will do the trick

You’ve got the bytes variable inside a lambda which is a string so you’re off to the races. You check to make sure its the message you want by checking the length or by checking if it starts with an “r” for resp.

Then take a substring to get the last 4x4 bytes. (str.substr). Then just use index operators on your string to get your bytes also known as characters.

You can do some char array (also known as byte array) to float conversion then just go id(my_sensor).publish_state(my_value) for each of the 4 sensors inside the lambda

You can convert an array of 4 bytes to a float using a memcpy or just by casting from a 4 byte array → int → float

Hopefully that nails it down and I could happily provide some more concrete code

I’ll take you up on that. This has been a slow-burn project due to the large jump in complexity for me.

I’ll take a stab at this. Especially since bytes length may be variable depending if there is data in the RESP or not.

On second thought, I would slap a Custom Component on it so you can do more advanced things later like manipulating settings. I’ll get you started…

Ultimately the goal was to have it HA configurable/updateable while having local UDP “send” ala custom_component somehow got working but don’t know how to combine yet…

#include "WiFiUdp.h"

class MyCustomSensor : public PollingComponent, public Sensor {
    public:
        // constructor
        MyCustomSensor() : PollingComponent(15000) {}
        WiFiUDP UDP;

    void setup() override {
        // This will be called by App.setup()
    }
    void update() override {
        // This will be called every "update_interval" milliseconds.
        unsigned int remotePort = 21324;
        IPAddress ip(192, 168, 0, 119);
        uint8_t wledsync[7];
        wledsync[0] = 4; //DNRGB
        wledsync[1] = 1; //RTN
        wledsync[2] = 0; //High
        wledsync[3] = 0; //Low
        wledsync[4] = 255;
        wledsync[5] = 5;
        wledsync[6] = 255;

        for (int i = 0; i <= 255; i++) {
	    ESP_LOGD("custom", "Sending UDP ", i);
            wledsync[3] = i; //Low
            // Send return packet
            UDP.beginPacket(ip, remotePort);
            UDP.write(wledsync,7);
            UDP.endPacket();
        }

    }
};

This is where I last left off, not working yet as the whole ESPHome custom serial bit is still mysterious to me (and types and whatever << is supposed to do!).

#include "esphome.h"

class kld7 : public Component, public UARTDevice {
 public:
  kld7(UARTComponent *parent) : UARTDevice(parent) {}

  void setup() override {
    // nothing to do here
  }
  void loop() override {
    // Use Arduino API to read data, for example
    const int BUFFER_SIZE = 25;
    byte TDAT[BUFFER_SIZE];
    int i = 0;
    while (available() >= BUFFER_SIZE) {
      // read the incoming bytes:
      TDAT[i] = read();
      i++;
    }
    if (sizeof(TDAT) == 25) {
      uint16_t kld7_distance = uint16_t(TDAT[18]) | (uint16_t(TDAT[19]) << 8);   
      uint16_t kld7_speed = uint16_t(TDAT[20]) | (uint16_t(TDAT[21]) << 8);   
      uint16_t kld7_angle = uint16_t(TDAT[22]) | (uint16_t(TDAT[23]) << 8);   
      id(distance).publish_state(kld7_distance);
      id(speed).publish_state(kld7_speed);
      id(angle).publish_state(kld7_angle);
      ESP_LOGD("kld7", "TDAT is: %s", TDAT);
    }
    while (available() > 0) {
      // read the incoming bytes:
      read();
    }
  }
};

@phillip1 makes it look easy and I know that it isn’t easy! Huge thanks!

Basic target location working for the K-LD7!!

A few questions for those experienced with custom devices in ESPHome - how can we avoid the pollingComponent?

Todo:

  • remove polling
  • add the numerous target customization parameters
  • add WLED target tracking
#include "esphome.h"

class kld7 : public PollingComponent, public UARTDevice {
 public:
  kld7(UARTComponent *parent) : PollingComponent(125), UARTDevice(parent) {}
  Sensor *distance_sensor = new Sensor();
  Sensor *speed_sensor = new Sensor();
  Sensor *angle_sensor = new Sensor();
  Sensor *db_sensor = new Sensor();
  
  void setup() override {

  }

  std::vector<int> bytes;

//  void loop() override {  
  void update() {
    while(available() > 0) {
      bytes.push_back(read());      

      //make sure at least 8 header bytes are available to check
      //ESP_LOGD("custom", "checking for RESP");
      if(bytes.size() < 8)       
      {
        continue;  
      }
      //ESP_LOGD("custom", "checking bytes");
      if(bytes[0] != 0x54 || bytes[1] != 0x44 || bytes[2] != 0x41 || bytes[3] != 0x54 || bytes[4] != 0x08 || bytes[5] != 0x00 || bytes[6] != 0x00 || bytes[7] != 0x00) {
      //if(bytes[9] != 0x54 || bytes[10] != 0x44 || bytes[11] != 0x41 || bytes[12] != 0x54 || bytes[13] != 0x08 || bytes[14] != 0x00 || bytes[15] != 0x00 || bytes[16] != 0x00) {
        bytes.erase(bytes.begin()); //remove first byte from buffer
        //buffer will never get above 8 until the header is correct
        continue;
      }      
      //ESP_LOGD("custom", "TDAT bytes");
      if(bytes.size() == 16) { //>=
	//ESP_LOGD("custom", "Found TDAT: %X", bytes);
        //ESP_LOGI("test", "data: 0:%d, 1:%d, 2:%d, 3:%d, 4:%d, 5:%d, 6:%d, 7:%d, bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7]);
        TwoByte distance;
        distance.Byte[0] = bytes[8];
        distance.Byte[1] = bytes[9];  
        distance_sensor->publish_state(distance.UInt16);
        //id(distance_sensor).publish_state(distance.UInt16);
	//ESP_LOGD("custom", "Parsed distance: %i", distance.UInt16);

        TwoByte speed;
        speed.Byte[0] = bytes[10];
        speed.Byte[1] = bytes[11];  
        speed_sensor->publish_state(speed.Int16/100);
        //id(speed_sensor).publish_state(speed.Int16/100);
	//ESP_LOGD("custom", "Parsed speed: %i", speed.Int16);

        TwoByte angle;
        angle.Byte[0] = bytes[12];
        angle.Byte[1] = bytes[13];  
        angle_sensor->publish_state(angle.Int16/100);
        //id(angle_sensor).publish_state(angle.Int16/100);
	//ESP_LOGD("custom", "Parsed angle: %i", angle.Int16);

        TwoByte db;
        db.Byte[0] = bytes[14];
        db.Byte[1] = bytes[15];  
        db_sensor->publish_state(db.UInt16/100);
        //id(db_sensor).publish_state(db.UInt16/100);
	//ESP_LOGD("custom", "Parsed db: %i", db.UInt16);

        bytes.clear();
      }
      else {
        //ESP_LOGD("custom", "no bytes");
      }    
    }    
  }

  typedef union
  {
    unsigned char Byte[2];
    int16_t Int16;
    uint16_t UInt16;
    unsigned char UChar;
    char Char;
  }TwoByte;};
1 Like

Hello @crlogic. Could you share your yaml from this device? I need resolve very similar problem.
Thanks!

Found here