How to transfer data from uart

The uart-logging-hack:

  • doesn’t have a setup() if you need to perform some initialization (esphome.on_boot could be work around)
  • is constrained to cases where the rx data packets can be handled by the uart debug after mapping
  • only gets called when data arrives

I think reliability and performance should be similar between both approaches. The uart-logging-hack is only called when there is data to process but the framework is still checking and buffering the data in the background.

1 Like

I have problems with a very similar set-up which is a modified version of your suggestion to capture a Rx into the uart

[22:38:04][D][uart_debug:158]: <<< "1:4.12\n"
[22:38:08][D][uart_debug:158]: <<< "1:4.12\n"
[22:38:10][D][uart_debug:158]: <<< "1:4.12\n"

For some reason I think it is not being ‘captured’ by the if statement and the ‘batVol’ sensors don’t update when a message is received (I have adjusted names of variables from your example)

esphome:
  name: "esp-now-hub-v1"

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:
  level: verbose

uart:
  id: uart_bus
  rx_pin: 16
  tx_pin: 17
  baud_rate: 115200
  debug:
    direction: BOTH
    dummy_receiver: true
    after:
      delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_string(direction, bytes);
          int sensorID=0;
          float sensorBatVol=0; 
          std::string str(bytes.begin(), bytes.end());
          //there are x sensors
          //where<<< id:batteryvoltage
          //"1:4.00\n"
          if (sscanf(str.c_str(), "%d:%f", &sensorID, &sensorBatVol) == 2 ) {
            if(sensorID==1) {
              id(batvol1).publish_state(sensorBatVol); 
            }
            if(sensorID==2) {
              id(batvol2).publish_state(sensorBatVol);  
            }
            if(sensorID==3) {
              id(batvol3).publish_state(sensorBatVol);  
            }
            if(sensorID==4) {
              id(batvol4).publish_state(sensorBatVol); 
            }        
          }



# Enable Home Assistant API
api:

ota:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esphome-Web-6Bdf8C"
    password: "hnLdPNIUXUZq"

captive_portal:

sensor:
  - platform: wifi_signal # Reports the WiFi signal strength/RSSI in dB
    name: "WiFi Signal dB"
    id: wifi_signal_db
    update_interval: 60s
    entity_category: "diagnostic"

  - platform: copy # Reports the WiFi signal strength in %
    source_id: wifi_signal_db
    name: "WiFi Signal Percent"
    filters:
      - lambda: return min(max(2 * (x + 100.0), 0.0), 100.0);
    unit_of_measurement: "Signal %"
    entity_category: "diagnostic"
  - platform: template
    name: "Battery Voltage 1"
    id: "batvol1"
    update_interval: 5s
  - platform: template
    name: "Battery Voltage  2"
    id: "batvol2"
  - platform: template
    name: "Battery Voltage  3"
    id: "batvol3"
  - platform: template
    name: "Battery Voltage  4"
    id: "batvol4"
button:
  - platform: template
    name: "Test Button 1"
    on_press:
      - uart.write: "1:4.1\n"
  - platform: template
    name: "Test Button 2"
    on_press:
      - uart.write: "1:2.2\n"  

I also incorporated the test buttons and loop back wire. The test button creates the response at the Rx but again the sensorBatVol never updates.

Things to note:

There are only 2 arguments. the Sensor number and the Battery voltage
I only seen to have a /n in my message (even though I added a /r as well on the Tx of the message sending ESP32.

Also - have I amended this line correctly?

 if (sscanf(str.c_str(), "%d:%f", &sensorID, &sensorBatVol) == 2 ) {
1 Like

btw just to add a bit more information. This is the write statement to the sending ESP 32 Tx port

  Serial2.write("1:4.12", 6); // write data to serial port for onward transmission to Home Assistant
  Serial2.write('\r\n');

I should be able to work this out but it’s got me beat! :expressionless:

ah ha found the error. It was formatting

I changed this:

      - lambda: UARTDebug::log_string(direction, bytes);
          int sensorID=0;

to this:

      - lambda: |-
          UARTDebug::log_string(direction, bytes);
          int sensorID=0;

Clearly it makes a difference!

1 Like

Hi.
I’m trying to use your code to extract specific values from the string received by UART.
I receive this string:

57,67,251,0,1,0,31,249

I would like to get the first two values from this string and add them to the HA entity.
Will you help?

Take a look at this post for some more general examples. Assuming you want the first two values in HA as numeric sensors, modify the section with sscanf to extract the first two values. Example was changed to use comma as delimiter between the values. The ==2 make sure sscanf reads two values.

          //Sample uart text protocol 
          if (sscanf(str.c_str(), "%d,%d", &val1, &val2) == 2 ) {
              id(sensor1).publish_state(val1); 
              id(sensor2).publish_state(val2);
          }
3 Likes

The code probably works but not entirely correctly.
Code:

    sequence:
    - lambda: |-
       UARTDebug::log_int(direction, bytes,',');

         int val1=0;
         int val2=0;
         std::string str(bytes.begin(), bytes.end());

       if (sscanf(str.c_str(), "%d,%d", &val1, &val2) == 2 ) {
       id(sensor1).publish_state(val1); 
       id(sensor2).publish_state(val2);
          }

sensor:
  - platform: template
    name: "KOCIOŁ"
    unit_of_measurement: "°C"
    id: "sensor1"
  - platform: template
    name: "ZASOBNIK C.W.U."
    unit_of_measurement: "°C"
    id: "sensor2"

And these two entities in HA still have the “unknown” status

How do the ESPHOME logs look?

Is this from UARTDebug::log_string or UARTDebug::log_int? If it is log_int then you would just need this in the debug lambda:

       UARTDebug::log_int(direction, bytes,',');
       id(sensor1).publish_state( bytes[0] ); 
       id(sensor2).publish_state( bytes[1]);

The log_string debug output treats the received byte vales as the ASCII index for each character. For the log_int, it prints the byte values received formatted as integers.

1 Like

Amazing, it works perfectly.
Thank you very much.

1 Like

Hi Gents,
@mulcmu this is a very old post with a very smart solution, that’s awesome!
You saved my day!
Best,
Luca

1 Like

hi all
can someone tell me what is wrong with my code ?

esp dont put values in templates , rawstring is ok
my code:

uart:
  baud_rate: 9600
  tx_pin: GPIO17 
  rx_pin: GPIO16
  debug:
    direction: RX
    dummy_receiver: true
    after:
      delimiter: "\n\r"
    sequence:
      - lambda: |-
          UARTDebug::log_string(direction, bytes);
          std::string str(bytes.begin(), bytes.end());
          id(rawString).publish_state(str.c_str()); 
          float sensors_values0 =0;
          float sensors_values1 =0;
          float sensors_values2 =0;
          if (sscanf(str.c_str(), "%f;%f;%f", &sensors_values0, &sensors_values1, &sensors_values2) == 3 ) {
              id(tempA1).publish_state(sensors_values0);            
              id(tempA2).publish_state(sensors_values1);
              id(tempA3).publish_state(sensors_values2);
          }         

sensor:
  - platform: template
    name: "A1"
    id: "tempA1"
  - platform: template
    name: "A2"
    id: "tempA2"
  - platform: template
    name: "A3"
    id: "tempA3"

thanks

The sscanf format string %f is for base10 numeric floating point values. The first value in your input “AA” can’t be converted. If it is a hexadecimal value then you can use %x.

thank you , it works fine .

does it change a lot if first value in input is “SR”

second problem : how in lambda check if first input is “AA” or “SR”

%s will read in a string. If you only have the two options “AA” or “SR” you could do two sscanf() statements with “AA;%f;%f” and “SR;%f;%f”

I’m trying to send and receive data from an ESP32 to an Arduino Mega via UART.

Sending goes from the ESP to the Mega in the direction of Mega.

I can see that in the Mega log and in the ESP log.

When data is sent from the Mega, I can still see it in the serial monitor in the Mega but nothing arrives in the ESP, at least that’s what it looks like.

Can someone please help me?

her is the Arduino-Mega Serial-Output.
first lines is from ESP-restart.

12:34:08.518 -> Empfangen vom ESP : [878781][D][Updater.cpp:133] begin(): OTA Partition: app0

12:34:20.850 -> Empfangen vom ESP : [891108][V][WiFiGeneric.cpp:360] _arduino_event_cb(): STA Disconnected: SSID: ASUS, BSSID: 40:b0:76:9a:4d:70, Reason: 8

12:34:20.881 -> Empfangen vom ESP : [891108][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino EvenCONNECTED

12:34:20.881 -> Empfangen vom ESP : [891115][W][WiFiGeneric.cpp:950] _eventCallback(): Reason: 8 - 3][V][WiFiGeneric.cpp:341] _arduino_event_cb(): STA Stopped

12:34:21.213 -> Empfangen vom ESP : [891128][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino Even[   753][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino Event: 0 - WIFI_READY

12:34:21.213 -> Empfangen vom ESP : [   786][V][WiFiGeneric.cpp:338] _arduino_event_cb(): STA Started

12:34:21.248 -> Empfangen vom ESP : [   787][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino Event: 2 - STA_START

12:34:27.149 -> Empfangen vom ESP : [  6710][V][WiFiGeneric.cpp:381] _arduino_event_cb(): SCAN Done: ID: 128, Status: 0, Results: 9

12:34:27.149 -> Empfangen vom ESP : [  6710][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino Event: 1 - SCAN_DONE

12:34:28.017 -> Empfangen vom ESP : [  7558][V][WiFiGeneric.cpp:353] _arduino_event_cb(): STA Connected: SSID: ASUS, BSSID: 40:b0:76:9a:4d:70, Channel: 10, Auth: WPA2_PSK

12:34:28.017 -> Empfangen vom ESP : [  7559][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino Even][V][WiFiGeneric.cpp:367] _arduino_event_cb(): STA Got New IP:192.168.2.119

12:34:28.061 -> Empfangen vom ESP : [  7574][D][WiFiGeneric.cpp:929] _eventCallback(): Arduino EvenFiGeneric.cpp:991] _eventCallback(): STA IP: 192.168.2.119, MASK: 255.255.255.0, GW: 192.168.2.1

12:36:47.411 -> Empfangen vom ESP : 
12:38:17.789 -> Empfangen vom ESP : DataToSend from button esp
12:38:44.242 -> Nachricht Serial.print von Mega To Esp: dd

The last lines 12:38:17.789 , was i pressed the button on the ESP
The last line shows a manual input from me on the Arduino Mega’s serial monitor

The Logfile from ESP32 (ESP32-S2-DevKit-M-1)

[12:34:30][I][app:100]: ESPHome version 2024.10.3 compiled on Nov 19 2024, 12:32:50
[12:34:30][C][wifi:600]: WiFi:
[12:34:30][C][wifi:428]:   Local MAC: 70:04:1D:FB:D9:DA
[12:34:30][C][wifi:433]:   SSID: [redacted]
[12:34:30][C][wifi:436]:   IP Address: 192.168.2.119
[12:34:30][C][wifi:440]:   BSSID: [redacted]
[12:34:30][C][wifi:441]:   Hostname: 'watercontroltest'
[12:34:30][C][wifi:443]:   Signal strength: -46 dB ▂▄▆█
[12:34:30][V][wifi:445]:   Priority: 0.0
[12:34:30][C][wifi:447]:   Channel: 10
[12:34:30][C][wifi:448]:   Subnet: 255.255.255.0
[12:34:30][C][wifi:449]:   Gateway: 192.168.2.1
[12:34:30][C][wifi:450]:   DNS1: 192.168.2.1
[12:34:30][C][wifi:451]:   DNS2: 0.0.0.0
[12:34:30][C][logger:185]: Logger:
[12:34:30][C][logger:186]:   Level: VERBOSE
[12:34:30][C][logger:188]:   Log Baud Rate: 115200
[12:34:30][C][logger:189]:   Hardware UART: USB_CDC
[12:34:30][C][uart.arduino_esp32:151]: UART Bus 0:
[12:34:30][C][uart.arduino_esp32:152]:   TX Pin: GPIO44
[12:34:30][C][uart.arduino_esp32:153]:   RX Pin: GPIO45
[12:34:30][C][uart.arduino_esp32:155]:   RX Buffer Size: 256
[12:34:30][C][uart.arduino_esp32:157]:   Baud Rate: 115200 baud
[12:34:30][C][uart.arduino_esp32:158]:   Data Bits: 8
[12:34:30][C][uart.arduino_esp32:159]:   Parity: NONE
[12:34:30][C][uart.arduino_esp32:160]:   Stop bits: 1
[12:34:30][C][uart.button:014]: UART Button 'UART String Output'
[12:34:30][C][captive_portal:089]: Captive Portal:
[12:34:30][C][web_server:153]: Web Server:
[12:34:30][C][web_server:154]:   Address: 192.168.2.119:80
[12:34:30][C][mdns:116]: mDNS:
[12:34:30][C][mdns:117]:   Hostname: watercontroltest
[12:34:30][V][mdns:118]:   Services:
[12:34:30][V][mdns:120]:   - _esphomelib, _tcp, 6053
[12:34:30][V][mdns:122]:     TXT: friendly_name = WaterControler_test
[12:34:30][V][mdns:122]:     TXT: version = 2024.10.3
[12:34:30][V][mdns:122]:     TXT: mac = 70041dfbd9da
[12:34:30][V][mdns:122]:     TXT: platform = ESP32
[12:34:30][V][mdns:122]:     TXT: board = esp32-s2-saola-1
[12:34:30][V][mdns:122]:     TXT: network = wifi
[12:34:30][V][mdns:122]:     TXT: api_encryption = Noise_NNpsk0_25519_ChaChaPoly_SHA256
[12:34:30][V][mdns:120]:   - _http, _tcp, 80
[12:34:30][C][esphome.ota:073]: Over-The-Air updates:
[12:34:30][C][esphome.ota:074]:   Address: 192.168.2.119:3232
[12:34:30][C][esphome.ota:075]:   Version: 2
[12:34:30][C][esphome.ota:078]:   Password configured
[12:34:30][C][safe_mode:018]: Safe Mode:
[12:34:30][C][safe_mode:020]:   Boot considered successful after 60 seconds
[12:34:30][C][safe_mode:021]:   Invoke after 10 boot attempts
[12:34:30][C][safe_mode:023]:   Remain in safe mode for 300 seconds
[12:34:30][C][api:140]: API Server:
[12:34:30][C][api:141]:   Address: 192.168.2.119:6053

The ESPhome-Code


uart:
  tx_pin: GPIO44
  rx_pin: GPIO45
  id: uart_bus
  baud_rate: 115200
  debug:
    direction: BOTH
    dummy_receiver: true
    #after:
    #  delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_string(direction, bytes);

button:
  - platform: uart
    name: "UART String Output"
    data: 'DataToSend from button esp'
#custom_component:
#- lambda: |-
#    auto my_custom = new MyCustomComponent(id(uart_bus));
#    return {my_custom};

captive_portal:

sorry all, now works fine… i used the wrong RX/TX Pins…

I have an evacuated tube HWS with a controller for the pump etc, The controller has a serial connection which I have connected an ESP32 and am reading the binary data coming from it.
I have talked with the manufacturer and they state that it must be read in binary.

the MSB of each byte is part of the version number
The next three bits contain the sequence number of the byte “0-7”
The remaining four bits contain data.
The first six bytes (last four bits of each or nibble) are the upper and lower temperature readings.
The last two bytes are status bits eg: pump on/off etc

I’ve managed to get the binary data in the logs with some trial and error.
I would like to be able to turn the data into sensors, three temperature readings and three statuses but I’m at a loss now and can’t find any examples reading binary data.

Here’s my yaml.
I’ve tried setting after bytes to 8 but it seems to cause timeout issues and disconnections. I randomly picked 80 and it seems to work without error but repeats which is not ideal.
Is there a way to get it to sucessfully read just eight bytes or is that irrelevant?

esphome:
  name: apricus-hws
  friendly_name: Apricus HWS

esp32:
  board: esp32-c3-devkitm-1
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "dXybdS3IqHoFyjf8j7CxUGN342/xFlZhNT9ea43JB3c="

ota:
  - platform: esphome
    password: "0f5953e7a20348dd1b540fe0e8f9e743"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  power_save_mode: HIGH

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Apricus-Hws Fallback Hotspot"
    password: ""

captive_portal:

uart:
  baud_rate: 4800
  rx_pin: 20
  debug:
    direction: RX
    dummy_receiver: true
    after:
      bytes: 80
    sequence:
      - lambda: UARTDebug::log_binary(direction, bytes, ';');

Heres a copy of some of the logs. You can’t write to it only receive and it just continuously sends data.

INFO ESPHome 2024.11.2
INFO Reading configuration /config/esphome/apricus-hws.yaml...
INFO Starting log output from 192.168.0.28 using esphome API
INFO Successfully connected to apricus-hws @ 192.168.0.28 in 0.113s
INFO Successful handshake with apricus-hws @ 192.168.0.28 in 0.075s
[20:25:57][I][app:100]: ESPHome version 2024.11.2 compiled on Dec  1 2024, 14:53:50
[20:25:57][C][wifi:600]: WiFi:
[20:25:57][C][wifi:428]:   Local MAC: 24:58:7C:36:52:4C
[20:25:57][C][wifi:433]:   SSID: 'TelstraC18C1E'[redacted]
[20:25:57][C][wifi:436]:   IP Address: 192.168.0.28
[20:25:57][C][wifi:440]:   BSSID: A2:B5:3C:C1:8C:26[redacted]
[20:25:57][C][wifi:441]:   Hostname: 'apricus-hws'
[20:25:57][C][wifi:443]:   Signal strength: -49 dB ▂▄▆█
[20:25:57][C][wifi:447]:   Channel: 6
[20:25:57][C][wifi:448]:   Subnet: 255.255.255.0
[20:25:57][C][wifi:449]:   Gateway: 192.168.0.1
[20:25:57][C][wifi:450]:   DNS1: 8.8.8.8
[20:25:57][C][wifi:451]:   DNS2: 8.8.4.4
[20:25:57][C][logger:185]: Logger:
[20:25:57][C][logger:186]:   Level: DEBUG
[20:25:57][C][logger:188]:   Log Baud Rate: 115200
[20:25:57][C][logger:189]:   Hardware UART: USB_CDC
[20:25:57][C][uart.arduino_esp32:151]: UART Bus 0:
[20:25:57][C][uart.arduino_esp32:153]:   RX Pin: GPIO20
[20:25:57][C][uart.arduino_esp32:155]:   RX Buffer Size: 256
[20:25:57][C][uart.arduino_esp32:157]:   Baud Rate: 4800 baud
[20:25:57][C][uart.arduino_esp32:158]:   Data Bits: 8
[20:25:57][C][uart.arduino_esp32:159]:   Parity: NONE
[20:25:57][C][uart.arduino_esp32:160]:   Stop bits: 1
[20:25:57][C][captive_portal:089]: Captive Portal:
[20:25:57][D][uart_debug:196]: <<< 0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 
[20:25:57][C][mdns:116]: mDNS:
[20:25:57][C][mdns:117]:   Hostname: apricus-hws
[20:25:57][C][esphome.ota:073]: Over-The-Air updates:
[20:25:57][C][esphome.ota:074]:   Address: apricus-hws.local:3232
[20:25:57][C][esphome.ota:075]:   Version: 2
[20:25:57][C][esphome.ota:078]:   Password configured
[20:25:57][C][safe_mode:018]: Safe Mode:
[20:25:57][C][safe_mode:020]:   Boot considered successful after 60 seconds
[20:25:57][C][safe_mode:021]:   Invoke after 10 boot attempts
[20:25:57][C][safe_mode:023]:   Remain in safe mode for 300 seconds
[20:25:57][C][api:140]: API Server:
[20:25:57][C][api:141]:   Address: apricus-hws.local:6053
[20:25:57][C][api:143]:   Using noise encryption: YES
[20:25:57][D][uart_debug:196]: <<< 0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 
[20:25:57][D][uart_debug:196]: <<< 0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000 (0xF0);0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000

This is the 0-7 bytes
0b00000000 (0x00);0b00010101 (0x15);0b00100001 (0x21);0b00110111 (0x37);0b01001101 (0x4D);0b01010110 (0x56);0b01100000 (0x60);0b11110000

for example bytes 0 and 1 give the binary number of 01010000 (add the last four bits of data together, second byte first) converted to decimal gives 80 minus 50 gives a temperature of 30degC. Through trial and error i found I had to minus 50 from the readings to get the temperature, I’m guessing it’s something to do with ADC.

I know it’s a lot but any help would be appreciated.

If I should start a new thread that’s fine.

I don’t know if this will help, but this setup shows some bitwise manipulation (although with ble data)

This thread has some more examples on parsing uart into sensors. But not necessarily binary processing.

I did this once but I’ll be buggered if I can remember how it works;) Maybe there is something around here that is of use. But maybe not.

Thanks guys that’s more information than I could find.

It looks to be very uncommon to be working with binary data.
This is all very new to me and I have extremely limited experience with the code, definitely have a lot to learn to figure this out.

Up until now I have found similar examples to what I need and have tweaked them to suit. Or in HA used the visual editor and worked things out.