Navien, ESP32 Navilink interface

the easiest option for me was having the bathroom light switch trigger the hot button.

So far this project has been very interesting. I am learning at every turn! This is great information! Thank You @suva! I took what you had, and tried with my limited python skills to pull it into python directly using a RS485/USB adapter. Also, built an ESPHome UART Sniffer as you describe above. In both cases - essentially, It reads the information you show above!! I am now trying to figure out how to manipulate the data to get the respective bits/bytes/strings out of this information.

This brings me to my next question - although you note, and I can see the f7 05 common data for all, the data streamed from the COM port starts long before I can see those values.

For example: The raw stream looks like this - (I **'d the f7 05 for clarity - it shows up 2x in this example.)

b'\xd8\xbc\x1a\x00\xb0\x02\x00\x00\x00\x00\xa6I\x00\x00\x00\x00\xb9\**xf7\x05PP**\x90"B\x00\x00\x05\x14x0*\x00\x00\x00\x00\x00\x00\x00\xaf\x00\x18\x00\x00\x00\x00\xb0\x02\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\xbb**\xf7\x05P**\x0f\x90*E\x00\x01\x01\x14\x07"\x00x0\x1b\x00\x00\x00\x00\x01\x00\x00\x9f\x1f\x00\x00\xd1\x03Z\n'

Interesting enough - that data stream also shows some weird values like the P or PP next to the 05 among other oddities. I have been able to convert it directly to a hex stream and received the following results which appear more accurate:

0xd8bc1a00b00200000000a64900000000b9f70550509022420000051478302a00000000000000af001800000000b00200000002000000000000bbf705500f902a450001011407220078301b000000000100009f1f0000d1035a0a

It seems like I should be able to slice the hex data stream better than the raw one as it appears more accurate? I am grateful for all the help!

Anyone on Next thoughts on python libraries, or ESPHome coding that helps break this down into a useable format?

I did find this post, but it doesnā€™t seem ā€œsolvedā€. It links to this post, which talks about custom devices in ESPHome, which currently appears past my abilities. I guess Iā€™m back to the reading and learning phase!

Ok - I worked my way through some really crude python and I have it displaying some information regarding the water, and printing to the REPL. Now that I can actually read it for what data it is supposed to be showing, It appears that I may have some differences from what @suva shows.

As it relates to index 18 & 19 (flow) on the water side. It seems that my water flow is actually index 18, as index 19 never changes for me. Index 19 is always zero. I confirmed that the front panel shows the value I am calculating on index 18 in the code, back to GPM.

Also, index 13 - that value seems strange to be inlet temperature. Guess I will have to keep watching and comparing that value to the front panel of the unit and see what it says.

I got my module today. Navilink Lite is using STM32F412 for MCU and MXCHIP EMC3280-E for WIFI.

Making some progress here! Again - thanks to @suva for all his decoding! I started with some generic python code to make sure I could actually talk/receive information from the device. The code I wrote to receive the information from my equipment is below. I am certain that itā€™s not efficient, and likely doesnā€™t follow too many rules for coding. If anyone has any tips here they are appreciated!

import time
import serial

print ("Opening port")
try:
  ser = serial.Serial('COM3',19200,bytesize=8,
                    parity=serial.PARITY_NONE,
                    stopbits=serial.STOPBITS_ONE,
                    timeout=0)
  print ("Port is open")

except serial.SerialException:
  serial.Serial('COM3', 19200).close()
  print ("Port is closed")
  serial_port = serial.Serial("COM3",19200)
  print ("Port is open again")

print ("Ready to use")

buffer = bytearray(b'')
buffer_length = 99
buf_list = []


while True:

  try:
    buffer += ser.read()
    if len(buffer) <= buffer_length:
      buffer += buffer
    elif len(buffer) >= buffer_length:
      buf_list = (list(buffer))
      # print(buf_list)
      ser.flushInput()
      buffer = bytearray(b'')
      # print(buf_list[0:23])
    if buf_list[5] == 34:
      # print("Water Info Received!")
      sys_power = buf_list[9]
      sys_power_text = "Off"
      set_temp = (((buf_list[11]/2)*9/5) + 32)
      outlet_temp = (((buf_list[12]/2)*9/5) + 32)
      inlet_temp = (((buf_list[13]/2)*9/5) + 32)
      wtr_flow = round(((buf_list[18]/10)* 0.2642),2)
      unknown_flow = buf_list[19]
      print(" ")
      print("*************Water Information**************")
      print(" ")
      if sys_power == 5:
        sys_power_text= "    On"
        print("The Power is:   ",sys_power_text)
      elif sys_power ==0:
        sys_power_text= "    Off"
        print("The Power is:   ",sys_power_text)
      else:
        print("The Power is Unknown")
      print("The Outlet Temp is: ",outlet_temp,"",u"\u00b0","F")
      print("The Setpoint is:    ", set_temp,u"\u00b0", "F")
      print("The Water flow is:  ",wtr_flow, "GPM")
      print("The Inlet? Temp is: ",inlet_temp,"",u"\u00b0", "F")
      print("The Unknown flow is:",unknown_flow, "  GPM")
    elif buf_list[5] == 42:
      # print("Gas Info Received!")
      print(" ")
      print("**************Gas Information***************")
      print(" ")
      total_gas_use = round(((buf_list[24]/10)*0.3508),2)
      low_byte_kcal = (buf_list[22])
      high_byte_kcal = (buf_list[23])
      total_kcal = high_byte_kcal + low_byte_kcal
      print("Total Gas Usage is: ", total_gas_use, "therm")
      print("Low Byte kCal is:   ", low_byte_kcal, "kCal")
      print("High Byte kCal is:  ", high_byte_kcal, "kCal")
      print("Total kCal:         ", total_kcal, "kCal")
    else:
      pass
      print("Bad Packet")
    time.sleep(1)
  except:
    pass

This was successful enough to produce the following output.

*************Water Information**************
 
The Power is:        On
The Outlet Temp is:  63.5  Ā° F
The Setpoint is:     140.0 Ā° F
The Water flow is:   0.0 GPM
The Inlet? Temp is:  68.9  Ā° F
The Unknown flow is: 0   GPM
 
**************Gas Information***************
 
Total Gas Usage is:  2.17 therm
Low Byte kCal is:    0 kCal
High Byte kCal is:   0 kCal
Total kCal:          0 kCal

So having this information I went towards my final goal of using ESPHome to create this similar interface. Below is the ESPHome Config.


# Include basic info in all ESP Home Configs
# Basic Info Includes
# logger:
# wifi:
# ap:
# web_server:
# captive_portal:
# button: to restart the device

<<: !include .base1.yaml


esphome:
  name: navien
  friendly_name: Navien Water Heater
  includes:
    - uart_read_line_sensor.h

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:
  level: NONE #makes uart stream available in esphome logstream
  baud_rate: 0 #disable logging over uart

# Enable Home Assistant API
api:
  encryption:
    key: "redacted"

ota:
  password: "redacted"

substitutions:
  comment: "redacted"
  friendly_name: Navien Water Heater

globals:
- id: byte_timer
  type: int
  initial_value: "1"

interval:
- interval: 5s
  then:
    - lambda: |-
        id(byte_timer) = (id(byte_timer) + 1);
        if (id(byte_timer) > 25) {
          id(byte_timer) = 1;
        }

# Example configuration entry
uart:
  id: uart_bus
  tx_pin: 1
  rx_pin: 3
  baud_rate: 19200
  data_bits: 8
  stop_bits: 1
  parity: NONE
  debug:
    direction: BOTH
    dummy_receiver: false
    after:
      delimiter: "\n"
    sequence:
      - lambda: |-
          UARTDebug::log_string(direction, bytes);
          //looking at this string f7 05 50 50 90 22 42
          //where the 22 represents water information to follow
          //and in the same byte if the number is 34 it is 
          //representative of gas information
          if (bytes[0] == 247 && bytes[5] == 34) {
          //id(uart_byte0).publish_state(bytes[0]);
          //id(uart_byte1).publish_state(bytes[1]);
          //id(uart_byte2).publish_state(bytes[2]);
          //id(uart_byte3).publish_state(bytes[3]);
          //id(uart_byte4).publish_state(bytes[4]);
          //id(uart_byte5).publish_state(bytes[5]);
          //id(uart_byte6).publish_state(bytes[6]);
          //id(uart_byte7).publish_state(bytes[7]);
          //id(uart_byte8).publish_state(bytes[8]);
          //id(uart_byte9).publish_state(bytes[9]);
          //id(uart_byte10).publish_state(bytes[10]);
          id(set_temp).publish_state(((bytes[11]/2)*9/5) + 32);
          id(outlet_temp).publish_state(((bytes[12]/2)*9/5) + 32);
          id(inlet_temp).publish_state(((bytes[13]/2)*9/5) + 32);
          //id(uart_byte14).publish_state(bytes[14]);
          //id(uart_byte15).publish_state(bytes[15]);
          //id(uart_byte16).publish_state(bytes[16]);
          //id(uart_byte17).publish_state(bytes[17]);
          id(water_flow).publish_state((bytes[18]/10)* 0.2642);
          //id(uart_byte19).publish_state(bytes[19]);
          //id(uart_byte20).publish_state(bytes[20]);
          //id(uart_byte21).publish_state(bytes[21]);
          //id(uart_byte22).publish_state(bytes[22]);
          //id(uart_byte23).publish_state(bytes[23]);
          //id(uart_byte24).publish_state(bytes[24]);
          } else if (bytes[0] == 247 && bytes[5] == 42) {
          id(low_byte_kcal).publish_state(bytes[22]);
          id(high_byte_kcal).publish_state(bytes[23]);
          id(total_gas_use).publish_state((bytes[24]/10)*0.3508);
          } else (bytes[0]  != 247); {
          }
# Example configuration entry
switch:
  - platform: template
    name: "HW Heater On/Off"
    id: navien_switch
    restore_mode: ALWAYS_ON
    lambda: |-
      if ((id(navien_switch).state) != (id(navien_switch).state)) {
        return true;
      } else {
        return false;
      }
    turn_on_action:
      - uart.write: !lambda 
          return {0xf7, 0x05, 0x0f, 0x50, 0x10, 0x0c, 0x4f, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xce};
          id(navien_switch).publish_state(true);
    turn_off_action:
      - uart.write: !lambda 
          return {0xf7, 0x05, 0x0f, 0x50, 0x10, 0x0c, 0x4f, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xa};
          id(navien_switch).publish_state(false);


text_sensor:
- platform: custom
  lambda: |-
    auto my_custom_sensor = new UartReadLineSensor(id(uart_bus));
    App.register_component(my_custom_sensor);
    return {my_custom_sensor};
  text_sensors:
    id: "uart_readline"

text:
  - platform: template
    name: "Temp Set - Write"
    optimistic: true
    min_length: 0
    max_length: 5
    mode: text
    on_value:
      then:
        - sensor.template.publish:
            id: num_from_text
            state: !lambda |-
              auto n = parse_number<float>(x);
              return n.has_value() ? n.value() : NAN;

sensor:
  # - platform: template
  #   name: "Byte 0"
  #   id: uart_byte0
  # - platform: template
  #   name: "Byte 1"
  #   id: uart_byte1
  # - platform: template
  #   name: "Byte 2"
  #   id: uart_byte2
  # - platform: template
  #   name: "Byte 3"
  #   id: uart_byte3
  # - platform: template
  #   name: "Byte 4"
  #   id: uart_byte4
  # - platform: template
  #   name: "Byte 5"
  #   id: uart_byte5
  # - platform: template
  #   name: "Byte 6"
  #   id: uart_byte6
  # - platform: template
  #   name: "Byte 7"
  #   id: uart_byte7
  # - platform: template
  #   name: "Byte 8"
  #   id: uart_byte8
  # - platform: template
  #   name: "Byte 9"
  #   id: uart_byte9
  # - platform: template
  #   name: "Byte 10"
  #   id: uart_byte10
  - platform: template
    name: "Temp Set - Read"
    id: set_temp
    unit_of_measurement: Ā°F
  - platform: template
    name: "Temp Outlet"
    id: outlet_temp
    unit_of_measurement: Ā°F
    accuracy_decimals: 2
  - platform: template
    name: "Temp Inlet"
    id: inlet_temp
    unit_of_measurement: Ā°F
  # - platform: template
  #   name: "Byte 14"
  #   id: uart_byte14
  # - platform: template
  #   name: "Byte 15"
  #   id: uart_byte15
  # - platform: template
  #   name: "Byte 16"
  #   id: uart_byte16
  # - platform: template
  #   name: "Byte 17"
  #   id: uart_byte17
  - platform: template
    name: "Water Flow"
    id: water_flow
    unit_of_measurement: GPM
  # - platform: template
  #   name: "Byte 19"
  #   id: uart_byte19
  # - platform: template
  #   name: "Byte 20"
  #   id: uart_byte20
  # - platform: template
  #   name: "Byte 21"
  #   id: uart_byte21
  # - platform: template
  #   name: "Byte 22"
  #   id: uart_byte22
  # - platform: template
  #   name: "Byte 23"
  #   id: uart_byte23
  # - platform: template
  #   name: "Byte 24"
  #   id: uart_byte24
  - platform: template
    name: "Total Gas Use"
    id: total_gas_use
    unit_of_measurement: therms
    accuracy_decimals: 2
  - platform: template
    name: "kCal Low Byte"
    id: low_byte_kcal
    unit_of_measurement: kCal
  - platform: template
    name: "kCal High Byte"
    id: high_byte_kcal
    unit_of_measurement: kCal
  - platform: template
    id: num_from_text
    name: "Temp Set - Write"
    internal: True

This seems to produce similar results as what I was seeing with the python code.

It looks like this:

Iā€™m struggling with a few things here that I could use some tips on.

  1. When using the switch I created in ESPHome to turn on/off the Heater, ideally Iā€™d like the switch to set itā€™s position based on the current status of the unit, then if the switch changes - send the appropriate command. Last, Iā€™d like to know that the command was received and that it has gone into that mode.

  2. Similar question regarding the setpoint. In my config, I created a second ā€œwriteā€ setpoint to try and begin configuring this function - but Iā€™m not sure where to go from here.

In both of those cases, Iā€™d like to have a single value/entity show on the ESPHome web page or entity in HA.

Any and all feedback is appreciated if you see something else here that would make this project better!

Hope this helps someone as they dive into their journey of doing the same!

3 Likes

Excitedly watching this from the sidelines! I have a NPE-240A2 that Iā€™d love to monitor and control via HASS, ideally without the dreadful NaviLink experiences detailed above (and elsewhere on the web).

Iā€™m a beginner with microcontrollers and ESPHome. Might poke around a bit with the above config and an 8266; still very much in the learning phase though.

@tsquared: Your code for uart_read_line_sensor.h appears to be python2. Curious if thereā€™s a reason for this, since both MicroPython and ESPHome support python3? Iā€™m happy to refactor it for python3 and implement a few best practices, unless thereā€™s a technical dependency on py2.

What interface did you use to communicate over rs485 with the Navien heater? I have an NPE240A and would love to ditch the Navilink in favor of a local ESP32 solution! Thank you so much for sharing your work!

Hello @brystmar! To be brutally honest, I am not a python or C programmer by any means. I have been using google, and just searching and patching lines of code together, and think I did ok to get here. I am confident there is a more elegant solution for all of this. :smile: So - certainly if you know of a better/different way - I am all ears!

As it relates to the 8266 and doing this - there was a reason I used the ESP32, and I think it had to do with the number of UARTā€™s on it, but I donā€™t specifically recall what I read that pointed me in that direction.

To answer your question regarding the uart_readline_sensor.h:

Not knowing how to program in C, or utilize the lambdas well enough to get the information out of the ESP32 that I have connected to the Navien, I utilized the Custom UART Text sensor listed here. Then, I utilized pieces of the code found in this thread. And coupled it with information found in this thread to get as far as I have. Again, Certainly not elegant, or likely efficient - but none-the-less, a semi-working copy!

You are more than welcome to refactor, or help with further development on this as I am sure we could all enjoy not having to use the Navilink Solution. Please let me know if I can provide more information.

@aruffell Currently, I have the following hooked up, and hoping to streamline it a bit further shortly.

From the 5 Pin Connector @ Navien to RJ-45 Connector - I purchased this

Once at the RJ45, I used a breakout connector that I purchased here

Then I wired the output of that breakout to the input of this

Then hooked the other end of that device, to the ESP32.

Iā€™d have to get back to you on the wiring between all of this as I am not in front of those devices currently.

Let me know if you need more!

Ah, I misread your earlier post. Sounds like that python code was merely a proof of concept before attempting the ESPHome solution, so thereā€™s no need for a refactor. Iā€™m sure someone with more experience in the ESPHome microcontroller world can help optimize that config.

We can definitely simplify things on the hardware side. No need for that proprietary Navien cable or any of the RJ45 adapters either. A properly sized 5-pin molex connector + some jumpers from your local hobby store should do the trick. The tricky part is identifying the exact shape & size of that connector. These types of connectors are typically sold in bulk for pennies apiece. Another task for someone with more experience in this space :slight_smile:

Have you considered starting a dedicated thread in the ESPHome category? Youā€™re already most of the way there, just need some tweaking of the existing solution from someone with the right experience.

I have that cable and can take a look if I recognise the connector. I have a few connector kits I use to crimp my own cables so with some luckā€¦

As for yhe rs485 adapter I see it coverts to 5V but that is not ideal for the esp32 as it is 3.3v, and only 5v tolerant. I did a quick search and saw some converters that have an rj45 connector on them or the space to solder one to simplify connectionā€¦ however if we make our own cable, we can skip the rj45 connectors altogether as you suggested.

I was going to ask how @tsquared was powering his esp32. We are targeting a self-powered solution (i.e.: drawing power from the Navienā€™s board), even if it needs to be stepped down to 3.3v, yes?

@brystmar - Correct, the Python was certainly my proof of concept! :crossed_fingers:

You make a good point about making a dedicated thread in the ESPHome category. Iā€™ll have to check that out. Not sure if I link back to this, or create a new one?

@brystmar Correct, self powered is the ultimate goal here. I know that on the 5 pin connector @ the Navien board, there are only 4 wires. RS-485 RX/TX and also 12VDC & GND. Iā€™m thinking about powering it from the 12 VDC and using one of these to step the voltage down to power the ESP32 & the RS-485 board.

I have included some pictures of the 5 pin connector if that helps anyone trying to cross reference it.

image

image

I have included a picture of my RJ-45 breakout for the pinout. The 4 wires I have labeled here are the only 4 in use. I left the rest hooked up as they were all attached to the ribbon like cable and didnā€™t see the need to tear it apart.

@aruffell You mention something about a RS-485 converter board with RJ-45 connectors? Would you have a link for this? Also, as it relates to the 5V/3.3V on the current converter board, it seems to be working as the current solution. I am not sure which voltage you are concerned about. Though I havenā€™t measured it, the secondary side of that board, is directly connected to one of the ESP32 UARTā€™s and it seems to be handling it ok.

Anyway - hope this helps!

Cheers!

@tsquared I would ask a mod to most post x (fill in) down to the last into a new thread in the esphome section.

The GPIO of the esp32 is 3.3V and only 5V tolerant so whole it works it would be better to use a rs485 to 3.3V board. I will look for one and report back. I will also dig into digikey / mouser to see if I can pinpoint that connector. Do you see any numbers or letters marked on the connector?

Edit: Could it be this one - VHR-5N JST Sales America Inc. | Connectors, Interconnects | DigiKey

Apologies in advance, I donā€™t know how/who to ask to move the topic, and unfortunately created a post in the ESPHome category before I realized that @aruffell posted that this could be moved. Do you just @ a moderator and they should see it?

If it does move, it probably should begin starting with the post below.

https://community.home-assistant.io/t/navien-hot-water-heater-navilink/330044/87?u=tsquared
The ESPHome category post is here for combining/changing etc.

Please let me know if I need to do something else.

Thank you all for your help here! Certainly appreciated!

Also, @aruffell Awesome find here! The connector you show certainly looks like the one I have. Do they usually come with any of the internal pins that you would need to connect the wires? Or can you get them with wires attached.

Could you post pinout of that connector? Which one is 12VDC, which are RS485 and ground pins.

I am thinking of using this RS485 base and add this controller to it. It seems that in that case it can directly power controller from 12VDC input of RS485 and looks neat. They also have RS485 in other forms for other controllers, also with 12-24VDC input to power controller.

Connector does not look like JST VH one. Here is the spec for VH connector. Note how bumps are on the outside edge while Navien connector has bumps little closer to the center of connector.

I realize now what you were saying about the 5vdc scenario. I remembered, I purchased this adapter instead. This is actually what is in the circuit now.

image

The link to this device is here.

I will update the original post to reflect this difference.

1 Like

Here is the pinout I have.

image

1 Like

@kkopachev - true, not the one. How about the JST XHB?

image

I canā€™t find dimensional drawings, but this picture looks really similar. I need to measure the pitch of the actual connectorā€¦ that would help narrow down the search.

1 Like