Convert this data to an understandable format

Third party integrations
1

I am now out of my depth, and I am callng on the geniuses that hang out around this forum :wink:

I have installed a breaker, with power & energy metering. The breaker seem to lose connection to Wifi every now and again, and even though according to my network, it is still active, there is no comms from this device, and even the Smartlife app says it is offline.

Strangely, though, I have been able to add parts of it to Local Tuya, allowing me to switch the device off and on, even though everything else says it is offline.

To get back to point, the breaker is a three phase breaker, with the phases being called phase a, phase b and phase c.

I am OK in keeping this device local, but have run into a problem I just cannot solve, not for lack of trying, I think my brain is just too small…

Typically on these devices, you have DP_IDs for

  • Power
  • Current
  • Voltage

They might require a multiplier to make sense of the value. For instance, the power might read 205, which need to be multiplied by 0.1 to get the accurate reading of 20.5W. This is undertandable, and I am manging just fine setting it up.

However, this breaker have a very stange method that it employs related to the data. So, Phase A has a single DPID, containing a text string. Apparently doing some Endian Big End (or something) magic, this text string contains the power, current and voltage.

Herewith an extract from Tuya IOT Data model for this device (Translated - as it is mostly Chinese). Relevant section underlined.

I am also posting a value here that I can see on Tuya IOT, and the readings I get from Smartlife.
Tuya IOT - Phase A:
{
“code”: “phase_a”,
“custom_name”: “”,
“dp_id”: 6,
“time”: 1716297638366,
“value”: “CWgABtwAABE=”
},

Smartlife - that is about 1.7A, 18W and 240.8V. It changes quickly, so that is why I am calling it “about”.

“model”: “{"modelId":"dzx2ek","services":[{"actions":,"code":"","description":"","events":,"name":"默认服务", "properties":[{"abilityId”:1,"accessMode":"ro","code":"total_forward_energy","description":"","extensions":{" iconName":"icon-dp_lightning","attribute":"1152"},"name":"Positive Total Active Power","typeSpec":{"type":"value","max":999999999," min":0,"scale":2,"step":1,"unit":"kW-h"}}},{"abilityId":6,"accessMode":"ro","code":"phase_a","description":"1 Phase A voltage, current and power\\n2, big-end mode, HEX format, 8 bytes in total\n3, unit accuracy: voltage, 2 bytes, unit 0.1V. current, 3 bytes, unit 0.001A. phase A active power, 3 bytes, unit 0.0001kW\n4, message format\n4, example: 08 80 00 03 E8 00 27 10 means phase A 217.6V, A-phase current 1.000A, A-phase power 10.000KW\n5, communication logic: \\n1) user into the panel, active query. The user enters the panel, the panel immediately sends down 0x08 to the meter, the meter reports the data to the cloud platform, the panel gets the data and then displays it. Note: Latest WIFI support. \\n2) The meter reports data based on a certain period. Suggestion: In WIFI mode, report once every 15 seconds. in NB mode, report once every 1 hour.

As such, how do I convert the Phase A: CWgABtwAABE= value to understandable power, current and voltage?

Like I said, I just cannot make progress here, not for lack of trying.

2

Seems pretty straightforwards :crazy_face:. Create a reusable template file:

/config/custom_templates/smartlife.jinja

with this content:

{% macro get_data(s) -%}
{% set ns = namespace(s="") -%}
{% for c in s|map('ord')|reject('==',61) -%}
{% set ns.s = ns.s+"{:0=6b}".format(c+{1:-71,c<91:-65,c<58:4,c<48:16,c<44:19}[true]) -%}
{% endfor -%}
{{ {"voltage": ns.s[0:16] |int(base=2) / 10,
    "current": ns.s[16:40]|int(base=2) / 1000,
    "power":   ns.s[40:64]|int(base=2)}|to_json -}}
{% endmacro -%}

Then either restart HA or call the homeassistant.reload_custom_templates service from Developer Tools / Services.

Now set up template sensors in the UI with the following state templates, and the appropriate device classes (voltage, current, power). In each case, replace sensor.smartlife_value with whatever entity holds the code e.g. CWgABtwAABE=

{% from 'smartlife.jinja' import get_data %}
{{ (get_data(states('sensor.smartlife_value'))|from_json)['voltage'] }}

{% from 'smartlife.jinja' import get_data %}
{{ (get_data(states('sensor.smartlife_value'))|from_json)['current'] }}

{% from 'smartlife.jinja' import get_data %}
{{ (get_data(states('sensor.smartlife_value'))|from_json)['power'] }}

Test:

image
image769×329 26 KB

I joke about the “straightforwards” of course. A couple of key references needed for this:

Disclaimer: give this a really good test before using it for anything critical. It’s all been thrown together quite quickly. Might be some b64 padding issues lurking.

3

Troon for Galactic Emperor…
Amazing and truly a genius at work. Wow.

It would have take me at least 64 days to the power of n to get even close to this. Definitely not straight forward.

I will try this tomorrow (when I have readings on that device again). Thank you.

1 Like
4

Made a rookie string slicing error in the original, now corrected above. If you’ve already made the template file, please re-do it.

The end point of the slice is exclusive:

image
image769×329 21.6 KB

I feel like I should probably explain what that macro is doing, as some of it may look a little arcane.

The input string is base64-encoded binary. Each letter corresponds to a 6-bit binary number as summarised here.

{% for c in s|map('ord')|reject('==',61) %}

My for loop converts the input string to a list of character codes (e.g. A=65) using ord (documented here), and rejects the = sign (code 61).

{% set ns.s = ns.s+"{:0=6b}".format(c+{1:-71,c<91:-65,c<58:4,c<48:16,c<44:19}[true]) %}

The next line builds up a binary string from those codes. "{:0=6b}".format(x) outputs x as a 6-bit-padded binary string.

The argument to the format string is the complex part, converting the character code to the position in the RFC. It contains a dictionary with a true / false condition as the key and a number as the value:

c + {    1: -71,          # maps a(97) to z(122) to 26-51
      c<91: -65,          # maps A(65) to Z(90)  to  0-25
      c<58:   4,          # maps 0(48) to 9(57)  to 52-61
      c<48:  16,          # maps /(47) to 63
      c<44:  19}[true]    # maps +(43) to 62

For the first key: 1 is equivalent to true, and works as a default in what is essentially an upside-down if-elif-else chain. For example, if the character is K, with code 75, that dictionary is:

{  true: -71, 
   true: -65,
  false:   4,
  false:  16,
  false:  19 }

which collapses to {true: -65, false: 19} (last key takes precedence), and I look up the true value (-65). So the argument to format is 75 + -65, which is 10, the position in the RFC table.

So at the end of the for loop we have a 66-bit (11 characters times 6 bits) binary number, the base64 decoding of the input string with the last two bits unused here.

The final output splits this binary string as described in both the first post and the Tuya GitHub link above, converting to decimal and scaling to the correct units. I’m mildly concerned that the power might be out by a factor of ten as the original post claims units of “0.0001kW” which is 0.1W; the GitHub link suggests the unit is W which is what I’ve implemented. The example numbers don’t seem to make much sense as 240.8V times 1.756A is about 423W unless this is some sort of reactive / power factor thing. Ties in with the “about 18W” from the app though.

This jerry-rigged base64 decoder is not at all tolerant of invalid input and will likely burst into flames if the input is anything other than 11 valid characters.

2 Likes