Unable to get the value template correct for a tcp sensor

Configuration
1

Hello All,

I have created a tcp server to act as a tcp sensor. The server responds with the following json string when it receives a request

{ "ch1": { "watts": "1000"},"ch2": { "watts": "2000"},"ch3": { "watts": "3000"},"total": { "watts": 6000}}

when i use the following sensor setup, the developer tools window shows a state of unknown.

    name: House Power Consumption
    host: 192.168.150.32
    port: 10000
    timeout: 5
    payload: "GET\n"
    value_template:  '{{ value.total.watts }}'
    unit_of_measurement: Watt

but, if i hard code the response (as below) into the value template, i get the expected response of 6000

    name: House Power Consumption
    host: 192.168.150.32
    port: 10000
    timeout: 5
    payload: "GET\n"
    value_template:  '{% set my_test_json = {"ch1": { "watts": "1000"},"ch2": { "watts": "2000"},"ch3": {"watts": "3000"},"total": {"watts": "6000"},}%} {{ my_test_json.total.watts }}'
    unit_of_measurement: Watt

having tried numerous combinations in the value template, (with no success), I think what the problem is, is that ‘value’ is treated as a string when it is received from the server BUT if i hard code the value, it is treated as a JSON object.
Having come to that conclusion, i tried to convert the string to a json object using either ‘value_json’ which returns blank or ‘value|to_json’ which returns the string BUT with every inverted comma being delimited with a slash e.g." becomes " .

Can anyone see what i am doing wrong?
Thanks
Stuart
P.S. I have done a fair bit of searching on this topic and have found alot of solutions…but none of them are working in my case.

2

  • The variable called value refers to the data received by the sensor. The data is not assumed to be in any particular format.

  • The variable called value_json also refers to the data received by the sensor but is assumed to be in JSON format.

Example 1

If the received data is this:

0.75

and I use this template:

value_template: '{{ value * 1000 }}'

it will produce this:

750

Example 2

If the received data is this:

{ "power": 0.75 }

and I use this template:

value_template: '{{ value_json.power * 1000 }}'

it will produce this:

750

For your application, the template should use value_json like this:

    value_template:  '{{ value_json.total.watts }}'
3

thankyou for the explanation, it does clarify some things for me.
while i was in the process of responding to you to let you know that solution doesn’t work, i found it kind of does work…
I found that if i look at the sensors state in the developers tools, its reported state is “unknown” BUT if i then click on the ‘i’ next to the sensors name, it brings up a graph of the sensor over time and that graph IS showing the correct value
Any idea why that may be?

Thanks
Regards
Stuart

4

If the sensor’s current state is unknown that means whatever data it received (if any) could not be processed by the template. The graph only shows numeric values, not unknown values, so it may not necessarily be showing the latest value.

If you’re claiming that its latest state is unknown yet the graph is reporting its latest state as a numeric value, then I have no explanation for that because that’s not supposed to be possible.

5

Hello,
If you’re claiming that its latest state is unknown yet the graph is reporting its latest state as a numeric value, then I have no explanation for that because that’s not supposed to be possible.
I was, but i was wrong. It was actually reporting the only valid value it had received all day…which was a hard coded value.
After alot of checks, it ultimately comes down to this error

Unable to render template of Template("{{ value_json.ch2.watts | float }}") with value: '{"ch1": {"watts": 1000}, "ch2": {"watts": 2000}, "ch3": {"watts": 3000}, "total": {"watts": 6000}}'

if i copy and paste the values and template into the template editor, it works fine.
Does anything jump out at you as to being wrong?
Thanks for your help
Stuart

6

When we use value_json, we are indicating to Home Assistant that the data is a JSON string and Home Assistant should attempt to convert it into a JSON object. If successful, you will then be able to access the object’s values using a template like:

{{ value_json.ch2.watts | float }}

However, if Home Assistant is unable to convert it to a JSON object, then any template using value_json will fail.

The usual reason that prevents the transformation from JSON string to object is that the string contains one or more illegal unprintable characters. In other words, instead of the usual \n at the end of the string (an unprintable character normally signifying end-of-line) there’s some other character. In order to prove/disprove this, you would need to examine the received data in hex format.

Alternately, you can use regex_findall_index function to search through the received data and extract the desired value. This function works on any string so we reference the received data using value and not value_json. Try this template in the sensor:

    value_template:  >
      {{ value | regex_findall_index(find='\"ch2\": {\"watts\": (\d+)}') }}
7

Hello,
So, I tried the regex solution in the value template with no luck. So I tried it in the template tool with the following

{% set my_test_json = '{"ch1": { "watts": "1000"},"ch2": { "watts": "2000"},"ch3": {"watts": "3000"},"total": {"watts": 6000}}'%} 
{{ my_test_json | regex_findall_index(find='\"ch2\": {\"watts\": (\d+)}') }}

and i get the error “Unknown error rendering template”
Thanks for suggesting an alterative solution, but i would like to get the JSON parsing working.
Part of my concern/reasoning is, I can modify the server’s response to output straigh text e.g. 6000
so i don’t need to do any json parsing or regex…and this will solve my issue in this situation
BUT, what happens when i query a device that i am unable to modify the JSON response of (e.g. my air conditioner)
Some of the investigation I have done:
I have packet sniffed (using wireshark) the response from my server and i can confirm that the JSON string i am returning ends in ‘\n’ and i have tried it with and without a terminating null…with no success.
I have also modified the servers reponse down to an absolute minimum (the following)

'{ "watts": 1000 }\n'

and i can confirm that it is being received that way through the developers tools…but this fails to be passed as JSON too.
Although the following is about the template sensor, i wonder if it also applies to the tcp sensor

does-template-sensor-support-value-json
Regards
Stuart
P.S. Thanks for your help, I have been learning alot about HA by going through this process :slight_smile:

8

Given that you have complete control over the payload’s content, try sending a JSON string that is not terminated by a newline (or anything else for that matter). I’m curious to know if it makes any difference here.

As for the failed regex_findall_index, I won’t bother delving into what caused its failure (and how to correct it) because, in this case, you have control over the payload. However, know that in cases where Home Assistant cannot interpret the JSON string (and you don’t have control over the string’s content), the regex technique is the standard ‘Plan B’.

I believe I just learned the reason why the regex function failed and it is probably due to the way the backslashes are handled in the regex pattern. pnbruckner explains it here:

In a nutshell, there are two interpreters at work (YAML and Jinja2) and the first one (YAML) interprets the escape sequence when it is meant for use by the second one (Jinja2). Therefore you have to escape the escape (use double backslashes) so that its meaning is retained when evaluated by the Jinja2 interpreter.