Template variable error: 'value_json' is undefined - mqtt

to clarify, this is your data

{
	"time": "<#timehhmmss>",
	"press": < #press rc = y > ,
	"temp": < #temp rc = y > ,
	"tempin": < #intemp rc = y > ,
	"tempavg24": < #avgtemp rc = y > ,
	"apptemp": < #apptemp rc = y > ,
	"tempchangeh": < #TempChangeLastHour rc = y > ,
	"raintoday": < #rfall rc = y > ,
	"rainmonth": < #rmonth rc = y > ,
	"rainratetoday": < #rrate rc = y > ,
	"humidity": < #hum > ,
	"humidityin": < #inhum > ,
	"avgbearing": < #avgbearing > ,
	"windavg10m": < #wspeed rc = y > ,
	"wgust10": < #wgust rc = y >
}

This is what is invalid
image

Notice how the time’s value is in quotes? "<#timehhmmss>"

Every other value needs to be in quotes in order for that to be valid json.

My guess it’s that is some metadata used to configure whatever tool.
The screenshot seems to show valid JSON, but ofc we don’t know unless the OP post a complete payload in text (not image) format.

EDIT: Actually, a random JSON validator tells me numbers like +0.3, with the “+”, is not valid JSON

The way he posted the data is confusing. what he posted is valid json, however the topics contents is not. Just take what’s in the data field (remove the exterior quotes) and run it through a json checker. It’s littered with errors.

Yep, I see that too. TIL. I guess it makes sense though.

1 Like

…and as HA cannot make up JSON from the payload, value_json is not created, leading to 'value_json' is undefined.
QED :wink:

It’s interesting, the company I work for purposely outputs +'s on positive numbers. However we are doing that on human readable text files. (It’s a readability thing to reduce mistakes). I would have never thought that was invalid json. Then again, I always have json libraries handle the creation of the numerical format when serializing.

I had already encountered that kind of semi-weird 'non valid" error on numbers with explicit “+” signs (not sure it was JSON), so that made me test it.

Well, the good news is that if OP wraps all those values in strings, home assistant will convert them to numbers.

Thank you all for taking your time!
The sensors has worked for almost a year with “<#timehhmmss>” , but started to drop out recently, random more or less. Yesterday it worked overnight and stopped in the morning.
I use it for my my Davis Vantage Pro 2+ weather station, connected through Cumulus MX software. The template mqtt that shows an example on how to properly config Cumulus MX to send JSON looks like this:

{"topics":[
    {
        "topic": "CumulusMX/Interval",
        "data": "{\"time\":\"<#timehhmmss>\",\"temp\":<#temp rc=y>,\"humidity\":<#hum>,\"wgust\":<#wgust rc=y>}",
        "retain": false
    }
]}

I have set the retain flag to “false” for now, published some empty messages, and for the moment my sensors are receiving data from this below:

{"topics":[
    {
        "topic": "CumulusMX/IntervalUpdate",
        "data": "{\"time\":\"<#timehhmmss>\",\"press\":<#press rc=y>,\"temp\":<#temp rc=y>,\"tempavg24\":<#avgtemp rc=y>,\"apptemp\":<#apptemp rc=y>,\"tempin\":<#intemp rc=y>,\"humidity\":<#hum>,\"humidityin\":<#inhum>,\"raintoday\":<#rfall rc=y>,\"rainmonth\":<#rmonth rc=y>,\"avgbearing\":<#avgbearing>,\"wgust10\":<#wgust rc=y>,\"windavg10m\":<#wspeed rc=y>}",
        "retain": false
    }
]}

I’m not sure if I need the timestamp, because I am just interested in the weather data.

See above. At least from the image of the payload you showed, the templates will only work as long as tempchangeh decreases. Wrap that one (you can do all actually) between double quotes.

1 Like

I’m having a similar issue I think, but I don’t understand from the comments here what the actual solution was in this case - and if I then can use that.

in my case I have simple dht22 sensor, that I have sending both

$ paho_c_sub -h 192.168.0.4 -t neo2/dht22
{"humidity":"26.420000","temperature":"21.300000"}

and

$ paho_c_sub -h 192.168.0.4 -t neo2/dht22
{"humidity":26.4,"temperature":21.3}

and still in both cases I get:
ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.temperature }}'

from my configuration.yaml

mqtt:
  sensor:
    - name: "DHT22 Temperature"
      state_topic: "neo2/dht22"
      unit_of_measurement: "°C"
      value_template: "{{ value_json.temperature }}"

looking at this: MQTT Sensor - Home Assistant

At least one of those should work, I would have thought — provided your MQTT setup is working. To check, please remove the value_template line altogether, and then share a screenshot of the Developer Tools / States page for the sensor once it has updated. Like this, but for your sensor:

Right now on mobile, but in home assistant log I see:

2023-02-14 14:19:34.773 WARNING (MainThread) [homeassistant.components.sensor] Sensor sensor.dht22_temperature has device class None, state class None and unit °C thus indicating it has a numeric value; however, it has the non-numeric value: {"humidity":26.51,"temperature":21.24} (<class 'str'>); Please update your configuration if your entity is manually configured, otherwise create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22

And

It looks as though it may be having trouble interpreting the input as valid JSON. Try this:

value_template: "{{ (value|from_json)['temperature'] }}"

Now I got:

2023-02-14 14:48:58.831 ERROR (MainThread) [homeassistant.util.logging] Exception in message_received when handling msg on 'neo2/dht22': '{"humidity":26.5,"temperature":21.23}'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/debug_info.py", line 44, in wrapper
    msg_callback(msg)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/sensor.py", line 307, in message_received    _update_state(msg)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/sensor.py", line 267, in _update_state
    payload = self._template(msg.payload, PayloadSentinel.DEFAULT)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/models.py", line 249, in async_render_with_possible_json_value
    rendered_payload = self._value_template.async_render_with_possible_json_value(
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 624, in async_render_with_possible_json_value
    return _render_with_context(self.template, compiled, **variables).strip()
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1984, in _render_with_context
    return template.render(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1889, in from_json
    return json_loads(value)
orjson.JSONDecodeError: unexpected content after document: line 1 column 38 (char 37)

So it really miss something in that json string, but I don’t see something wrong.

Bizarre. And that’s using value not value_json as per my prior post?

Yep

mqtt:
  sensor:
    - name: "DHT22 Temperature"
      state_topic: "neo2/dht22"
      unit_of_measurement: "°C"
      value_template: "{{ (value|from_json)['temperature'] }}"

You’re sending a character that you shouldn’t be sending at the end of your response. That’s 37 characters long, your error is indicating it’s failing at the 38th character. The encoding that you’re sending is most likely off, make sure you use ascii or utf-8 or utf-16 and strip all whitespace before dropping it in the topic.

1 Like

Yes, you are right. Now made a minimal test-case from the cpp code to a python mqtt client, that failed in a similar way. Problem was this cpp code, that I made from borrowed code on std::string to char. And that extra byte should apparently NOT be there - weird.

                const auto s = j.dump(); // serialize to std::string
                char msgToPublish[s.length() + 1];
                strcpy(msgToPublish, s.c_str());
                printf("%s", msgToPublish);
                pubmsg.payload = msgToPublish;

But now it works with:

                const auto s = j.dump(); // serialize to std::string
                char msgToPublish[s.length()];
                strcpy(msgToPublish, s.c_str());
                printf("%s", msgToPublish);
                pubmsg.payload = msgToPublish;
2 Likes

This code is very very dangerous. Don’t do that.

char msgToPublish[s.length()]; // <= This is on stack
...
pubmsg.payload = msgToPublish; // <= You are setting the payload pointer to a stack variable that'll be cleared when leaving the function, thus payload will be a dangling pointer  

The former code was better since you need to account for the zero byte in the end, but you need to initialize it. A correct code should have read:

const auto s = j.dump(); // serialize to std::string
pubmsg.payload = strdup(s.c_str()); // Make a copy on heap, including the zero byte end

// Don't forget to free the payload after publishing to avoid memory leak:
free(pubmsg.payload);