Gathering data from different sensors

[tom_l]

Edit: there’s an error in my second value_template, the leading double quote should be a single quote. Also…

Ugh. States are always stored as strings. As the serial sensor is not capable of storing the received json in an attribute (which can be any type, not just a string like states) we need to convert it back to json. Try this

  - platform: template
    sensors:
      tmp102_temperature:
        friendly_name: TMP102 Temperature
        unit_of_measurement: "°C"
        value_template: '{{ (states("sensor.serial_sensor")|from_json)["5C:CF:7F:8E:32:E4"]["Temperature"] }}'
      tmp102_voltage:
        friendly_name: TMP102 Voltage
        unit_of_measurement: "V"
        value_template: '{{ (states("sensor.serial_sensor")|from_json)["5C:CF:7F:8E:32:E4"]["Voltage"] }}'

We may still have an issue when the serial sensor state does not contain the payload we are looking for. But let’s get this bit working first then trap that error later.

[petro]

Ok, I’m sure there’s a better way, but I’d go about this with the new template integration and the shortest payload possible.

Change your payload back to this:

{
  "MAC": "5C:CF:7F:8E:32:E4",
  "Sensor": "TMP102",
  "Temperature": 24.06,
  "Voltage": 3.21
}

Then make your sensor

sensor:
  - platform: serial
    serial_port: /dev/ttyUSB0
    baudrate: 9600

Then your new template

template
  - trigger:
      - platform: template
        value_template: >
          {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:32:E4' }}
    sensor:
      - unique_id: tmp102_temperature
        name: TMP102 Temperature
        unit_of_measurement: "°C"
        state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
      - unique_id: tmp102_voltage:
        name: TMP102 Voltage
        unit_of_measurement: "V"
        state: "{{ (states("sensor.serial_sensor")| from_json).Voltage }}'
  - trigger:
      - platform: template
        value_template: >
          {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:31:E1' }}
    sensor:
      - unique_id: bme280_temperature:
        name: TMP102 Temperature
        unit_of_measurement: "°C"
        state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
      - unique_id: bme280_voltage:
        name: TMP102 Voltage
        unit_of_measurement: "V"
        state: "{{ (states("sensor.serial_sensor")| from_json).Voltage }}'

But you’ll have to send a payload of {} between each send because 2 payloads in a row with an identical mac will not trigger an update.

[ChrisKolan (Chris)]

I’m beginning to wonder if home assistant receives data through the serial port at all. On my raspberry pi I run home assistant in docker container. Do I need to somehow activate the serial port when home assistant runs in docker? I receive data in correct format (checked with minicom) when I connect directly to my raspberry pi via SSH.

Now I see these errors:

Logger: homeassistant.helpers.event
Source: helpers/template.py:391
First occurred: 8:11:04 PM (2 occurrences)
Last logged: 8:11:04 PM

Error while processing template: Template("{{ (states(“sensor.serial_sensor”)|from_json)[“5C:CF:7F:8E:32:E4”][“Temperature”] }}")
Error while processing template: Template("{{ (states(“sensor.serial_sensor”)|from_json)[“5C:CF:7F:8E:32:E4”][“Voltage”] }}")
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/helpers/template.py”, line 389, in async_render
render_result = _render_with_context(self.template, compiled, **kwargs)
File “/usr/src/homeassistant/homeassistant/helpers/template.py”, line 1358, in _render_with_context
return template.render(**kwargs)
File “/usr/local/lib/python3.8/site-packages/jinja2/environment.py”, line 1304, in render
self.environment.handle_exception()
File “/usr/local/lib/python3.8/site-packages/jinja2/environment.py”, line 925, in handle_exception
raise rewrite_traceback_stack(source=source)
File “”, line 1, in top-level template code
File “/usr/src/homeassistant/homeassistant/helpers/template.py”, line 1310, in from_json
return json.loads(value)
File “/usr/local/lib/python3.8/json/init.py”, line 357, in loads
return _default_decoder.decode(s)
File “/usr/local/lib/python3.8/json/decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/local/lib/python3.8/json/decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/helpers/template.py”, line 505, in async_render_to_info
render_info._result = self.async_render(variables, strict=strict, **kwargs)
File “/usr/src/homeassistant/homeassistant/helpers/template.py”, line 391, in async_render
raise TemplateError(err) from err
homeassistant.exceptions.TemplateError: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Logger: homeassistant.components.template.template_entity
Source: components/template/template_entity.py:72
Integration: Template (documentation, issues)
First occurred: 8:11:04 PM (2 occurrences)
Last logged: 8:11:04 PM

• TemplateError(‘JSONDecodeError: Expecting value: line 1 column 1 (char 0)’) while processing template ‘Template("{{ (states(“sensor.serial_sensor”)|from_json)[“5C:CF:7F:8E:32:E4”][“Temperature”] }}")’ for attribute ‘_state’ in entity ‘sensor.tmp102_temperature’
• TemplateError(‘JSONDecodeError: Expecting value: line 1 column 1 (char 0)’) while processing template ‘Template("{{ (states(“sensor.serial_sensor”)|from_json)[“5C:CF:7F:8E:32:E4”][“Voltage”] }}")’ for attribute ‘_state’ in entity ‘sensor.tmp102_voltage’

[tom_l]

Change your message format back and try Petro’s solution. But before making the template sensors just make the serial sensor and check that its state changes when you send a message. You can check its state in the Developer Tools / States menu.

[ChrisKolan (Chris)]

I am able to receive the data without template sensors. I will try @petro solution.

[ChrisKolan (Chris)]

Now I got:

• Invalid config for [template]: [tmp102_temperature] is an invalid option for [template]. Check: template->sensor->0->tmp102_temperature. (See /config/configuration.yaml, line 203).
• Invalid config for [template]: [bme280_temperature] is an invalid option for [template]. Check: template->sensor->0->bme280_temperature. (See /config/configuration.yaml, line 216).

[petro]

You should check out the docs for the template section. I adjusted (edited) my example and it should be correct now. I shouldn’t have copied and pasted. Either way, formatting issues will be highlighted by looking at the template examples in the docs.

[ChrisKolan (Chris)]

I am still not sure if I put the code in right place. Right now I put it into my /config/sensors.yaml file. In order to save the file I had to remove the “template”. So my file looks like this:

- platform: serial
  serial_port: /dev/ttyAMA0
  baudrate: 9600

- trigger:
    - platform: template
      value_template: >
        {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:32:E4' }}
  sensor:
    - unique_id: tmp102_temperature
      name: TMP102 Temperature
      unit_of_measurement: "°C"
      state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
    - unique_id: tmp102_voltage
      name: TMP102 Voltage
      unit_of_measurement: "V"
      state: '{{ (states("sensor.serial_sensor")| from_json).Voltage }}'
- trigger:
    - platform: template
      value_template: >
        {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:31:E1' }}
  sensor:
    - unique_id: bme280_temperature
      name: BME280 Temperature
      unit_of_measurement: "°C"
      state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
    - unique_id: bme280_voltage
      name: BME280 Voltage
      unit_of_measurement: "V"
      state: '{{ (states("sensor.serial_sensor")| from_json).Voltage }}'

But when I check the configuration I get the following error:

Configuration invalid

Invalid config for [sensor]: required key not provided @ data[‘platform’]. Got None. (See /config/configuration.yaml, line 12). Invalid config for [sensor]: required key not provided @ data[‘platform’]. Got None. (See /config/configuration.yaml, line 12).

[123 (Taras)]

You’re using petro’s example which is a Trigger-based Template Sensor and must be defined under the template domain (not the sensor domain). You have added it to yoursensor.yaml file which means you have put it under the sensor domain (which is invalid).

Remove the Trigger-based Template Sensors from that file and add them to your configuration.yaml file under the template domain.

template:
- trigger:
    - platform: template
      value_template: >
        {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:32:E4' }}
  sensor:
    - unique_id: tmp102_temperature
      name: TMP102 Temperature
      unit_of_measurement: "°C"
      state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
    - unique_id: tmp102_voltage
      name: TMP102 Voltage
      unit_of_measurement: "V"
      state: '{{ (states("sensor.serial_sensor")| from_json).Voltage }}'
- trigger:
    - platform: template
      value_template: >
        {{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:31:E1' }}
  sensor:
    - unique_id: bme280_temperature
      name: BME280 Temperature
      unit_of_measurement: "°C"
      state: '{{ (states("sensor.serial_sensor")| from_json).Temperature }}'
    - unique_id: bme280_voltage
      name: BME280 Voltage
      unit_of_measurement: "V"
      state: '{{ (states("sensor.serial_sensor")| from_json).Voltage }}'

Alternately, you can add them to a new templates.yaml file and then add the following to your configuration.yaml file:

template: !include templates.yaml

[ChrisKolan (Chris)]

Thanks @123 I moved template into separate templates.yaml file and included it in configuration.yaml. The configuration is again valid but I still get errors:

Error while processing template: Template("{{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:32:E4' }}")
Error while processing template: Template("{{ (states('sensor.serial_sensor') | from_json).MAC | default == '5C:CF:7F:8E:31:E1' }}")
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 389, in async_render
    render_result = _render_with_context(self.template, compiled, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1358, in _render_with_context
    return template.render(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1310, in from_json
    return json.loads(value)
  File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 505, in async_render_to_info
    render_info._result = self.async_render(variables, strict=strict, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 391, in async_render
    raise TemplateError(err) from err
homeassistant.exceptions.TemplateError: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[123 (Taras)]

Seems like the error message is complaining about not getting a value where it was expecting one.

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Just a guess but if the Serial Sensor’s value fails to be converted into JSON then it might generate that error (for example, if the sensor has no value or a non-JSON value).

[tom_l]

Possibly due to this?

@petro

But you’ll have to send a payload of {} between each send because 2 payloads in a row with an identical mac will not trigger an update.

Sent this instead?

{ "MAC": "0" }

[ChrisKolan (Chris)]

I carried out the following test. I disconnected the microcontroller sending data through the serial port and restarted home assistant. In the log the same error appeared as displayed above. I then connected the microcontroller and started sending data and at this time started home assistant. The same errors and new warning appeared:

Error evaluating 'template' trigger for 'Trigger Update Coordinator': JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It looks like home assistant is not refreshing the serial port when new data arrives. The port is pulled on startup, when there is no data yet (then errors occur), and then when data arrives it is not refreshed. This is also confirmed by the fact that I have to press the refresh button to get data in Developer Tools → states

[ChrisKolan (Chris)]

It appears that refreshing the serial sensor is a known problem:

Using serial sensor for reading JSON output from Arduino?

Hello, I’m trying to setup serial sensor based on HASS documentation: So… I wrote sketch on my arduino delivering json results (according to documentation on “sensor.serial”) local picocom returning following results: $ picocom -b 115200 /dev/ttyUSB0 ... {"channel_1":1,"temperature_1":15.6,"humidity_1":50,"battery_1":90} I setup serial sensor on… hass side: sensor: - platform: serial baudrate: 115200 serial_port: /dev/ttyUSB0 It looks like that hass launching serial sensor co…

Serial sensor doesn't work anymore

Hi, I use the serial sensor input to read mine Arduino one in JSON style. But there is noting changed and it doesn’t work anymore. als not after reboot from hassio or the Arduino or reconnect the USB. Can anyone give me a reason why und maybe a solution? the log say’s hassio find the Arduino one. 21-05-29 20:46:41 INFO (MainThread) [supervisor.hardware.monitor] Detecting HardwareAction.ADD usb hardware /dev/bus/usb/001/009 21-05-29 20:46:41 INFO (MainThread) [supervisor.hardware.monitor] De…

readline() will not read serial data without newline ending

[petro]

Then alter the template trigger to check for data before serializing

[ChrisKolan (Chris)]

I’m not sure that changing the trigger will solve the problem. It seems to me that the problem is with the serial port refreshing. I added new line after sending the data as suggested by @macle here:

Using serial sensor for reading JSON output from Arduino?

I found the problem! I answer myself… maybe it helps someone. The problem was in… lack of “\n” and the end of each line received over serial (readline expects that!). In my case my arduino sketch sent just json’s lines…without proper line ending. So I added: Serial.println(); after sending my json string in my sketch and… it resolved the problem. @petro: thank you! you helped me a lot. Sometimes just pointing at some direction results in different perspective of the problem.

But it does not work for me. This not working behavior is also confirmed by @Tom92. I have also tried many different combinations, none of which refresh the data:

Serial2.println((const char*)data);

Serial2.println((const char*)data);
Serial2.println();

Serial2.print((const char*)data);
Serial2.println();

Serial2.print((const char*)data);
Serial2.print('\n');

Serial2.print((const char*)data);
Serial2.print('\r');

Serial2.print((const char*)data);
Serial2.print("\r\n");

Serial2.print((const char*)data);
Serial2.print("\n");

[petro]

Are you recompiling your code?

[ChrisKolan (Chris)]

Yes, I am recompiling the code. Actually, the first approach should work without adding the second line with print.

@ChrisKolan

Serial2.println((const char*)data);

This line sends the data and appends new line automatically as one-liner. This is also what I see with minicom.

[petro]

if there’s a compile error, it’ll use cache. Delete the pyc file and try again.

[ChrisKolan (Chris)]

I ended up using AppDaemon and PySerial.