How to sanitise sensor status for automation (null characters in sensor state)

Configuration
1

Hello all,

I am adding an interface to a charging point for an electric vehicle to my Home Assistant setup. I encounter an ‘interesting’ issue in the sense that I can read the status of various sensors in the charging point hardware but I can’t create an automation that runs as expected when a specific state change occurs.

The device itself is connector over Modbus TCP/IP, so I need to define the addresses and field types for that various registers I want to read out. There is a specific register called mode3_state that presents the charge point status as a text string. This allows me to check if there is a car connected to the charge point or not, and if it is charging or not. The status values of interest to me are ‘E’ (meaning no car is connected) and ‘A’, ‘B2’ and ‘C2’ (meaning a car is connected, if power is supplied to the car of if it is actually charging).

Now, onto what I want to implement: as soon as a car is connected at night I want to enable the front lights of the house. I thought this morning this would be easy: just create an automation that triggers on a state change of the mode3 sensor and be done with it. Of course it did not work as quickly as expected :slight_smile: …

Onto the ‘interesting’ part. I found out that as soon as I added a ‘from’ or ‘to’ condition to the trigger the automation would not run. I would see the value of the charing point mode3 sensor change on the web interface of Home Assistant, but it would not trigger the automation.

So I added a notification to the automation to print the state of the sensor. Also this would work as expected and print what I expected over the notification when there was not ‘to’ or ‘from’ restriction. But as soon as I added one the automation would not trigger. Hmmm…

I found out using the ‘step details’ trace of the automation that the sensor state contains a few trailing null characters ("\0"). Apparently those null characters are not stripped from the sensor value when evaluating the trigger condition of the automation. Those trailing zeros were not transmitted via the notification system (a web hook which strips the \0) so I had no clue they were there. The fact that the trailing null characters are there are due to the way the modbus interface works: I need to define how many bytes to read from the register address, and this is specified by the manufacturer of the device so I cannot really change that.

So OK, it was then clear why the automation was not triggered when I added a ‘to’ or ‘from’ rule. When I state that the automation should run whenever the sensor state changes from E to some other value, then it is logical that it is not triggered if the actual sensor state is E\0\0\0\0\0\0\0\0.

On the other hand: the fact that the sensor state has in fact trailing null characters is not easily visible in the user interface of Home Assistant (not in the web UI and not in the ‘development tools’ section of the web interface). It was only after printing the sensor state to a notification and using the automation step details page that this became clear.

My question is: what is the most logical way to fix this? Can I easily work around this by changing how I define the trigger rules for the automation? Any tips and tricks are very welcome.

Second thought: should Home Assistant attempt to ‘sanitise’ sensor values and strip trailing \0 by default to avoid that users are scratching they heads trying to figure out why an automation is not triggered as expected?

Thanks,
Hollie

2

Repeat the experiment (that allowed you to see the null characters) but this time add the trim filter to the template.

{{ states('sensor.your_modbus_sensor') | trim }}

It’s supposed to eliminate any leading and trailing whitespace. However, I am not certain if the Jinja2 processor includes nulls in its definition of what constitutes ‘whitespace’ so you’ll need to test it.

If that fails then maybe you can try the replace filter. Sorry I can’t be more definitive but I don’t have data with nulls to test.

3

I’m by no means a Jinja2 black belt, but if @123 's solution doesn’t work, you can make a new template sensor entity and map the states into something usable. The template is pasted in your configuration.yaml file under the sensor: heading.

The link below is where I tried to help someone with it, but it wasn’t received too well: :man_facepalming:

https://community.home-assistant.io/t/yes-no-binary-sensor-new-device-class/327070

Here are some documentation links:

https://www.home-assistant.io/docs/configuration/templating/

https://www.home-assistant.io/integrations/template#renaming-sensor-output

4

Hello @123

thanks for your reply. I have tested with adding the trim option, but this does not remove the trailing nulls.

However, your suggestion to add a modifier to the template lead me to this, which is working as expected:

{{states('sensor.laadpaal_mode3_state') | regex_replace(find='\0+', replace='', ignorecase=False) }}

Thanks for your input!
Lieven.

5

Hello @pocket ,

thanks for taking the time to reply. As you can see un my previous comment I now found a working template modifier to strip the trailing null characters from the sensor value.

Do I understand it correctly from the links you posted that the recommended way to handle sensors that need a custom processing in Home Assistant is to first map them to a second sensor via a template and then use that templated sensor value as input to the trigger of the automation?

I’m just wondering and trying to learn best practices from other peoples experience.

Kind regards,
Hollie.

6

Given that trim isn’t effective in this case, I had suggested you try replace. Try this and let me know if it works:

{{states('sensor.laadpaal_mode3_state').replace('\0', '') }}

If that also fails, then use your version that employs regex_replace. However, be advised that if the sensor’s value does not contain a \0 then regex_replace will fail and cause an error message. That’s why I originally suggested to try replace because if it doesn’t find a null character nothing bad happens (i.e. no error message).

1 Like
7

I’m not sure if what I do is best practice. I find something that works and I use it until I stumble on something better. Reading the posts from @123 is instructive! :slight_smile:

1 Like
8

Hello @123

thanks for the further clarification, I can report that your code with the .replace does work. I had been searching in the template documentation page for replace when reading your previous post and only found hits at regex_replace so I thought that was what you meant with replace…

Interesting to learn that I can use .replace too and that it is potentially safer to use than the regex_replace. I just learned something new, thanks for sharing!

1 Like
9

For future reference: I ended up with the following code to be able to detect the status of the chargepoint:

  • one sensor to sanitise the output
  • one sensor to translate the sanitised sensor to a simple human readable string that can easily be used in an automation
template:
  - sensor:
      - name: mode3_sanitized
        state: "{{states('sensor.laadpaal_mode3_state').replace('\0','') }}"
      - name: chargepoint_status
        state: >
          {% if is_state('sensor.mode3_sanitized', 'E') %}
            available
          {% elif is_state('sensor.mode3_sanitized', 'B2') or is_state('sensor.mode3_sanitized', 'B1') or is_state('sensor.mode3_sanitized', 'A')%}
            connected
          {% elif is_state('sensor.mode3_sanitized', 'C2') %}
            charging
          {% else %}
            "{{states('sensor.mode3_sanitized')}}"
          {% endif %}

Regards,
Hollie

10

If you wish, it can be reduced to this:

template:
  - sensor:
      - name: mode3_sanitized
        state: "{{states('sensor.laadpaal_mode3_state').replace('\0','') }}"
      - name: chargepoint_status
        state: >
          {% set m3 = states('sensor.mode3_sanitized') %}
          {% if m3 == 'E' %} available
          {% elif m3 in ['B2', 'B1', 'A'] %} connected
          {% elif m3 == 'C2' %} charging
          {% else %} {{ m3 }}
          {% endif %}
1 Like
11

Hey @123

that is even more elegant, thank you.
I have in the mean time updated the full set of options from the datasheet of the manufacturer with:

      - name: chargepoint_status
        state: >
          {% set m3 = states('sensor.mode3_sanitized') %}
          {% if m3 in ['A', 'E'] %} available
          {% elif m3 in ['B1', 'B2', 'C1', 'D1'] %} connected
          {% elif m3 in ['C2', 'D2'] %} charging
          {% else %} {{ m3 }}
          {% endif %}