Denoising analog data

Configuration
1

Hi,
being new to HA, so please excuse if I am asking a stupid question.

I am trying hard to de-noise an analog sensor signal.

My setup:
I connected a modbus module “CWT-MB308Z 24AI 0-5V Analog Input Rs485 Modbus Io Module with Isolation Protection”.
The sensors are connected via very long lines (>10-60m).

modbus:
 - name: cwt_ai24_a
    type: tcp
    host: 192.168.0.123
    port: 502
    timeout: 9
    sensors:
      - name: cwt_ai24_a_1
        unique_id: cwt_ai24_1
        slave: 1
        address: 50
        input_type: holding
        data_type: int16
        scan_interval: 1

So naturally the readings include some noise which is then treated as a change of sensor data by HA almost every second for 24 sensors.
To reduce the load I want to define these sensors to only report new values if a certain delta to the previous value exists.

Filters from my understanding will just create an additional sensor. But anyways I did not find a delta function among the filters.

I also did not find a way to modify the sensor in configuration.yaml where I could for example divide by 100 and round to integer.

Any help appreciated.

2

Welcome - really interesting use case with remote sensing - 60m is a long way!

You can definitely use templates to modify the state Templating - Home Assistant

an example from my set up
state: ‘{% if ‘‘un’’ in trigger.json.Cortex_HAAPI.State.split(’‘-’')[1] | slugify
%}off {% else %}yes {% endif %} ’

Jinja2 also includes round function and many others.

TBH I find it quite obscure, but there are plenty here who offer great detailed help

3

I don’t think you can do that, i mean the Modbus is a local polling integration.

Meaning, HA asks for a sensor update, regardless if it is changed or not.
This in contradiction of a push integration, where the sensor pushes its value .
This means, the update frequency should be adjusted at the sensor side (if the devise allows it possible)

However, what you can do, is throttling by introducing a minimal scan interval setting:

scan_interval integer (Optional, default: 15)

Update interval in seconds. scan_interval = 0 for no polling. Entities are read shortly after startup and then according to scan_interval. Remark, when restarting HA the last known value is restored.

4

Thank you very much for your suggestion.
I missed to explain, that the 1s Polling is a requirement which cannot be changed, as the system must respond within max 2 seconds on changes.
So unfortunately it is not a solution to reduce the number of measurements per time.
An easy fix for this would probably be to increase

scan_interval: 1

in configuration.yaml where the timing is not critical.

5

@Jonaz80: Thank you for your suggestion.
I am currently trying to understand where this templating could take place. The documentation describes pretty detailed how to template, but not where to hook it into HA at my first reading.
Will come back after further investigation.

6

Where possible I would normally remove the noise right at the sensor, as for example in my esphome devices. In this case it is 3rd party hardware - as usual in modbus integrations.
I am wondering if such feature wouldn’t be of general interest in the modbus impementation of HA, but don’t dare to open a FRQ as I am still a rookie in HA.
So I am currently looking at the code to see if I can add it to
homeassistant/components/modbus/sensor.py for testing.

7

I am no expert but I use templates to massage a JSON response into a value for the state of a rest sensor.

value_template: "{{ value_json.CortexAPI.PortEvent.Value | round(1) }}"

I don’t know modbus, but can you add a template into the sensors section of your device definition? Something like this would give some filtering

{{ sensor_value/10 | round(2) *10 }}
8

If you just want to filter, just create a new template sensor that uses the raw modbus data.

9

I tired to add your line to one of the sensors:


Invalid config for 'modbus' at configuration.yaml, line 135: 'value_template' is an invalid option for 'modbus', check: modbus->1->sensors->0->value_template
10

@aceindy: Thank you. I will keep this in mind as an optimization in case that it cannot be done on sensor level.

11

What I was trying to show was the use of a template to round a number coming in from another source, in this case JSON via a webhook.The options available vary by integration, but you might find that you can use a template similar to this to modify the input data from Modbus.

I strongly recommend you use the template page under dev tools (/developer-tools/template) to read the input and see if you can modify it, you can then use the template sensor mentioned by @aceindy to ‘de-noise’ the modbus signal and put the result into a new sensor variable.
so Raw = modbus; renoised = template sensor
HTH

12

OK, so I created templates for each modbus analog sensor and removed the analog sensors from HA’s history in configuration.yaml.
Except of the issue that, when opening the details of one of the modbus sensors, the system has heavy workload and the UI freezes for a while, everything should be fine.
Nevertheless I think it makes sense to modify the modbus implementation to be able to handle some data preprocessing before everything goes into the HA stack.

Thank you all for your assistance!