[node-red] Update sensor at a specific date

Good morning,

This is not a problem strictly speaking, but more an improvement in my configuration.

I have an okofen boiler. I retrieve the summary emails from the day before with my Synology mailbox. in my VM home assistant I use node-red to retrieve the csv attachment from the previous day’s email and to calculate the pellet consumption of my boiler.

so far, no problem.

I created mqtt sensors in HA like this:

- name: pellematic_conso_total
unique_id: pellematic_conso_total
unit_of_measurement: “kg”
value_template: “{{ value_json }}”
state_topic: “pellematic/conso/total”

I use a “call service” node to send the calculated value.

I tested with two services: mqtt.publish and homeassistant.update_entity, both work.

my problem is that if node red screws up over 10 days, when it restarts, it will recalculate my values and send them all to me on the same day…

I would like the value calculated for a date to appear in the HA history on the given date… and not on that of the calculation (or the change in sensor status)

My configuration

release core-2024.3.1
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.2
os_name Linux
os_version 6.6.20-haos
arch x86_64
timezone Europe/Paris
config_dir /config
Home Assistant Community Store
GitHub Content OK
GitHub Web OK
GitHub API Calls Remaining 4994
InstalledVersion 1.34.0
Internship running
Available Repositories 1400
Downloaded Repositories 8
Home Assistant Cloud
logged_in false
can_reach_cert_server OK
can_reach_cloud_auth OK
can_reach_cloud OK
Home Assistant Supervisor
host_os Home Assistant OS 12.1
update_channel stable
supervisor_version supervisor-2024.03.0
agent_version 1.6.0
docker_version 24.0.7
disk_total 30.8 GB
disk_used 12.0 GB
healthy true
supported true
board ova
supervisor_api OK
api_version OK
installed_addons File editor (5.8.0), Samba share (12.3.1), Node-RED (17.0.9), Zigbee2MQTT (1.36.0-1), Mosquitto broker (6.4.0)
dashboards 1
resources 8
views 14
fashion storage
oldest_recorder_run March 10, 2024 at 08:47
current_recorder_run March 16, 2024 at 7:38 p.m.
estimated_db_size 950.88 MiB
database_engine sqlite
database_version 3.44.2

This is not expected behavior. Normally you need to take specific steps in order to trigger a flow on restart. Can you provide the flow that triggers on restart?

What? sorry but I don’t understand. If HA or Node-red addons go down, the values calculated each day are not… and they will be calculated late when HA or Node-red restarts.

They will be sent to HA to modify the sensor… and the history of this sensor will have the values… but all on the same day (separated by 1min of delay, until all the values are calculated by node- red).

I would like, when passing the calculated value, to also pass the date to which this value corresponds so that in the HA history, it appears on the correct date.

Generally HA does not support inserting historic data in the database.
The spook integration however provide an option to do just that, but it is provided as is and might have issue that can crash your entire HA installation, so be warned and make backups!

I would first of all try to turn your NR into a ‘proxy’, and perhaps also store the data there in an SQLite Database or similar to reduce duplication (or whatever you feel comfortable with)

Spook looks scary :ghost: though! be careful with that :slight_smile: (although it is something I definitely missed as well)

Some suggestions and thoughts:

Trying to update Home Assistant history records is generally thought to be not a good idea!

Keep HA and Node-RED running, which eliminates the primary problem. Apart from electric supply outages, my HA and Node-RED are remarkably stable. I am recording solar PV data every hour to a Maria DB and have not (fingers crossed) missed a record in 300 days.

Extract the event date from the email, and record the date / consumption in another way:-

Use a text file, either NR file write or HA notification to file, with a text string of date-consumption appended for each email processed. This file can be read by NR for summary reporting, and has the advantage of being human readable and easily edited to correct errors. I do this from HA for my daily hot tub energy consumption.
If you don’t like text, use JSON (stringify object).

Use another database. I have MariaDB added separately to the HA SQLite, and this works very well for me.

Store the history as an array in a HA sensor attribute. I don’t know how long you are keeping HA history so as to be able to see consumption over time, but I am using Node-RED to read sensor attributes (arrays have to be held in entity attributes) and to add new record to the end / front of the array, then saving back to the sensor. This works also by using Node-RED persistent context store. I can keep a ‘ring buffer’ of data by adding the new to the front and trimming the array to a set size to drop old records. I do this for modbus read errors and NR memory tracking. All data in HA can be copied or summarised to a new HA sensor.

You can keep an array with a JSON object for date & consumption, or a JSON object with each key as the date string and consumption as the value.

This action should be able to be mitigated. I track my oil burner usage and tank level in nodered. I create sensors for daily weekly and monthly usage. I get a summary notification on Sunday triggered by an inject at 11 am.

Nothing recalculates, nothing fires randomly on restart. Quite honestly, if it did, NR would be a pretty useless program.