Hi. Did anyone solve this or make a workaround?
Yes, with this workaround my energy usage in HA comply with the graph from the electricity supplier in Norway, plus minus a few per cent. However, when I look at the state history for the current Nord Pool price, it is updated exactly at XX:00:00 for the NEXT hour. So I expect the costs to be miscalculated.
I have Tibber and use the Tibber intergration for electricity price.
From what I can see the daily cost calculation in HA match my Tibber reports within ± 50 øre.
The AMS HAN Home Assistant integrasjon is probably a better solution, but I have not yet tested it.
I have made a work-around for this.
Don’t know how that affects the price calculation.
Since the meters in Sweden (also Norway and Denmark?) use the HAN port, for eg. read consumed energy (accumulated kWh in the meter history), it is also possible to access the actual power (kW).
Then use “Integration - Riemann sum integral” to convert kW to kWh. Then you get the actual consumption in real time.
The disadvantage of this approach is that a possible downtime (in my case the Easee solution… cloud based) does not capture the consumption afterwards to correct it… that data is then lost.
But I will use the Riemann solution in the meantime…I am hoping HA will do some adjustment. The ackumulated kWh from the meter should be the best approach.
I have the same thing happen here on Danish Kamstrup meters. 1 hour off.
Any news on development?
My readings from Kaifa meter is almost always 10-11 seconds after whole hour. I changed the system time in Linux 20 seconds backwards. Now HA sees the received data 10 seconds before new hour kicks in.
This might be helpful, especially if you have some script anyway which fetches the consumption data from the utility company.
Some you might already know but there is an EU directive about this Smart Metering and Home Area Network port (HAN port) access. I have updated a Wikipedia page about this:
EU directives are not - unlike regulation, instantly effective in member states. That means that this directive 2019/944 needs to be ratified in member states.
Relevant parts to this thread. Chapter 19 is about Smart metering and consumer:
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L0944#d1e2607-125-1
Chapter 20:
(a) the smart metering systems shall accurately measure actual electricity consumption and shall be capable of providing to final customers information on actual time of use. Validated historical consumption data shall be made easily and securely available and visualised to final customers on request and at no additional cost. Non-validated near real-time consumption data shall also be made easily and securely available to final customers at no additional cost, through a standardised interface or through remote access, in order to support automated energy efficiency programmes, demand response and other services;
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L0944#d1e915-125-1
(26) ‘near real-time’ means, in the context of smart metering, a short time period, usually down to seconds or up to the imbalance settlement period in the national market;
Finnish law says “10 seconds or faster”.
All this in practice means HAN-port in meters that has RJ12 modular connector and RS422 signaling, it squirts ASCII text periodically. HAN port has also RTS-input, which is said to allow trigger even faster updates.
Popular smart meters like Landis+Gyr e360 supports this, among some others. See the links at the end of Wikipedia page.
All this has been legislated for home automation systems like Home Assistant, so it would be nice if the actual target software would support it properly, in near real time.
I still have this issue with hours shifted
I’m using the AmsToMQTTBridge and the local interface on my ESP8266 shows the usage for the correct hour, as with my power provider’s web interface. In home assistant it’s shifted one hour.
I really bugs me since the spot pricing used for that hour in home assistant i not the correct one and the total calculations on price is off
The data from AmsToMQTTBridge is stamped with rtc time at 55 seconds past the hour. But something in HA must handle that the data is for the hour before, as it’s not possibe to get the data before the hour has past.
Provided your meter gives you energy readings every few seconds, you can create your own sensor that will calculate hourly consumption for you.
Just go to Settings->Helpers and create a new integration sensor. Use your energy sensor from AmsToMQTTBridge as input, and a “Metric prefix” of “k”. This will give you a new sensor for your hourly consumption.
Then you use that as input in your energy configuration. It should give you data that is extremely close to the hourly reports from your meter.
Just chiming in with the same issue. Using the integral-sensor is an okay workaround, but it is sensitive to downtime, so it would be better if the energy calculations were delayed by a few seconds/minutes to allow the new data to come in before populating the energy graph.
Why not simply add a offset option to the graph to adjust input values corresponding to correct hour?
As long as you are not using realtime power to calculate present consumption, this will always be an issue the way HA calculates.
It have been years. Any update on this? Or was there no actual development on this?
No updates
i havent read it all, but i dont agree the problem is the sensor (Tibber).
if i create a new chart, outside of the energy dashboard, i get the correct results…
if the sensor was the problem, then i would get the wrong values on both places right?
Maybe im not fully understanding, or its a too simplistic solution?! i dont know, but it would be nice to actually have the Energy Dashboard correct
Look at the times each value is in the database. The energy table takes from the start of hour to the end of the hour.
The history panel just shows you raw data. There’s a very likely chance that your information is coming in slightly after the hour, which is skewing the results.
EDIT: Actually, that’s 100% happening, look at 11am for you. It’s showing the value all the way up and the drop occurs after.
BTW I found it more flexible to rely on Power metering (instead of energy one).
Power is more continuous (from HA POV) and calculation of the energy might be done on HA side, respecting the local clock.
It has downsides: it has finite resolution down to sampling frequency. So if power measurement updates HA every second, it misses changes in-between. Still, it might be better than X-seconds or Y-minutes shift.
Yep, that’s what I do too. And now the integration integration properly calculates when your data is static for long periods.
Ahah, just found out that i probably need glasses… mine is actually not 1 hour delay
however, in bold you can see the difference between what i get from Tibber app and HA, in both the Energy tab and the ‘chart’ i made.
5 to 6 = 0.75 (0.01)
6 to 7 = 0.55 (0.01)
7 to 8 = 0.31 (0.01)
8 to 9 = 0.33 (0.01)
However, in the Energy Dashboard:
10 to 11 = 0.25 (0.30)
im not sure where home assistant is getting the values, i was assuming it was from the same place, but it probably not.
in any way, its not that important to me this small differences, important is that i may need glasses.
(sorry by wasting your time)
I have been dealing with this for a few years now, by using Node-Red to modify the data in the mysql statistics table 1 min after each hour when the data is received from the meter. This has been working great for a long time now. The code is too complicated to share here, but it basically loads the last 2 values, and run a new sql query to update the record with corrected data.
When Tibber provides the one day delayed consumption and cost data, I again update the corresponding records in statistics table to be 100% synced with the data from Tibber, using Tibber integration and Mysql queries from Node-Red.
However the other day I got this idea: What if you instead just create a mysql trigger on the statistics table that shifts the timestamp before it is actually written to the database, and only for the consumption and cost entities. I have tested this in an test HA installation, and it works great, if you can live with that the table data is populated 1 hour later than normal. With this trigger the data is saved in the correct timeframe.
CREATE TRIGGER `shift_statistics_time` BEFORE INSERT ON `statistics`
FOR EACH ROW BEGIN
-- Only adjust if the metadata_id is 2 or 3 (for me energy and cost entites)
IF NEW.metadata_id IN (2, 3) THEN
-- Ensure NEW.start_ts is not NULL
IF NEW.start_ts IS NOT NULL THEN
-- Subtract 1 hour (3600 seconds) from start_ts
SET NEW.start_ts = NEW.start_ts - 3600;
END IF;
END IF;
END
And for short term:
CREATE TRIGGER `shift_statistics_short_time` BEFORE INSERT ON `statistics_short_term`
FOR EACH ROW BEGIN
-- Only adjust if the metadata_id is 2 or 3
IF NEW.metadata_id IN (2, 3) THEN
-- Ensure NEW.start_ts is not NULL
IF NEW.start_ts IS NOT NULL THEN
-- Subtract 5 minutes (300 seconds) from start_ts
SET NEW.start_ts = NEW.start_ts - 300;
END IF;
END IF;
END
Warning! This is not meant to be a guide, but ideas on a workaround approach for this problem. This may break your HA installation. Make sure you understand what you are doing before trying to repeat any of this. Always test in a test system first. I cannot help you if something goes wrong.
If anyone can come up with a py code for HA, that allows you to correct existing statistics without using direct sql queries, I could probably write a HA integration that would sync the data with delayed consumption/cost data from the Tibber API.