Utility_meter cyles

tom_l · September 5, 2020, 10:14am

The capacity tariff is charged on the basis of the ‘average monthly peak’ (kW). This average monthly peak is calculated as the average of your last 12 ‘monthly peaks’. The monthly peak is the highest quarter-hour power (or ‘peak power’) that you have used in a month.

So what you actually need for this is:

a statistics sensor, to get the peak (max) reading every 1/4 hour
a template sensor to record the highest 1/4 hour peak for each month.
a second template sensor to record the average of the 12 monthly peaks.

That is an understatement!

You could probably use a statistics sensor for the whole month to find each monthly peak, this would essentially be the same, but it would require quite a large number of samples.

fb22 · September 6, 2020, 12:49pm

The quarter-hour power (kW) is defined as the average power during a quarter hour. So the utility meter with a cycle of 1/4 hour could be used. This gives you the energy (kWh) consumed in 1/4 hour. This value multiplied with 4 is equal to quarter-hour power.

gijbelsy · September 6, 2020, 8:02pm

I’m running some tests with the statistics sensor as tom_I suggested.
It could be coincidence, but the system is running a lot more unresponsive since this sensor is gathering data. Doing some further testing on that matter.

Besides that: is the utility meter configurable with a cycle of 15 minutes?

evb · September 13, 2020, 3:29pm

@gijbelsy
Have you been able to configure something successfully?

I was looking to see if I could do something in HA to calculate this new peak consumption every 15 minutes. Not really successful so far.

If I understand correctly the Fluvius website, it is not the peak or maximum reading every 1/4 hour, but the average peak power every 1/4 hour:

The monthly peak corresponds to the highest quarter-hourly power - or average power on a quarter-hourly basis - that you have used in a month; not the highest instantaneous power.

First an overview of what we need (what I think we need, correct me if I did interpret not correctly the Fluvius website…):
In my case, I have a sensor giving me each second the power consumption in W(att), so I get 900 samples each 15 minutes.
Example:
We have 4 x 15 minutes in a hour and we are starting our first month
00:00:01 - 00:15:00 : calculate the average of the 900 samples, take the value and store it as max
00:15:01 - 00:30:00 : calculate the average of the 900 samples, take the value and see if > than previous value, if not do nothing, if yes, store it as max
00:30:01 - 00:45:00 : calculate the average of the 900 samples, take the value and see if > than previous value, if not do nothing, if yes, store it as max
00:45:01 - 01:00:00 : calculate the average of the 900 samples, take the value and see if > than previous value, if not do nothing, if yes, store it as max
…
At the end of the month, I take the max value of the first month.
If it is < 2.5 kW store 2.5 kW (minimum) else store the value.
Then the cycle begins again for the second month.
And so on.
After 12 months we calculate the average of the 12 months.
if it is < 2.5 kW take 2.5 kW else take the calculated value.
That will be the capacity peak to pay.

Now we must configure that into HA
Someone an idea where to begin?
Is it even possible in HA with the existing platforms like Min/Max, Statistics, History-Stats, Utility_meter, …?

gijbelsy · September 13, 2020, 5:05pm

I’m running some tests here with the suggested statistics sensor.
For now the configuration is the mean value of the power consumption. The data is reset every 15 hrs.
What I can’t figure out from the Fluvius website is the calculation of the 15 minute peak value:
Is this every quarter of the hour, or gliding 15 minutes?
I’ll report back on that.

evb · September 18, 2020, 8:26pm

Thinking about the idea of @fb22 to use the utility_meter to get the average power

kW x time = kWh
or
kW = kWh / time

for example :
1 kWh per quarter of an hour
a quarter of an hour = 0.25 hour
1 / 0.25 = 4 kW average peak power per quarter of an hour

The big problem, the standard utility_meter component does not have a cycle of a quarter of an hour.

I’ve made a custom_component based on the existing utility_meter and extended it with a QUARTERHOUR cycle option.
I will test this and if it works and gives me the needed information, I will create a pull request to the home assistant core to extend the default utility_meter component with the quarter-hour cycle.

to be continued.

gijbelsy · September 19, 2020, 10:47am

Here is what I’m currently testing with:

# Kwartierpiek
- platform: statistics
  entity_id: sensor.energie_400V_totaal_actief_vermogen
  name: energie_kwartierpiek
  precision: 1
  sampling_size: 900
  max_age:
    minutes: 15

This gives me the statistics of my active power over 15 minutes.
Then with a template I visualize the mean value:

- platform: template
  sensors:
    # Energie: kwartiergemiddelde
    energie_kwartierpiek_gemiddelde:
      value_template: '{{ states.sensor.energie_kwartierpiek.attributes.mean  }}' 
      friendly_name: 'Energie kwartier piekgemiddelde' 
      unit_of_measurement: 'W'

This leaves me with 2 other statistics sensors, one for the monthly and one for the yearly peak values.

evb · September 23, 2020, 6:29pm

@gijbelsy,
I have created your two sensors in my configuration with my total grid consumption sensor.
The two sensors are giving me the same values, what, I think, is correct.

From the Statistics page (Statistics - Home Assistant)

The statistics sensor platform consumes the state from other sensors. It exports the mean value as state and the following values as attributes: count , mean , median , stdev , variance , total , min_value , max_value , min_age , max_age , change , average_change and change_rate . If it’s a binary sensor then only state changes are counted.

The first sensor ‘energie_kwartierpiek’ will give you already the mean value as state.
The second sensor ‘energie_kwartierpiek_gemiddelde’ will give you the same value via his attribute ‘mean’.

This configuration is giving the average peakpower for the last 15 minutes.
But that is not what we must have if I understand it correctly from the Fluvius website, see one of my previous posts : Utility_meter cyles - #7 by evb

My tests with the custom utility_meter with quarter-hour cycle, did reveal that I did had a 3 kW peak in the last 2 days…
(not the 5 days, I had to correct my statistics sensor for the month because the sampling_size was not correct → one sample per second → 900 samples per quarter-hour, x 4 per hour, x 24 per day, x 31 per month = 2678400, did take now as sampling-size 2700000)
But the main question is still : is my assumption correct for calculating the average quarter-hour peakpower? Utility_meter cyles - #9 by evb

still to be continued

evb · October 22, 2020, 4:22pm

Meanwhile, @thomasdelaet did do a pull request for extending the utility_meter integration with a quarter-hourly option : Add quarter-hour period feature for utility_meter component by thomasdelaet · Pull Request #41999 · home-assistant/core · GitHub

In the utility_meter component, add support for 15 minute intervals (quarter-hours).

one (and my use case for this is that in Belgium, a significant part of your electricity bill is going to be based on your peak usage per 15 minute interval. Measuring and managing this peak usage can help to reduce cost.

Documentation at home-assistant/home-assistant.io#15290

I hope to see his pull request accepted and done for the version 0.117.0 of HA.

I switched also to the influxdb v2 integration to store de raw data, because my HA database is limited to store only 5 days of data, otherwise it is explosing in size.

On the influxdb side I added two graphs on the dashboard, one based on the value calculated by the utility_meter integration based quarterhourly and on based on the raw value of my realtime smappee sensor giving the power consumption of the grid in W each second.

Calculated value utility_meter quarterhourly

maxAverageQuarterHourData = from(bucket: "homeassistant")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["friendly_name"] == "Gemiddeld piekvermogen per kwartier")
  |> filter(fn: (r) => r["_field"] == "value")
  |> filter(fn: (r) => r["_measurement"] == "kW") 
  |> aggregateWindow(every: 15m, fn: max, createEmpty: false)
  |> yield(name: "maxAverageQuarterHour")

maxMonthData = maxAverageQuarterHourData
  |> aggregateWindow(every: 1mo, fn: max, createEmpty: false) 
  |> yield(name: "maxMonth")

maxMonthData
  |> aggregateWindow(every: 1y, fn: mean, createEmpty: false) 
  |> yield(name: "averageYear")

Raw value of smappee sensor grid consumption

averageQuarterHourData = from(bucket: "homeassistant")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["friendly_name"] == "Net afname")
  |> filter(fn: (r) => r["_field"] == "value")
  |> filter(fn: (r) => r["_measurement"] == "W") 
  |> aggregateWindow(every: 15m, fn: mean, createEmpty: false)
  |> yield(name: "averageQuarterHour")

maxMonthData = averageQuarterHourData
  |> aggregateWindow(every: 1mo, fn: max, createEmpty: false) 
  |> yield(name: "maxMonth")

maxMonthData
  |> aggregateWindow(every: 1y, fn: mean, createEmpty: false) 
  |> yield(name: "averageYear")

Of course to get and see the 12 month values, we must have the data of 12 months…
You can experiment by setting for example 1 day instead of 1 month and 12 days instead of 1 year.
At that moment you can already test the graph layout.

Why two graphs?
You see that the two graphs are grosso modo identical, meaning that the assumption to use the calculated utility_meter quarterhourly is correct (see my previous post)

Hereunder an example of my output. The line you see between the two blocks is when the influxdb v2 integration did have a bug when going from HA 0.0115.6 to 0.116.0.
During several days no data was inserted into the influxdb till I discovered the problem and corrected it by adding the optional precision option. It was optional for influxdb v1.x, but required for influxdb v2.
A bug correction was done in HA for the next version 0.117.0? to make it also optional for influxdb v2.

gijbelsy · March 12, 2021, 7:17pm

The problem is that for the moment there is too little information about how the peak values are calculated by the grid supplier.
Therefore I paused this until things get more clear.
As we know, things can change a lot in Belgium, especially when it’s about energy.

baz123 · May 24, 2021, 3:49pm

True, but this cannot be calculated from measured kWh values. By definition, the kWh figure is gained by determining how much power (kW) was used in the measured time (h). A 1 second load of 10kW is the same kWh value as a 10 second load of 1kW. If you only get an energy reading every 10s, simply dividing the kWh figure by 10 will not give you the max power during that 10s.

For each reading period, the measuring device needs to return the Max power seen (kW) - you can take those figures and find the max for the quarter hour and then the average for the month. You cannot reverse engineer the figure from measured energy (kWh).

gijbelsy · May 24, 2021, 6:04pm

I’m not suggesting that this is derived from kWh. Indeed peak power is measured in kW.
That’s why my sensor is based on reactive power (VAR), not consumed energy (kWh).
The only thing that’s currently unclear is the timeframe of the so called ‘peak value’.
I hope this information will be provided soon…

baz123 · May 24, 2021, 6:08pm

What sensor do you have to measure the max figure over the recording cycle?

marchano · June 28, 2021, 2:28pm

just a little clarification: Energie(Energy): kWh; Vermogen(Power): kW

JohanBraeken · June 29, 2021, 8:56am

What is being measured is called in Dutch “het kwartiervermogen”.
It can be loosely translated as “quarter-hour power”.

Here is how it is determined and how you can get it from your utility meter data:

First things first: It is NOT a moving average.

It is the average demand in a quarter-hour, with 4 values computed for every hour of the day, representing the time periods

From x:00:00h to x:14:59h,
From x:15:00h to x:29:59h,
From x:30:00h to x:44:59h, and
From x:45:00h to x:59:59h.

It represents the load you have on the electricity grid.

In order to determine the quarter-hour power value, you calculate the energy consumed in the quarter-hour by taking the total consumption at the end of the quarter minus the consumtion at the beginning of the quarter-hour.
Ex: At x:15:00h your utility meater read 4998,00 kWh and at x:29:59h it said 5000,00 kWh. 5000,00 kWh - 4998,00 kWh = 2,00 kWh consumed in 15 minutes.

Then you multiply this value by 4, because there are 4 quarters in a hour.
Ex: 2kWh consumed in 15 minutes represents 2 x 4 = 8 kW of power demand.
(Because this consumption in 15 minutes really is equivalent of 1/4th of the average power demand in that quarter-hour.)

Of all the quarter-hour values determined, only the maximum of each calender month is retained.
(Important: If the monthly peak value is less than 2,5kW still 2,5kW is used.)
And with these max monthly values, the yearly average (The simple arithmetic mean) is calculated, which has (will have) an impact on your invoice. Users with higher loads on the grid will pay more (in the future).
(If the year is not complete, it is the arithmetic mean of the incomplete dataset.)

marchano · July 16, 2021, 10:21am

any idea how to calculate this within homeassistant?
I’m logging all these values in a mysql db

tmatheussen · August 5, 2021, 7:01am

I’m currently attempting a couple of different methods (including logging values to influxDB and graphing in there) but so far the most correct method seem to be to use the utility meter, with the quarter-hourly cycle.
As input I use the tarif 1 and tarif 2 consumption sensors coming from the DSMR integration.
My next steps will be to get the monthly max peak and all the peak from the last year/12 months, but with the new statistics charts that shouldn’t be too hard (that info is also kept indefinitely as opposed to the state changes)

EDIT: This is my graph from yesterday:
The yellow bars are values coming from the DSMR integration, every 1 second, grouped per 15minutes by the apex-charts card. The blue one is the tarif1 utility meter, the red one is the tarif2 utility meter (red one has no values just yet, only activated this morning)

As you can see, grouping and taking the average from the actual Power values from DSMR seems to align pretty much with the utility meter integration. I’m going to keep both of them active for a while and compare results in the coming days. if the result stay the same I’ll be removing the direct DSMR graphs and changing the interval back to 30seconds (default), the 1 second interval causes a lot of data to come into home assistant which really isn’t needed.

gigatexel · August 30, 2021, 10:30am

You might get slightly different results in HA versus Mijn Fluvius, because DSMR is not sending data in a time-synced fashion.

What I am doing, is sum both tarifs in NodeRed using an inject node, which is cron under the hood.

Function node:

const globalHomeAssistant = global.get('homeassistant');
var output;
var afname_1 = parseFloat(globalHomeAssistant.homeAssistant.states["sensor.energy_consumption_tarif_1"].state);
var afname_2 = parseFloat(globalHomeAssistant.homeAssistant.states["sensor.energy_consumption_tarif_2"].state);


output = (+afname_1) + (+afname_2);
output = output.toFixed(3);
return [{payload: output}];

Capacity tarif is only applicable to downstream energy.

tmatheussen · August 30, 2021, 11:07am

I actually don’t really care about 100% accurate results, just a rough estimate is good enough for me.
The main reason I don’t need 100% accurate results is that my meter actually does not send any data to Fluvius, so I don’t get any info from MijnFluvius at all

gigatexel · September 1, 2021, 12:38pm

Newsflash.

New HA 2021.9 Utility Meter will support cron-based resets. So you can easily let UM reset every 15min of the hour, multiply by 4 et voila.