Tracking down Energy Spikes

[MrCaspan]

I have a Utility Meter with 3 different tariffs (On-Peak, Off-Peak, Mid-Peak) I have started to notice that all my spikes seem to be happening right around the same time as the Utility Meter changed tariff. It might be coincidental but it happens a lot. I have been trying to track down all the information I can on where this is happening…

For instance I have a power plug that has an energy entity on it. Currently its at 18kWh so my understanding is, when you connect this to a Utility Meter it will take the first initial read form the sensor as a offset so to speak, an initial reading that does not count… the second reading will then be compared to the first one to see the changes.

2. When a spike like this happens what can I check to understand where the spike came from?

I am looking in the database at the states table for the metadata_id of the Utility Meter collecting the energy use and I see the following

state_id	entity_id	state	attributes	event_id	last_changed	last_changed_ts	last_reported_ts	last_updated	last_updated_ts	old_state_id	attributes_id	context_id	context_user_id	context_parent_id	origin_idx	context_id_bin	context_user_id_bin	context_parent_id_bin	metadata_id
1083126	NULL	unknown	NULL	NULL	NULL	NULL	1737688764.7467737	NULL	1737688764.746543	NULL	39028	NULL	NULL	NULL	0	AZSWUlFKQ3huVkltYTex9Q==	NULL	NULL	803
1083129	NULL	unknown	NULL	NULL	NULL	1737688764.746543	1737688764.7473576	NULL	1737688764.7471926	1083126	39029	NULL	NULL	NULL	0	AZSWUlFLBs6RH+nDhAEkmQ==	NULL	NULL	803
1083144	NULL	unknown	NULL	NULL	NULL	1737688764.746543	1737688781.615027	NULL	1737688781.615027	1083129	39028	NULL	NULL	NULL	0	AZSWUpMvmVjle5IvwtV3pA==	NULL	NULL	803
1093476	NULL	unknown	NULL	NULL	NULL	1737688764.746543	1737720000.4088442	NULL	1737720000.4088442	1083144	39029	NULL	NULL	NULL	0	AZSYLu+YO5EW9jp8fZ0A8Q==	NULL	NULL	803
1095103	NULL	0	NULL	NULL	NULL	NULL	1737727161.1436756	NULL	1737727161.1440692	1093476	39776	NULL	NULL	NULL	0	AZSYnDM4Cma2rlmMjInYfQ==	NULL	NULL	803
1095106	NULL	60129557486	NULL	NULL	NULL	NULL	1737727161.145827	NULL	1737727161.145827	1095103	39779	NULL	NULL	NULL	0	AZSYnDM5oVxh3nMorJ+Vhg==	NULL	NULL	803
1095112	NULL	60129557486	NULL	NULL	NULL	1737727161.145827	NULL	NULL	1737727196.9873047	1095106	39780	NULL	NULL	NULL	0	AZSYnL87XOpFmYh5a3LsoQ==	NULL	NULL	803

This is a brand new Utility Meter so you can see it was unknown for four reads and then went to 0 then jumped up to 60129557486 kWh (60.1 Billion kWh!!!)

When I look up the metadata_id for the energy sensor i get this

state_id	entity_id	state	attributes	event_id	last_changed	last_changed_ts	last_reported_ts	last_updated	last_updated_ts	old_state_id	attributes_id	context_id	context_user_id	context_parent_id	origin_idx	context_id_bin	context_user_id_bin	context_parent_id_bin	metadata_id
935911	NULL	0	NULL	NULL	NULL	NULL	1736873812.1324253	NULL	1736873812.1324253	NULL	178	NULL	NULL	NULL	0	AZRlvyCkVOHp4sBbPfbq1g==	NULL	NULL	173
935916	NULL	16	NULL	NULL	NULL	NULL	NULL	NULL	1736873891.3788078	935911	178	NULL	NULL	NULL	0	AZRlwFYyhccjNC6+TFyJkQ==	NULL	NULL	173
939478	NULL	16	NULL	NULL	NULL	NULL	1736904068.110462	NULL	1736901599.255516	NULL	178	NULL	NULL	NULL	0	AZRnZyAXlI9x8sXeiv27lQ==	NULL	NULL	173
940175	NULL	unavailable	NULL	NULL	NULL	NULL	1736904071.25484	NULL	1736904071.25484	939478	33820	NULL	NULL	NULL	0	AZRnjNhWVcDQcZqxArgAnw==	NULL	NULL	173
940368	NULL	16	NULL	NULL	NULL	NULL	1736948828.6671357	NULL	1736904074.5657725	940175	178	NULL	NULL	NULL	0	AZRnjOVFU3jfnPMZdcKxug==	NULL	NULL	173
945985	NULL	18	NULL	NULL	NULL	NULL	1736949209.0210357	NULL	1736949209.0210357	940368	178	NULL	NULL	NULL	0	AZRqPZe9nvGPKwT0efbaQw==	NULL	NULL	173
946027	NULL	16	NULL	NULL	NULL	NULL	1737003331.9304888	NULL	1736949513.0974257	945985	178	NULL	NULL	NULL	0	AZRqQjuJZzg3DzdyZO9kJA==	NULL	NULL	173
953617	NULL	17	NULL	NULL	NULL	NULL	1737028410.717656	NULL	1737003488.0600984	946027	178	NULL	NULL	NULL	0	AZRtedM8sIB29La3323Wmg==	NULL	NULL	173
957351	NULL	1179712	NULL	NULL	NULL	NULL	1737032558.8018315	NULL	1737032558.8018315	953617	178	NULL	NULL	NULL	0	AZRvNWjR2nQQ4ZGKH5jAmQ==	NULL	NULL	173
957406	NULL	17	NULL	NULL	NULL	NULL	1737058676.8158064	NULL	1737032856.5588806	957351	178	NULL	NULL	NULL	0	AZRvOfPuwWozRBB4OBk0Ow==	NULL	NULL	173
961577	NULL	77313635585	NULL	NULL	NULL	NULL	1737060343.6965005	NULL	1737060343.6965005	957406	178	NULL	NULL	NULL	0	AZRw3V+Q5Gx2kTldhTx1Og==	NULL	NULL	173
961624	NULL	17	NULL	NULL	NULL	NULL	1737211305.432954	NULL	1737060647.8832712	961577	178	NULL	NULL	NULL	0	AZRw4gPLYfT7A5+t86ZUfA==	NULL	NULL	173
985945	NULL	302006921	NULL	NULL	NULL	NULL	1737211784.7911315	NULL	1737211784.7911315	961624	178	NULL	NULL	NULL	0	AZR55C5XCugb2Cp5JQlrKQ==	NULL	NULL	173
985991	NULL	17	NULL	NULL	NULL	NULL	1737291231.9444633	NULL	1737212089.3696983	985945	178	NULL	NULL	NULL	0	AZR56NQZqVowfZqd0Jl6tw==	NULL	NULL	173
994684	NULL	14	NULL	NULL	NULL	NULL	1737292789.5208461	NULL	1737292789.5208461	985991	178	NULL	NULL	NULL	0	AZR+uDcQJu2SeRew+Aat0w==	NULL	NULL	173
994705	NULL	17	NULL	NULL	NULL	NULL	1737430216.4147894	NULL	1737292944.6086001	994684	178	NULL	NULL	NULL	0	AZR+upTgahqS5E6YPx3EHQ==	NULL	NULL	173
1018245	NULL	18	NULL	NULL	NULL	NULL	NULL	NULL	1737430347.890794	994705	178	NULL	NULL	NULL	0	AZSG6zByYrNRT4hWInRdbg==	NULL	NULL	173
1026837	NULL	18	NULL	NULL	NULL	NULL	NULL	NULL	1737481814.5736053	NULL	178	NULL	NULL	NULL	0	AZSJ/IItzQxYWSy88sG7OA==	NULL	NULL	173
1038885	NULL	18	NULL	NULL	NULL	NULL	1737599661.3273947	NULL	1737551880.45238	NULL	178	NULL	NULL	NULL	0	AZSOKaEEw36qYclL9o35Wg==	NULL	NULL	173
1048218	NULL	15410362319107	NULL	NULL	NULL	NULL	1737600170.9695263	NULL	1737600170.9695263	1038885	178	NULL	NULL	NULL	0	AZSRCnvZxV6ZdMCG8kaDYQ==	NULL	NULL	173
1048242	NULL	18	NULL	NULL	NULL	NULL	1737726970.4645069	NULL	1737600258.1760218	1048218	178	NULL	NULL	NULL	0	AZSRC9CAYSuKZxfPtkSv4A==	NULL	NULL	173
1095100	NULL	60129557504	NULL	NULL	NULL	NULL	1737727161.1419084	NULL	1737727161.1419084	1048242	178	NULL	NULL	NULL	0	AZSYnDM1YIyylIMPmbQl/A==	NULL	NULL	173
1095111	NULL	18	NULL	NULL	NULL	NULL	NULL	NULL	1737727196.9861915	1095100	178	NULL	NULL	NULL	0	AZSYnL86zOfdtWtgV5DScw==	NULL	NULL	173

These readings are all over the place LOL the one for 15.4 trillion!!! that really throws off your calculations for cost

I am using the Hubitat integration from HASC store.

When I look at the readings form Hubitat they are all 18kWh and don’t show any of these spikes.

So somewhere between Hubitat and HomeAssistant these reading spiked… How do I track it down to see what is causing it… I know I can just throw a filter on the reading to remove these kinds of spikes but I would also like to make sure I am correcting an issue if there is one.

EDIT: I just realized I did not have logging enabled for the MakerAPI on the Hubitat so I have enabled that to possibly see what it is reporting to HomeAssistant.

Curious were to start of if its a moot attempt to even look and just use a filter and move on with my life!

[Sir_Goodenough ((SG) WhatAreWeFixing.Today)]

Hello MrCaspan,

This might help prevent weird spikes.
There is also some filtering helpers that would detect a spike and ‘spike’ it back down…

Why an availability template is important for energy template sensors

When defining a template sensor for use in the energy dashboard it is important to also use the availability template option. Without this you may see incorrect values in your energy dashboard. For example, consider this template sensor that sums two energy sensors to obtain a total: template: - sensor: - name: "Energy Total" unique_id: 094412db-6903-dead-beef-b40e0e41ea28 device_class: energy state_class: total_increasing unit_of_measurement: kWh …

[Rudd-O]

Congratulations on purchasing Chernobyl!

[MrCaspan]

@Rudd-O

Congratulations on purchasing Chernobyl!

Right LOL I have seen spikes but I got up this morning and my energy use cost was $20 Billion dollars I almost did a spit take

@Sir_Goodenough

This might help prevent weird spikes.

So for this suggestion I am having difficulties applying this to my scenario. If I have an energy entity that attached to a Utility Meter with 3 tariffs… each of the tariffs will have a sensor that holds the kWh that was used. I then create a number helper that calculates the total value by doing the math of hydro costs for each tariff and adds them up

{% set s = 'sensor.patio_lanterns_utility_meter' %}
{% set mid_peak_usage = states(s ~ '_mid_peak') | float(0) %}
{% set off_peak_usage = states(s ~ '_off_peak') | float(0) %}
{% set on_peak_usage = states(s ~ '_on_peak') | float(0) %}

{% set i = 'input_number.hydro_rates' %}
{% set mid_peak_rate = states(i ~  '_midpeak') | float(0) %}
{% set off_peak_rate = states(i ~  '_offpeak') | float(0) %}
{% set on_peak_rate = states(i ~ '_onpeak') | float(0) %}
{{ (mid_peak_usage * mid_peak_rate) + (off_peak_usage * off_peak_rate) + (on_peak_usage * on_peak_rate) }}

So where would I put this availability? In the utility meter I have checked of the setting for Sensor always available (the sensor will always be show the last known value, even if the source entity is unavailable or unknown) Just incase the Hubitat goes off line or is not available for some reason…

I am just not sure where I would add this availability?

[Fanful]

First, I would start by checking the input sensor entity. Does it have those weird spikes in history or not? If it does, then this is not an issue of the utility meter.

If it doesn’t, then the issue lies within the utility meter. Maybe it’s somehow configured wrong. Like, are you sure you set the correct input sensor.

[MrCaspan]

@Fanful

First, I would start by checking the input sensor entity. Does it have those weird spikes in history or not? If it does, then this is not an issue of the utility meter.

Yes that 2nd table of data I posted shows that the reading were form the entity not the utility meter. SO I believe you are right it is the sensor that is the issue. Problem with that is on the Hubitat side there is nothing in the history for these readings. they are all normal. So Either the Maker API on the Hubitat or the Integration on the HA is causing it…

I know normally if you have an energy meter with a very high reading it is possible for it to go unavailable and then available and unavailable reports as a 0 it seems so it looks like you used all this energy, but this energy reading was at 18 kWh and even if it did go unavailable and then back available to get to 60Billion would be a lot so I fell like this issue is different the readings were all over the place for some reason as you can see in that second table, like what would cause it to do this!!

[Fanful]

Yeah, looks like it’s an issue with the integration, you should report it on their GitHub.

In the meantime you can use filter helper, which was already mentioned in this thread. So this means creating a filter helper that would take as the input your energy sensor, and then your utility meter helper would use that filter helper as a source.

The filter helper is only available through yaml configuration though. The outlier one seems like it should fit here.

Filter

[MrCaspan]

yeah I created a filter today because I want to see if it helps at all I used an outlier using a sample size of four and a radius of four. so we’ll see how that does.

I’ve been having more fun clearing out all the logs for these sensors to ensure none of these spikes are in the many more!!