How are people using utility meters in practice?

I’m not sure if I’m misunderstanding how to use these, so am looking for someone to check my thinking. I’ll give a simple example for concreteness:

Let’s say I have a simple sensor entity which acts as a basic energy meter that just tracks total energy usage (in kWh) by some device (for instance, a charger for an electric car) since the meter was initiated. The sensor is a “total cumulative” kind of sensor, which starts at zero and never resets - it only ever increases as energy is consumed. Let’s say it gets a new value every 15 minutes, and that the entity is called sensor.electric_car_energy.

A common thing to want to do is to track the daily, monthly, and yearly energy usage by this device. Then according to the documentation, it is suggested that a good way to do this is to use the Utility Meter integration, and define something like this:

utility_meter:
  daily_energy:
    source: sensor.electric_car_energy
    cycle: daily
  monthly_energy:
    source: sensor.electric_car_energy
    cycle: monthly
  yearly_energy:
    source: sensor.electric_car_energy    
    cycle: yearly

1) Question About Duplication

After putting the above into your configuration.yaml, then each of these utility meters automatically generates an associated entity (which will be something like sensor.daily, sensor.monthly, sensor.yearly). Now, since all three of these have the same source (sensor.electric_car_energy), and this source updates every 15 minutes, then the data points that arrive every 15 minutes will all appear duplicated in all 4 entities. Is my understanding correct here? If so, what is the logic to justify this duplication?

2) Question About Plotting

Let’s consider now the simple case where I just want plot some bar charts:

  • a daily chart, showing energy used each day over the last 7 days
  • a monthly chart, showing energy used each month over the last 12 months
  • a yearly chart, showing energy used each year for the last 5 years

Take for instance the daily chart. Then in order to make it I need x7 data points - total energy used on Monday, total used on Tuesday, … etc. Is my understanding correct that the sensor.daily entity alone cannot do this directly? because it seems to be that only the value from the last_period is saved as a single attribute (i.e. the previous day’s total, before the meter reset it back to zero). Instead, are people using some “group by” statistics or something, which looks back over the past 7 days and grabs only the max from each period? Because this would require a lot more overhead if it’s done on-the-fly directly in the dashboard chart with such a large amount of historical data stored by the sensor.

3) Question About Unnecessary Fine-Grained Data Being Saved

If my energy meter sensor.electric_car_energy updates every 15 minutes, then is all this data stored in the long-term statistics and recorder history for all three utility meter sensors as well? Because it seems to me that the logic of defining a “daily” sensor in the first place is implicitly because we only care about the final daily value. I have seen some justification of the reason for this being because people generally want to see a value for “how much have I used so far today”, and not have to wait until the end of the day to see it. This makes total sense, and therefore makes sense save the most recent value - but why save every single fine-grained 15 minute data point over multiple days?

I’m for sure not complaining here, but just hope someone can correct / enhance my understanding. I’m not sure I’m using the utility meter in the right way, and the documentation gives the example of setting up the utility meter sensors without talking about these limitations I’ve mentioned above.

It seems that its main appeal is that it provides an automatic reset function at the end of each cycle. So are people generally turning off the recorder history on all these utility sensors, and grabbing only the very few important (e.g. daily, monthly, yearly) values using template sensors?

Interested to hear anyone’s thoughts on this. Thanks.

I use the utility meters for water and electric. Yes it is driven from the same variable but it gets reset at different times. Daily, weekly, monthly and yearly. attached are two screenshots that show this.

For Energy Usage:

And for water Usage:

I lost my router a few months ago and during the rebuild I lost some of my statistics due to the change in subnets. I think the graphs still tell the story. My water usage comes from eyeonwater, and it is updated on an irregular basis and is provided for free from my water utility: so I don’t complain about the choppiness of the data. I live in an arid climate, so the usage is correct even though it seems weird.

I can only answer the part quoted above.

Basically, yes and no. According to the docs, the recorder history will have every 15-minute entry until this is purged, since this uses short-term storage.

For Long-term statistics (anything past the recorder retention period), that data is aggregated into hourly values, so you’ll get a single hourly reading of the 4 previous values you’re updating every 15 minutes.

Okay I see. But that doesnt depend on the Utility Meter cycle that you have set?

So, even if you have a yearly cycles utility meter sensor, it would still only aggregate hourly? If so, a huge amount of additional fine-grained data it would seem to me.

I have no idea about that part, which is why I didn’t address it in my earlier reply :slight_smile:

1 Like

Sensors with a total or total_increasing state_class keep different statistics. They don’t keep max, min and average. The keep sum, state and change.

That’s fine, but I assume by “keep” that you are referring to the same thing as ShadowFist above, i.e that the data is downsampled from its original frequency to hourly. Regardless of the utility meter cycle. The fact that it aggregates by change, state, and sum doesnt materially change much from the point of view of what I discussed in the OP, right?

The daily and monthly meters reset on different cycles so this is not true after the first day:

The sum will be different.

This is unnecessary duplication though:

:slightly_smiling_face: That i can agree with immediately.

The state and change stats will also be different for the first entry after midnight.

But again, im having a hard time seeing why its useful. Even if its different as you say, why would it make sense to care about downsampled-to-hourly stats on a monthly or yearly cycled utility meter sensor?

It is actually down sampled to hourly and daily stats.

So you can do this:

and this:

and this:

In the energy dashboard direct from the sensor without the need for utility meters. They are only required if you want to use the data in templates without resorting to SQL sensors.

Are each of these from 3 different autogenerated utility meter sensors, on different cycles? Can you share the details of the sensor used in each? Maybe that will clear it up for me.

Your use case doesn’t require any utility meter entities.

You can do everything you want with your existing sensor.electric_car_energy entity, assuming it is configured correctly to generate long-term statistics. It should have a state_class set to either total or total_increasing. That can be changed if it is not currently configured like that.

Once you have statistics being generated, that will give you an hourly data point of the sensor’s running total (sum) as mentioned in the statistics documentation.

A statistics graph card on your dashboard can take this sum and then plot out hourly, daily, weekly, monthly, or yearly plots from that single source of data. And since statistics are retained indefinitely, you don’t need to extend the duration that you keep short term history of all your data.

Okay. So if I have it right, you are using the same single sensor entity as the input for all 3 of those plots you showed, that display different time scales - your monthly, daily, and hourly plots.

So, the same sensor contains fine-grained data (albeit downsampled) even for all the previous months. Just to be clear (and thanks for your patience), what I am saying is that this is taking up unnecessary disk space. But you are saying the benefit of this is that you can see the hourly data in past months (e.g. it allows you to look back at what happened hour by hour in January).

I am aware of the hourly downsampling, as is documented, but it’s the first time I’m hearing about downsampling to daily values. Do you mean that there is an extra column or something in the SQL database, which contains only one value per day, in addition to the hourly data? Is this documented somewhere?

Maybe I’m getting closer to seeing the problem. I thought the simple example I gave in the OP was a perfect use case for what the utility meter integration was supposed to achieve. But now it’s been hinted a couple of times that perhaps it in unnecessary to use them:

Especially given that the utility meter has all kinds of quirks, such as not being able to use a monetary device class that is total_increasing type, as well as not being able to submit consecutive delta readings that are the same. It leaves me thinking why should I bother with the utility meter at all. I’m yet to see a convincing use case for it.

In fairness, one thing it does handle well is multiple tariffs, i.e if you have a single source sensor, it will automatically increase the relevant meter entity depending on which tariff is selected at the time. Which is useful.

This is helpful, thanks for putting it clearly.