Warning state_class and device_class are incompatible

cydoniaP · July 12, 2024, 4:29pm

I am currently running Core 2024.7.2 using the HassOS image running in a Proxmox VM on a stand-alone x86-64 based machine. Supervisor and HAOS versions are 2024.06.2 & 12.4 respectively.

Several months back, in an attempt to create a 24 hour rain tally, from a weather integration hourly rain sensor, I created a Utility Meter sensor as follows:

sensor.daily_rain:
  device_class: precipitation
  unit_of_measurement: 'in'

My utility_meter.yaml file is included from configuration.yaml

utility_meter: !include utility_meter.yaml

This daily rain sensor is woking as expected, however, I’ve noticed in the system logs the following warning after every HA restart:

“WARNING (MainThread) [homeassistant.components.sensor] Entity sensor.daily_rain (<class ‘homeassistant.components.utility_meter.sensor.UtilityMeterSensor’>) is using state class ‘total_increasing’ which is impossible considering device class (‘precipitation_intensity’) it is using; expected None or one of ‘measurement’; Please update your configuration if your entity is manually configured, otherwise create a bug report at Issues · home-assistant/core · GitHub”

In reviewing the sensor entity docs I believe the ‘state class’ of ‘total_increasing’ is correct for this sensor:
Similar to total, with the restriction that the state represents a monotonically increasing positive total which periodically restarts counting from 0, e.g. a daily amount of consumed gas, weekly water consumption or lifetime energy consumption. Statistics of the accumulated growth of the sensor’s value since it was first added is updated every 5 minutes. A decreasing value is interpreted as the start of a new meter cycle or the replacement of the meter.

But the ‘device class’ should be ‘precipitation’ not ‘precipitation_intensity’ and thus the ‘unit_of_measurement’ should be ‘in’ and not ‘in/h’.

To modify the ‘device class’ and ‘unit_of_measurement’ I added the following to my customize.yaml file:

sensor.daily_rain:
  device_class: precipitation
  unit_of_measurement: 'in'

My customize.yaml file is included from configuration.yaml:

homeassistant:
  customize: !include customize.yaml

I restarted HA and then from Developer tools > States I verified the dail_rain sensor attributes:
state_class: total_increasing

status: collecting*
last_period: “0”*
last_valid_state: None*
last_reset: “2024-07-12T05:02:00.001544+00:00”*
next_reset: “2024-07-13T00:02:00-05:00”*
unit_of_measurement: in <== new value, was in/h*
device_class: precipitation <== new value, was precipitation_intensity*
icon: mdi:weather-pouring*
friendly_name: Daily Rain*

Unfortunately there is now a different warning message:
The unit of sensor.daily_rain (in) cannot be converted to the unit of previously compiled statistics (in/h). Generation of long term statistics will be suppressed unless the unit changes back to in/h or a compatible unit. Go to Link to Developer tools: statistics – My Home Assistant to fix this

Going to the provided link took me to: http://homeassistant.local:8123/developer-tools/statistics

Here the daily rain sensor was listed indicating the unit of this entity changed from in/h to in and a FIX ISSUE button was highlighted.

Clicking the Fix Issue popped up an actionable dialog:
The unit of this entity changed to ‘in’ which can’t be converted to the previously stored unit, ‘in/h’.

If the historic statistic values have a wrong unit, you can update the units of the old values. The values will not be updated.*
Otherwise you can choose to delete all historic statistic values, and start over.*

How do you want to fix this issue?
(1) Update the unit of the historic statistic values from 'in/h' to 'in', without converting.
(2) Delete all old statistic data for this entity

I chose the first option and after the action completed I restarted HA again to verify there were no further daily_rain sensor warnings.

Well damn, the exact original warning is back. Going back to Developer tools > States I can see the ‘device class’ and ‘unit_of_measurement’ are still what I modified them too. The warning still indicates, however, that the device_class is precipitation_intensity, which it no longer is, per the Dev tools.

In further exploring this issue I discovered that the core.entity_registry file contains the following, note in this file it has the ‘device_class’ as “precipitation_intensity”: {“aliases”:[],“area_id”:“santaritaranch”,“categories”:{},“capabilities”:{“state_class”:“total_increasing”},“config_entry_id”:null,“device_class”:null,“device_id”:null,“disabled_by”:null,“entity_category”:null,“entity_id”:“sensor.daily_rain”,“hidden_by”:null,“icon”:“mdi:weather-pouring”,“id”:“6d749158bc572c11631233b7e99a0d64”,“has_entity_name”:false,“labels”:[],“name”:null,“options”:{“conversation”:{“should_expose”:false},“sensor”:{“display_precision”:3}},“original_device_class”:“precipitation_intensity”,“original_icon”:null,“original_name”:“Daily Rain”,“platform”:“utility_meter”,“supported_features”:0,“translation_key”:“utility_meter”,“unique_id”:“daily_rain_single_tariff”,“previous_unique_id”:null,“unit_of_measurement”:null},

I manually edited core.entity_registry (yeah I know this might be risky, but I had a full backup & a backup of this file) changing original_device_class to precipitation and then restarting HA. The original warning was still seen in the logs. Looking at the core.entity_registry file again I found the device_class was again precipitation_intensity.

I also found that the file core.restore_state contains the following, which shows the modifications as well as the original settings:
{

 "state": {"entity_id":"sensor.daily_rain","state":"0","attributes":{"state_class":"total_increasing","status":"collecting","last_period":"0","last_valid_state":"None","last_reset":"2024-07-12T05:02:00.001544+00:00","next_reset":"2024-07-13T00:02:00-05:00","unit_of_measurement":"in","device_class":"precipitation","icon":"mdi:weather-pouring","friendly_name":"Daily Rain"},"last_changed":"2024-07-12T15:09:19.063063+00:00","last_reported":"2024-07-12T15:09:19.062959+00:00","last_updated":"2024-07-12T15:09:19.063063+00:00","context":{"id":"01J2KQVMAQE7SB5B31KW60X164","parent_id":null,"user_id":null}},*

```
 "extra_data": {*
```
```
   "native_value": 0,*
```

   "native_unit_of_measurement": "in/h",*

```
   "last_period": "0",*
```

   "last_reset": "2024-07-12T05:02:00.001544+00:00",*

```
   "last_valid_state": null,*
```
```
   "status": "collecting",*
```

   "input_device_class": "precipitation_intensity"*

```
 },*
```

 "last_seen": "2024-07-12T15:09:23.783175+00:00"*

}*

I manually modified the input_device_class and native_unit_of_measurement with the new values and restarted HA. Just like the core.entity_registry, the contents of core.restore_state reverted back to previous values.

Not sure where these persistent values are stored, but it’s clear there’s a lot I still don’t yet understand about HA.

I’ve re-verified, via Dev tools, the attributes of the daily_rain sensor and the ‘device_class’ and ‘unit_of_measurement’ are set to the values defined within the customize.yaml file.

The one thing I haven’t tried yet is removing the sensor altogether and then recreating it, but since this has been working for months, I would first really like to understand why my daily_rain attribute changes are not resolving this issue or even why the initial settings were as they appeared, especially since HA knows the two different settings it assigned are not compatible.

I’d really appreciate it if someone could educate me on the proper way to setup a utility meter sensor for the purpose of accumulating hour data over a 24 hour period.

mterry63 · July 12, 2024, 4:41pm

I believe you have to shut down Home Assistant when editing these files. I believe they are loaded in memory on startup so if you edit while running, they will be overwritten.

cydoniaP · July 12, 2024, 6:28pm

So, I did go ahead and delete the daily_rain sensor from utility_meter.yaml, restarted HA, then deleted the daily_rain sensor (was indicating unavailable) from HA entity UI.

I then reentered the daily_rain sensor details back into my utility_meter.yaml file:

rain_daily:
  source: sensor.openweathermap_rain
  name: Daily Rain
  unique_id: daily_rain
  cycle: daily
  offset:
    minutes: 2

This is the exact same code as I original defined. I again restarted HA and checked the logs and the same warning could be seen.

No clue on what to try next. It was suggested that I needed to 1st shutdown HA and then edit the two fore mentioned core files, however, since shutting down HA stops all associated services, including Samba & SSH, it is not clear how I would gain access to these files in order to edit them.

mterry63 · July 12, 2024, 7:04pm

You can enable SSH at the OS level (see here how to do that) and edit the files while home assistant is shutdown.

mterry63 · July 12, 2024, 7:10pm

I see you are running under Proxmox, you can also edit the files in the console cli.

Open the Home Assistant web UI.
Open Developer Tools > Services
Select the service homeassistant.stop
Click the “Call Service” button.
In Proxmox navigate to the console of your VM.
At the ha prompt enter login
cd /mnt/data/supervisor/homeassistant/.storage/

You only have the vi editor at this point, but you should be able to make that work.

Edwin_D · July 12, 2024, 7:16pm

DON’T EDIT THE REGISTRY OR ANY OTHER FILE IN .storage BY HAND!!

Precipitation intensity is most likely a momentary value, so you are not entirely wrong in assuming the unit_of measurement is misleading. But you should not edit the registry manually, and definitely not if you do not know what you are doing. You could permanently break HA! You are lucky you did not break anything just yet.

If you want to change the unit_of_measurement or device_class, use the customize integration. Problems with long term statistics can be fixed from developer tools.

BUT: it is not your problem. The utility meter is not suited to sum in/h over a period of time. There are numerous explanations in the docs and this forum why utility meter cannot turn kW into kWh - and this is exactly the same.

If you want to convert in/h to in then you need an integral, and the Riemann sum integral sensor does exactly that. It will give you a sensor you can put into utility meter to get daily/weekly/monthly values.

mterry63 · July 12, 2024, 7:47pm

Take a Proxmox snapshot before you edit the files and roll back if things go pear shaped.

cydoniaP · July 12, 2024, 9:37pm

So I’ve added a Riemann sum integral sensor:

- platform: integration
  source: sensor.openweathermap_rain
  name: Total Rain
  unique_id: total_hourly_rain
  round: 3
  method: left
  unit_time: h

and modified the utility_meter sensor:

rain_daily:
  source: sensor.total_hourly_rain
  name: Daily Rain
  unique_id: daily_rain
  cycle: daily

I won’t know if this is working until we actually get some rain, which this time of year is scarce. Unfortunately, however, I can say that the same warning is still seen in the logs after an HA restart.
*Entity sensor.daily_rain (<class ‘homeassistant.components.utility_meter.sensor.UtilityMeterSensor’>) is using state class ‘total_increasing’ which is impossible considering device class (‘precipitation_intensity’) it is using; expected None or one of ‘measurement’; Please update your configuration if your entity is manually configured, otherwise create a bug report at Issues · home-assistant/core · GitHub
*

Edwin_D · July 13, 2024, 2:31pm

That is because precipitation intensity is for a momentary value, not a total count. So the device class is indeed not correct for a utility meter. It is like saying you want to give your car odometer (total distance travelled) a unit of measurement that is meant for speed.

I guess utility meter was not designed with rain in mind, but you can probably set a better device_class in customize. Or did you still have one there that it is compaining about?

cydoniaP · July 13, 2024, 10:48pm

As noted in my original posting, I do have a device_class and a unit_of_measure configured in my customize file for the utility_meter sensor:

sensor.daily_rain:
  device_class: precipitation
  unit_of_measurement: 'in'

As also noted, in my original post, Developer tools > States does show sensor.daily_rain device_class as ‘precipitation’, so the customization is seen and set.

The Warning message, within the log, however, still indicates the device_class as precipitation_intensity. This makes no sense to me.

I may need to investigate other options to obtain daily rainfall totals. I originally tried to scrape this from a local weather site, but the site is running java scripts based on user actions so it won’t scrape successfully using the HA integration.