How can I stop multiple attributes being added to each entity here?

teeeeee · January 25, 2023, 6:47pm

I am a Home Assistant beginner, and don’t fully grasp attributes vs entities vs devices yet, so please bear with me…

I have added a Zigbee temperature sensor through Zigbee2MQTT, and Home Assistant has automatically created seperate entity IDs for all of the things the sensor exposes (battery voltage, humidity, temperature, etc). Here is a screenshot:

I am trying to send the temperature value only to InfluxDB, so have added the following to my config file:

influxdb:
  host: 192.168.0.48
  port: 8086
  database: homeassistant
  username: $$
  password: $$
  tags_attributes:
    - friendly_name
  include:
    entities: 
      - sensor.snzb_02_sensor_1_temperature
  ignore_attributes:
    - device_class
    - icon
    - state_class

The problem is that if I do it this way then there are actually many “fields” appearing in InfluxDB, corresponding to the other non-temperature related things (including a field called “value”). See the output from the InfluxDB database below:

If I examine the temperature entity in Home Assistant, I can see that each of these supposedly seperate entites actually has “attributes” for the other values, like this:

Has the entity been set up wrong? How can I fix this?

Thank you!

NathanCu · January 25, 2023, 7:02pm

Nope it’s working as designed.

Im about to GROSSLY over simplify this:
Entities are basically your things.
Devices are basically logical grouping of related entities

An entity has a state. Which is the main thing people are usually interested in (in this case the temperature) but often times thats one of many pieces of information an entity can provide but it’s not the ‘main’ thing. …in this case humidity. Sure, the manufacturer could expose it as a separate entity but in this case they did not.

In my case I have an Aquara curtain rod driver. The state is on/off (open/closed) but if I want there are attributes for battery, actual position etc. It really depends on how the manufacturer chose to do it.

So you need to be aware of these and plan accordingly.

teeeeee · January 25, 2023, 7:07pm

But in this case, the manufacturer does expose them as seperate entites, seen here:

It’s just that every piece of information for which an entity has been defined is also listed as an attribute.

NathanCu · January 25, 2023, 7:13pm

And?

You will see this often. If you dont want those entities you can disable them. It’s manufacturer and integration dependent. You will have to evaluate each one on its own merit.

Its not right or wrong it just IS.

teeeeee · January 25, 2023, 7:23pm

Thanks. The problems I have are the following:

Information is duplicated in the InfluxDB database (the exact same value appears in the temperature field as in the value field, as you can see in the screenshot in the OP).
At the moment, in order to look at the humidity (for example) I have to access it with this kind of query:
SELECT humidity FROM "°C"
i.e humidity resides inside the measurement that is named after the unit of temperature - which is non-intuitive. This is the situation I will be stuck with if I disable the other entities as you suggest (unless I’m missing something).

Alternatively, I could instead keep these other entities and add them to my configuration.yaml file in the InfluxDB section, like this:

include:
    entities: 
      - sensor.snzb_02_sensor_1_temperature
      - sensor.snzb_02_sensor_1_humidity

which would then result in a new InfluxDB measurement with the name “%”, but then all the data in the attributes would be duplicated as fields in each of the new measurements.

michaelblight · January 26, 2023, 3:40am

It looks like that integration does both (I don’t use Zigbee2MQTT) - it creates separate entities for each item (battery, hum, temp, etc) and on the temperature entity it also creates attributes for each. You might be able to configure Zigbee2MQTT to get what you want, but does it matter? You can always add the unwanted ones to “ignore_attributes” in the influxdb integration if it bothers you.

It’s also pretty common where the state is a number for integrations to also have it as an attribute. This is because an entity’s state is always a string value, whereas an attribute can be a number (or other type). I expect this is why you’re seeing “value” and “temperature” - I’m guessing “value” comes from the state and is probably a string in the database, whereas “temperature” is probably a number. You could choose to add “temperature” to “ignore_attributes”, but having it there as a number is more useful. You could try adding “value” to “ignore_attributes” in the hope that it doesn’t send the state, but I’d be surprised if it worked. At the end of the day, does it matter that influx db has a few extra columns? You might want to use them one day.

As for your comment

humidity resides inside the measurement that is named after the unit of temperature - which is non-intuitive

…yes it is a bit confusing and I have no idea why the influxdb integration thinks the measurement name should default to “unit_of_measurement”, and it confused me when I first used influxdb/grafana. However this can be changed to “entity_id” if you want. Then at least your select would be from “sensor.snzb_02_sensor_1_temperature” rather than “°C” - you are getting the data from the entity after all, and it does have an attribute called humidity. Also, the reason humidity only resides on the temperature measurement is because you’re not sending the humidity entity to influxdb.

teeeeee · January 26, 2023, 11:16pm

@michaelblight Hi Michael, thanks for your response - that’s very helpful, and I think I understand a little better now. You are saying that it’s the integration itself which sets what is passed as an attribute and what is an entity. This is something I didn’t grasp before (I thought it’s something we as the user prescribed).

“You can always add the unwanted ones to “ignore_attributes” in the influxdb integration if it bothers you.”

I have tried to follow your suggestion, and the best I could do was this messy looking config file:

influxdb:
  host: 192.168.0.48
  port: 8086
  database: homeassistant
  username: homeassistant
  password: *password*
  include:
    entities:
      - sensor.snzb_02_sensor_1_temperature
      - sensor.snzb_02_sensor_2_temperature
      - sensor.snzb_02_sensor_1_humidity
      - sensor.snzb_02_sensor_2_humidity
      - sensor.snzb_02_sensor_1_battery
      - sensor.snzb_02_sensor_2_battery
      - sensor.snzb_02_sensor_1_linkquality
      - sensor.snzb_02_sensor_2_linkquality
      - sensor.snzb_02_sensor_1_voltage
      - sensor.snzb_02_sensor_2_voltage
  component_config_glob:
    sensor.snzb_02_sensor_*_temperature:
      override_measurement: temperature
      ignore_attributes:
        - battery
        - humidity
        - linkquality
        - voltage
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement
    sensor.snzb_02_sensor_*_humidity:
      override_measurement: humidity
      ignore_attributes:
        - battery
        - temperature
        - linkquality
        - voltage
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement
    sensor.snzb_02_sensor_*_battery:
      override_measurement: battery
      ignore_attributes:
        - temperature
        - humidity
        - linkquality
        - voltage
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement
    sensor.snzb_02_sensor_*_linkquality:
      override_measurement: linkquality
      ignore_attributes:
        - battery
        - temperature
        - humidity
        - voltage
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement
    sensor.snzb_02_sensor_*_voltage:
      override_measurement: voltage
      ignore_attributes:
        - battery
        - temperature
        - linkquality
        - humidity
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement

(note that I decided to include override_measurement for each, because I actually realised I had both battery and humidity in the “%” measurement, which was what I was trying to avoid). Doing it this way works reasonably well:

“It’s also pretty common where the state is a number for integrations to also have it as an attribute. This is because an entity’s state is always a string value, whereas an attribute can be a number.”

This is very helpful to know - thanks!

“You could choose to add “temperature” to “ignore_attributes”, but having it there as a number is more useful.”

Can you give an example of a situation where the number is more useful than the string in the InfluxDB database? For example, in Grafana, I am able to graph both “value” and “temperature” interchangably (assuming one is a string and one is a float, as you say).

“At the end of the day, does it matter that influx db has a few extra columns?”

Yes, it matters just because the size of the database will grow faster. I have seen this thread, where this issue seemed to have been run into as a result of all the extra attributes.

“However this can be changed to “entity_id” if you want.”

The problem with this is that you can never use the GROUP BY() function to group the data by tags. You can see in the screenshot below that using GROUPBY(entity_id) allows you to automatically populate all the graphs in a single query (which can be very useful if you have dozens of sensors). If each measurement was given by its entity_id, then you would have to individually put the queries in.

I guess one other option would be to actually make use of all the attributes of the temperature entity, and don’t bother sending the other entities (humidity, etc) to InfluxDB. Then call the measurement name something like RHTsensors. Something like this:

influxdb:
  host: 192.168.0.48
  port: 8086
  database: homeassistant
  username: homeassistant
  password: *password*
  include:
    entities:
      - sensor.snzb_02_sensor_1_temperature
      - sensor.snzb_02_sensor_2_temperature
  component_config_glob:
    sensor.snzb_02_sensor_*_temperature:
      override_measurement: RHTsensors
      ignore_attributes:
        - device_class
        - friendly_name
        - state_class
        - unit_of_measurement

which results in this:

Do you have any other thoughts / suggestions based on this, that might be a good way to go?

Thanks very much for your comments.

michaelblight · January 27, 2023, 12:16am

Since “state” in HA is a string, I assumed “value” in InfluxDB was also a string. However, “show field keys” indicates that it is a number (float). I was thinking Grafana was doing the work to convert strings to numbers if you use “value”, but it seems it’s probably the InfluxDB integration doing to conversion. If strings were stored and you manipulated the data yourself, you would need to convert to do min/max/avg etc. This is what I meant by it being more useful.

So yes, if an integration duplicates the state into an attribute, you will get two identical float columns in InfluxDB. But your sensor would have to store 125,000 entries for this extra field to only add 1MB of data - that’s 3 months of data if it updates once a minute (mine seem to update every 5 min at best, so <1MB growth every year).

If you really want to control what gets sent to InfluxDB, you could create your own sensor(s) with just the data you want, and then just send that. This would simplify the InfluxDB config (after 6 months I’d be looking at that “_config_glob” stuff asking myself “why did I do this again?”). One option would be a template sensor, but that’s cumbersome to template a lot of attributes. An MQTT sensor would be less code, if you’re already using that integration - an automation to collect the data from one or more sensors and publish JSON to MQTT, and an MQTT sensor to load it all into attributes. Doing this would make your InfluxDB smaller, but your HA database larger (but only for the ‘purge’ period and then stable).

One integration I use changed their attribute names in a version update, so I ‘lost’ the history in Grafana. Creating my own sensor to send to InfluxDB would have solved this, but my laziness has no solution.

teeeeee · January 30, 2023, 11:03am

Hi Michael, thanks for your response. I understand much better what is going on now, and the difference between entities/attributes. It is useful to be able to use the “ignore_attributes” flag in the InfluxDB config, but I think you are right that the best thing to do if I want full control over which numbers are sent to the database (and avoid duplicates) is to use a custom sensor. This was not something I was aware of before, so thanks for nudging me in that direction.

For example, I am able to extract the humidity attribute only from within the tempearture entity using this kind of custom template sensor, and have everything I need now to go from here:

  - platform: template
    sensors:
      mysensor:
        friendly_name: "NEW HUMIDITY"
        unit_of_measurement: "%"
        value_template: "{{ ( state_attr('sensor.snzb_02_sensor_1_temperature', 'humidity')) }}"

Thanks for the advice.