InfluxDB exclude.yaml

None of that explains exclude.yaml

ok and what’s the difference in configuration.yaml and exclude.yaml ? … where ever i’ve looked, it’s same syntax

ahh, sorry missed some of your question … yes make your own xxx.yaml refer to it in configuration.yaml

that’s what I was asking

I guess I’ll just use trial and error. I haven’t seen any examples, or I misunderstood what was posted. Where have you seen the syntax for exclude.yaml?

exclude.yaml, it’s just what you name it, can call it what ever, as long as there is an !include in configuration

I guess I need to find another way to ask my questions since I’m getting explanations that aren’t congruent with what I am trying to understand.

1 Like

you use “extra” xxx.yaml files just to make life easier, not a big mess and huge configuration.yaml file, that’s why you can make additional yaml-files for “separate” purposes

edit: in short, exclude.yaml is not different from configuration.yaml … and yes you place that file in /config

yes, you should have search for how to make yaml file, or read the doc about yaml-file, maybe

It seems like you’re looking for a way to make a file called exclude.yaml where you put something in it and immediately the behavior of the InfluxDB integration changes. That doesn’t exist.

The way you exclude entities from being recorded in Influx is by adding include and exclude options to the influxdb configuration in your configuration.yaml. Those options are documented here. As @boheme61 noted, it works just like the recorder integration.

If you want, you can move this piece of the InfluxDB integration into a separate file as you can with anything else. See splitting up the configuration for the ways you can do that. If you really want a file called exclude.yaml where you put your exclude filters for example then you can can put this in your configuration.yaml (or wherever you have influxdb defined):

influxdb:
  exclude: !include exclude.yaml

This file doesn’t have to be called exclude.yaml though, there’s no required name. It can be called my_giant_list_of_stuff_to_exclude_from_influxdb.yaml if you want, doesn’t matter as long as you !include it where you want to use it.

2 Likes

I did multiple searches before posting. So how does it help now that you tell me I should have searched when I already had?

Looking at more documentation just makes me even more confused

You realize that every time someone clicks a link in this forum the link shows a count of how many people clicked on it next to it right? :wink:

Anyway here’s the example from the docs showing how to do exclude stuff for influx:

influxdb:
  host: 192.168.1.190
  port: 20000
  database: DB_TO_STORE_EVENTS
  username: MY_USERNAME
  password: MY_PASSWORD
  ssl: true
  verify_ssl: true
  max_retries: 3
  default_measurement: state
  exclude:
    entities:
       - entity.id1
       - entity.id2
    domains:
       - automation
  include:
    entities:
       - entity.id3
       - entity.id4
  tags:
    instance: prod
    source: hass

With this configuration state changes to entity.id1, entity.id2 and any automations will not be recorded in Influx. So that’s what you need to do.

Now if we want we can modify this example like this:

influxdb:
  host: 192.168.1.190
  port: 20000
  database: DB_TO_STORE_EVENTS
  username: MY_USERNAME
  password: MY_PASSWORD
  ssl: true
  verify_ssl: true
  max_retries: 3
  default_measurement: state
  exclude: !include exclude.yaml
  include:
    entities:
       - entity.id3
       - entity.id4
  tags:
    instance: prod
    source: hass

And then in exclude.yaml put this:

entities:
  - entity.id1
  - entity.id2
domains:
  - automation

These two do the exact same thing.

4 Likes

CentralCommand Mike spelled out how to do it in his post.
either include all by default, then put the ones you don’t want in exclude file
or exclude all by default and then put the ones you want in an include file
or a mix of the two.

What does that have to do with anything? I click on a lot of threads for many different reasons. Why does it matter that it has the number of clicks?

What I quoted is really all I needed to know from the beginning. I acknowledged earlier that I need to find another way to ask my questions. If I had that would have been the first response without all of the other information included in the the responses.

The reason I only care about exclude is because only including exclude means everything else is included. At least that is how I understand Excludes, no includes - only exclude specified entities

From the influxdb doc:

By default, no entity will be excluded. To limit which entities are being exposed to InfluxDB, you can use the include and exclude parameters.
Filters are applied as follows:

  1. No includes or excludes - pass all entities
  2. Includes, no excludes - only include specified entities
  3. Excludes, no includes - only exclude specified entities

I know I’m kind of beating a dead horse here but it is kind of spelled out in the doc. Still I do acknowledge that you found the doc tough (and I know it is a bit dense at times) so if you have any suggestions for how we can improve it please let me know. I am actually the code owner of this particular integration so I tried to make it understandable but there is a lot of info, especially with the two different versions.

We probably shouldn’t clog up this particular thread more so feel free to either DM me or just make a PR/issue in the docs repo and @ me (I’m mdegat01 on github).

Sorry bad joke. You replied to me looking at more doc made you more confused but I was noticing that it looked like the links I shared hadn’t been clicked. Doesn’t matter, we got there in the end.

1 Like

What is spelled out in the doc? What part of my previous statements are you referring to?

I’ll reiterate. All I needed was excluded entities. I did not need included entities. I also wanted the excluded entities to be in a exclude.yaml file so that the long list was in a separate file. You did answer that part but it took a couple of posts including one that had more information than I needed. Now the part you quoted above does not mention exclude.yaml. I does have what I had copied and pasted which is ‘Excludes, no includes - only exclude specified entities’ To me that means everything except what I excluded will be included. If that is not it means then there needs to be clarification. If it needs more clarification then you are wrong about it being spelled out. Or are you saying something else was spelled out or are you trying to say I was right about the meaning but you still had to beat your chest to say it was there the whole time. I really hope it wasn’t the last.

You need to remember that not everyone understands everything the same. I have even read about engineers or software developers or computer scientists getting confused about many aspects of HA or yaml or some add-on such Node-red.

It looks like the counter is broke. Either way I had already read the documentation before creating the thread. You can see how that went.

@CentralCommand, thank you for the detailed explanation in the above thread. I want you to know it was very clear (even for someone who is “not a coder”) and is greatly appreciated by most everyone.

Hi all, might be worth asking a question here since there seem to be experience with include exclude in influx.

I have size issue with my influx as it is ever growing. I am trying to change from exclude to include but I experience troubles… it’s not working as expected and before I change “everything” I need to be sure the mechanism works. Do any of you have experience with this?

Like ‘sender’ in their unanswered post above, my database is getting very big as are the backups because of it (17Gb). Previously nothing was excluded/included and retention was infinite. I’ve now set that to 800d; I’ve been running HA for 4 years now.
I recognise that I also need to start using includes/excludes but to be honest I don’t know where to start. I must have thousands of entities now as lots of my integrations, like cameras, seem to add 50 or so at a time. Most of the stuff I don’t want retained, but things like temperatures, watts, daily costs etc i do.
Can you give or point to guidance on best approaches?
I have a similar issue with entities that I want exposed to Alexa and Google Home!

Edit:

I took the plunge and decided to use include: instead of exclude:. My influxbd entry in the configuration.yaml file now looks like this…

influxdb:
  host: a0d7b954-influxdb
  port: 8086
  database: homeassistant
  username: grafanauser
  password: !secret influxpwd
  max_retries: 9
  default_measurement: state
  include: !include influxdb_include.yaml

and the influxdb_include.yaml file looks like.

entities:
  - sensor.indoor_temperature_0
  - sensor.temperature_1
  - sensor.nsphall_temperature_2
  - sensor.hot_tub_temperature_delta
  - sensor.heating_setpoint
  - input_number.required_temperature
  - sensor.indoor_humidity_0
  - sensor.humidity_1
  - sensor.boiler_burn_since_new
  - sensor.burn_count_by_day
  - sensor.daily_radiation_count
  - sensor.daily_background_radiation_dose
  - sensor.processor_use
  - sensor.disk_use
  - sensor.geiger_cpm
  - sensor.geiger_usvph
entity_globs:
  - sensor.*temperature
  - sensor.*humidity
  - sensor.*energy
  - sensor.*energy_today
  - sensor.*energy_yesterday
  - sensor.*energy_total
  - sensor.*power
  - sensor.*power_left
  - sensor.*power_right
  - sensor.*litres*
  - sensor.oil_tank_remaining*
  - sensor.oil_kwh*
  - sensor.*cost*
  - sensor.*wind_speed  
  - sensor.boiler_runtime*
  - sensor.web_scrape*
  - input_number.meter_reading_electricity*
  - sensor.*pressure
  - sensor.speedtest*
  - sensor.*kib_s*
  - sensor.daily_bytes*
  - sensor.*total_?x_gb
domains:

Not too many entries after all by using the * wildcard. I spent a day going through my Grafana dashboards making sure I included all the entities the graphs needed. I do wish I had a decent naming convention in place when I started on this HA journey 4 years ago.

One thing I found out is that the ‘Developer Tools/YAML/Restart/Quick Reload’ doesn’t get through to InfluxDB - a full reboot is needed!

My final issue now is that having reduced what’s included AND having set a retention of 800 days, I expect the database size to gradually drop over the next 800 days. Unfortunately I don’t think that happens and I don’t know how to force a purge. If anyone does know then please tell me.

Includes, no excludes - only include specified entities

This is not working. If I have a filter that only contains one entity to be included, the influxdb add-on stored everything.

This is my filter. In this example I only wanted to include the entity sensor.bezug. But no success.

influxdb:
  username: xxxx
  password: xxxx
  tags:
    source: HA
  tags_attributes:
    - friendly_name
  default_measurement: state
  include:
    entities:
      - sensor.bezug