Including files to clean up configuration yaml. Help needed

Hello everyone…
I normally think that i am smart and normally see structures pretty easy, but for some reason this !include thing does not stick to me well
I have so much stuff that i would like to store in a sepperate file and just include it, but as soon i copy the code into a file and try to include it, for example battery : !include battery.yaml i either get integration not found or something similar to this: Invalid config for [sensor]: required key not provided @ data[‘platform’]. Got None. (See /config/configuration.yaml, line 11).
do those variables before the “:” have to be known types or can they be fictios?
And why does the code work when in the configuration.yaml, but a 1:1 copy into a different file and an include on top of the configuration.yaml does not?
Does the syntax need to be changed in the external file?
As far i understood the subject so far is, that the include just copies the code into the configuration.yaml in the background…

That is exactly what is happening, but in different formats. :wink: So it does matter very much how you include things. YAML is very picky with indentations, and merging two YAML files is not trivial (at least not software wise).

It does matter if you include a file as a list, or if many files should be included or if a folder with files should be included. For all these variants there are specific commands on how to include, see here.

But, there is something to your rescue, that won’t solve your actual problem, but it makes including files and things easier:

Thx for your answer.

So instead of include i should do the include merge thing than?
I really need to declutter this file cause the whole weather and apc stuff makes it several pages long and that is very difficult to manage. Hated spaghetti code already in my c and c++ times

So where does the syntax have to be changed to just copy content ibto a different file and include its logic?

Is that somewhere exampled?

Thx alot again

Don’t get me wrong, but you will have to read up on this a lot more, before you decide what to do. :wink: This is an essential part of your installation, you’ll likely stick with the system you choose now for years, so be cautious. :wink: I know, not what you wanted to hear, but trust me, you will want to give some more thought into this. :slight_smile:

The examples for the different includes are in the first link I posted above. :wink:

It all depends on how you want to work from now on. That’s why I suggested packages. But this is your decision to make, not mine. We can help in implementing whatever way you choose, but you have to choose one upfront => you need to know at least the basics about splitting up the config or using packages.

Take a look here and here (#14) for some more explanations. :wink:

Have to somehow find the aha moment to see what is needed, cause right now i just see lots of question marks. i think i do it the way it is supposed to be but being not even close.
Very frustrating.
Already had to give up on the include secrets since it never worked. All pw are clear text in the config yaml. Not very favorable
Will try to find i fo that helps me

Thx

Have you read the links I gave in the last post? :wink: Especially the first one? I think I explained packages there very well. Hint: I can see that the linked wasn’t clicked even once. :rofl:

Anyway, if you have lots of question marks, than ask these questions. :slight_smile: As you see, I and others are willing to help, but you give us nothing to work on. From my POV you don’t know what to do, and want someone to decide what you should do. That can’t work, sorry! :slight_smile:

As I said, I linked to that specific post from myself, because there are packages explained. Meaning, it’s the way I’ve structured my config, but this is me. Others use packages totally different.

And to know, if you’re close, you should post some YAML, as otherwise we have no idea, what you’re talking about. :wink: Same goes for the secrets file. Don’t tell me - show me, what you’re doing, and if it doesn’t work the way you want it to, give some notes what you want to achieve. And please, don’t say “I want my config split up”, you will need to be more specific, like “I have sensor A, B and C. How do I get these sensors into a seperate file? This is my configuration.yaml now: <CODE_HERE>”. :slight_smile: You get the idea.

I know how that must sound to you, like I don’t want to give any concrete advice, but it is a little different: the ways on how to get you going are that wide ranged, that I’d need to narrow down, where you want to go (and possibly where you come from config wise). And for now, I honestly have not much to go on.

TL;DR Post some of your code, preferably without passcodes or other private information, and tell me exactly, what you want to achieve! :slight_smile:

PS: And I’m sure, we can find a way for your secrets file as well, but as above, show some code. Trust me, it won’t hurt. :slight_smile: :wink: :laughing:

From reading through the package stuff i decided to give at least secrets another chance.
And guess what, i seem to have fallen to the same mistake most did. Included the secrets.yaml file even though you dont have to. But that wasnt all that clear.
What i want is kinda straight forward.
Take voluminous sections like weather or apc stuff out of configuration.yaml and separate it out to own files and include them back in for funkction.
Example? Ok…just didnt want to spam lots of code here, but here we go:
This is the content of battery.yaml

#UPS test section start

sensor:
  - platform: snmp
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.2021.10.1.3.1
    
  - platform: snmp
    name: ups_smart_capacity
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.2.2.1.0
    accept_errors: true
    unit_of_measurement: '%'
  - platform: snmp
    name: ups_smart_runtime_remaining
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.2.2.3.0
    accept_errors: true
    value_template: >-
      {% set time = (value | int) / 100 | int %}
      {% set minutes = ((time % 3600) / 60) | int %}
      {% set hours = ((time % 86400) / 3600) | int %}
      {% set days = (time / 86400) | int %}
    
      {%- if time < 60 -%}
        Less than a minute
        {%- else -%}
        {%- if days > 0 -%}
          {{ days }}d
        {%- endif -%}
        {%- if hours > 0 -%}
          {%- if days > 0 -%}
            {{ ' ' }}
          {%- endif -%}
          {{ hours }}h
        {%- endif -%}
        {%- if minutes > 0 -%}
          {%- if days > 0 or hours > 0 -%}
            {{ ' ' }}
          {%- endif -%}
          {{ minutes }}m
        {%- endif -%}
      {%- endif -%}
  - platform: snmp
    name: ups_smart_load_percentage
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.4.2.3.0
    accept_errors: true
    unit_of_measurement: '%'
  - platform: snmp
    name: ups_smart_battery_temperature
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.2.2.2.0
    accept_errors: true
    unit_of_measurement: '°C'
  - platform: snmp
    name: ups_smart_battery_status
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.2.2.4.0
    accept_errors: true
    value_template: >-
      {%if value == '1' %}
        Good
      {% elif value == '2' %}
        Failed
      {% endif %}
  - platform: snmp
    name: ups_smart_type
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.1.1.1.0
    accept_errors: true
  - platform: snmp
    name: ups_smart_input_voltage
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.3.2.1.0
    accept_errors: true
    unit_of_measurement: 'V'
  - platform: snmp
    name: ups_smart_last_transfer_reason
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.3.2.5.0
    accept_errors: true
    value_template: >-
      {%if value == '1' %}
        No events
      {% elif value == '2' %}
        High line voltage
      {% elif value == '3' %}
        Brownout
      {% elif value == '4' %}
        Loss of mains power
      {% elif value == '5' %}
        Small temporary power drop
      {% elif value == '6' %}
        Large temporary power drop
      {% elif value == '7' %}
        Small spike
      {% elif value == '8' %}
        Large spike
      {% elif value == '9' %}
        UPS self test
      {% elif value == '10' %}
        Excessive input voltage fluctuation
      {% endif %}
  - platform: snmp
    name: ups_smart_output_load
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.4.2.3.0
    accept_errors: true
    unit_of_measurement: '%'
  - platform: snmp
    name: ups_smart_output_current
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.4.2.4.0
    accept_errors: true
    unit_of_measurement: 'A'
  - platform: snmp
    name: ups_smart_last_self_test_result
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.7.2.3.0
    accept_errors: true
    value_template: >-
      {%if value == '1' %}
        OK
      {% elif value == '2' %}
        Failed
      {% elif value == '3' %}
        Invalid Test
      {% elif value == '4' %}
        Test In Progress
      {% endif %}
  - platform: snmp
    name: ups_smart_last_self_test_date
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.7.2.4.0
    accept_errors: true
  - platform: snmp
    name: ups_smart_communication_status
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.8.1.0
    accept_errors: true
    value_template: >-
      {%if value == '1' %}
        on
      {% elif value == '2' %}
        off
      {% endif %}
  - platform: snmp
    name: ups_smart_status
    host: 192.168.178.102
    baseoid: 1.3.6.1.4.1.318.1.1.1.4.1.1.0
    accept_errors: true
    value_template: >-
      {%if value == '1' %}
        Unknown
      {% elif value == '2' %}
        On Line
      {% elif value == '3' %}
        On Battery
      {% elif value == '4' %}
        On Smart Boost
      {% elif value == '5' %}
        Timed Sleeping
      {% elif value == '6' %}
        Software Bypass
      {% elif value == '7' %}
        Off
      {% elif value == '8' %}
        Rebooting
      {% elif value == '9' %}
        Switched Bypass
      {% elif value == '10' %}
        Hardware Failure Bypass
      {% elif value == '11' %}
        Sleeping Until Power Returns
      {% elif value == '12' %}
        On Smart Trim
      {% endif %}

#UPS test section 

And this the include for it
Battery: !include battery.yaml

Error resulting is: integration battery not found
Since i only want this logic included and do not call it anywhere else in the code i wonder what i might do wrong here.

Ok, so we can make a check mark on the secrets, these are working! :white_check_mark: :slight_smile:

Let’s see, if we can solve the next problem as well. :wink: I take your file as example.

If you take all the posted code and “outsource” it, you would need to include the specific domain in your configuration, not a name you choose.

# configuration.yaml
sensor: !include battery.yaml

# battery.yaml
- platform: xxx

See, no domain (sensor) in the included file, because you include a list. But that gives you the problem, that you don’t have the sensor domain available to include from another file. So a fans.yaml with sensors in it, wouldn’t work.

You can now split this in another way, like include sensors.yaml and list all sensors there.

Or you could use directory includes. You can include one folder (with subs) as a list or as a directory, so you could have something like this

# configuration.yaml
sensor: !include_dir_xxx

# ./sensors/battery.yaml
- platform: xxx

# ./sensors/fans.yaml
- platform: xxx

See the examples I linked in my first post, there are the directory includes explained with some examples. I cut this short, because from your naming of the file, I think you’d be a candidate for packages like me. :wink:

Packages give you the benefit, that you can mix and match everything you want, which gives a more logical approach to sorting your things. Packages work in another way of inclusion.

# configuration.yaml
homeassistant:
  packages: !include_dir_named packages

# ./packages/battery_usp.yaml
sensor:
  - platform: xxx

binary_sensor:
   - platform: xxx

whatever_component:
  - platform: xxx

# ./packages/battery_clocks.yaml
sensor:
  - platform: xxx

binary_sensor:
   - platform: xxx

whatever_component:
  - platform: xxx

As you can see, in package files you can use every domain you want, as long as it is only used once in each file. You don’t even need to include any of the domains in configuration.yaml.

See the first example in my second post on how I split up my packages.

Hope this helps! :wink:

And let me know how things work out for you, or ask if something isn’t clear. :slight_smile:

Hello.

Thx alot that u r taking so much time to shed some light here.
See, i wasnt aware that the “sensor” thing wasnt freename choice and that for each “domain” as u called it is only one call possible.
That already helps alot.
How many “domains” are there?
Is there a list to choose from?
I will try to play and try around with thos answers now and hopefully come back with an heureka report answer
Thx again

See for yourself, what is running in your HA instance. :slight_smile: Go to Developer Tools > Template and paste this instead of the demo code:
All domains used:

- {{ states|map(attribute='domain')|unique|list|join('\n- ') }}

Or all available entities:

- {{ states|selectattr('state', 'ne', 'unavailable')|map(attribute='entity_id')|list|join('\n- ') }}

Shamelessly copied from here

You will likely don’t need to choose from a list of domains, as these are mostly determined by the integration you want to install. From your example, you get something offered from your UPS integration, in your case sensors. Eg. you can’t change yourself what the integration provides, so you can’t make a binary_sensor out of a sensor provided by an integration. There are ways to work around this (hint: template sensors), but that is beyond the scope of this topic here.

On a sidenote, if you want to dig deeper inside these things, I’d recommend to install Spook. It’s from Frenck, one of the main developers of HA. There you can get some more statistic values about entities. You can get it via HACS just search for Spook. The docu is here. But this is unrelated to your config questions, it is just a recommendation in general for statistic fans. :slight_smile:

PS: Bookmark this post, you’ll likely need that domain command when you take a look at your history and database sizes in the near future. :rofl: :rofl: See here. :slight_smile:

Hello again…packages work right away with the include dir part…
Battery and weather stuff is outsourced and survived a reboot of host machine…
Finally !!\

Thx!

You’re welcome! Glad I could help! :slight_smile: