YAML `<<: !include` or package substitution variables?

samburner3 · November 18, 2022, 1:02am

Hi all,

I am building this modded light switch project using esphome-state-machine.

I have separated out my state machine logic from my main config file like so:

# master_bedroom_closet.yaml

# State machine(s) setup
<<: !include state_machine_light_pir/state_machine_external_components.yaml

state_machine:
  - name: Light Brightness State Machine 1
    id: sm1
    <<: !include state_machine_light_pir/state_machine.yaml

# state_machine_light_pir/state_machine.yaml

    initial_state: "OFF_STANDBY"
    states:
      - name: "OFF_STANDBY"
        on_enter:
          - light.turn_off: light1
          - light.turn_on:
              id: light_led1
              brightness: 30%
      - name: "OFF_STANDBY_TIMEOUT"
        on_enter:
          - light.turn_off: light1
          - light.turn_on:
              id: light_led1
              brightness: 50%
          - script.execute: delayTimeout
          - script.wait: delayTimeout
          - state_machine.transition:
              id: sm1
              input: TIMEOUT
        on_leave:
          - script.stop: delayTimeout
         ...

This works well for a light switch with a single gang (1 button, 1 relay etc).

However I now want to use this same state machine model for a 3 gang switch (3 buttons, 3 relays etc).

In the state_machine.yaml you can see I have hardcoded the light name, ie light1, light_led1 etc.

I can’t figure out how to dynamically import this state_machine.yaml model 3 times, each instance with different variables. I would hope something like this would work:

# master_bedroom_closet.yaml

# State machine(s) setup
<<: !include state_machine_light_pir/state_machine_external_components.yaml

state_machine:
  - name: Light Brightness State Machine 1
    id: sm1
    <<: !include state_machine_light_pir/state_machine.yaml - 
        {light_id : 'light1', light-led_id : 'light_led1', sm_id : 'sm1'}
  - name: Light Brightness State Machine 2
    id: sm2
    <<: !include state_machine_light_pir/state_machine.yaml
        {light_id : 'light2', light-led_id : 'light_led2', sm_id : 'sm2'}
  - name: Light Brightness State Machine 3
    id: sm3
    <<: !include state_machine_light_pir/state_machine.yaml
        {light_id : 'light3', light-led_id : 'light_led3', sm_id : 'sm3'}

# state_machine_light_pir/state_machine.yaml

    initial_state: "OFF_STANDBY"
    states:
      - name: "OFF_STANDBY"
        on_enter:
          - light.turn_off: ${light_id)
          - light.turn_on:
              id: ${light-led_id)
              brightness: 30%
      - name: "OFF_STANDBY_TIMEOUT"
        on_enter:
          - light.turn_off: ${light_id)
          - light.turn_on:
              id: ${light-led_id)
              brightness: 50%
          - script.execute: delayTimeout
          - script.wait: delayTimeout
          - state_machine.transition:
              id: ${sm_id)
              input: TIMEOUT
        on_leave:
          - script.stop: delayTimeout
         ...

This way I would not have to keep 3 different identical instances of state_machine.yaml, and when I update the master model, it just works for all instances.

I know we can use substitutions however this I don’t think works when trying to import 3 instances of the same model file with different variables for each instance, as substitutions is a global find and replace.
I have read through the configuration-types page quite a few times in detail and can’t see an option to achieve what I am after here.

If I can do this, it would also be very beneficial i imagine for having more control over 1 set of firmware for multiple similar devices.

justone · November 18, 2022, 2:49pm

in “# state_machine_light_pir/state_machine.yaml”
the following line looks pretty strange to me:
-name light.turn_off: $ { light_id )
Normally the type of opening bracket has to be followed by the same bracket type for closing.

And yes a substitution is sort of a replacement … but If I do think about a variable … I can say i=0 and then i=1 and at another stage i=10 … which also means i was given differnt values multiple times. No problem.

glyndon · November 18, 2022, 2:59pm

I can’t codify/exemplify this idea yet, but what if you import the state machine just once, and reference it from 3 instances of the light, and somehow ‘pass’ the target to it in globals (presuming the whole system is single-threaded, interrupt service routines are atomic, and that therefore no race conditions are likely to arise)?
It might entail changing some of your state-machine code into a lambda, to make use of the globals’ content as pointers to the subject instance’s functions.

muxa · November 18, 2022, 11:58pm

This would be easy if packages supported ability to load multiple instances of the same package with per-instance substitution, e.g:

packages:
  - state_machine: !include state_machine_light_pir/state_machine_external_components.yaml
    substitutions:
       light_id: 'light1'
       light-led_id : 'light_led1'
       sm_id : 'sm1'
  - state_machine: !include state_machine_light_pir/state_machine_external_components.yaml
    substitutions:
       light_id: 'light2'
       light-led_id : 'light_led2'
       sm_id : 'sm2'

But unfortunately this is not supported.

muxa · November 19, 2022, 12:16am

A possible workaround is to have state_machine_light_pir/state_machine.yaml as a template, and then create 3 copies with substitutions and import those copies individually, e.g:

state_machine_template.yaml:


    initial_state: "OFF_STANDBY"
    states:
      - name: "OFF_STANDBY"
        on_enter:
          - light.turn_off: ${light_id}
          - light.turn_on:
              id: ${light_led_id}
              brightness: 30%
      - name: "OFF_STANDBY_TIMEOUT"
        on_enter:
          - light.turn_off: ${light_id}
          - light.turn_on:
              id: ${light_led_id}
              brightness: 50%
          - script.execute: delayTimeout
          - script.wait: delayTimeout
          - state_machine.transition:
              id: ${sm_id}
              input: TIMEOUT
        on_leave:
          - script.stop: delayTimeout

Run in terminal:

light_id=light1 light_led_id=light_led1 sm_id=sm1 envsubst < state_machine_template.yaml > state_machine_1.yaml
light_id=light2 light_led_id=light_led2 sm_id=sm2 envsubst < state_machine_template.yaml > state_machine_2.yaml
light_id=light3 light_led_id=light_led3 sm_id=sm3 envsubst < state_machine_template.yaml > state_machine_3.yaml

And your device config:

state_machine:
  - name: Light Brightness State Machine 1
    id: sm1
    <<: !include state_machine_1.yaml
  - name: Light Brightness State Machine 2
    id: sm2
    <<: !include state_machine_2.yaml
  - name: Light Brightness State Machine 3
    id: sm3
    <<: !include state_machine_3.yaml

mulcmu · November 19, 2022, 2:44am

It looks to me like this functionality was added but the documentation associated with it was not.

github.com/esphome/esphome

Add variable substitutions for !include

esphome:dev ← jimtng:include-param

opened 11:27PM - 28 May 22 UTC

jimtng

+95 -17

# What does this implement/fix? Support variable substitutions in !include … ## Types of changes - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Other **Related issue or feature (if applicable):** fixes <link to issue> **Pull request in [esphome-docs](https://github.com/esphome/esphome-docs) with documentation (if applicable):** esphome/esphome-docs#2100 ## Test Environment - [ ] ESP32 - [ ] ESP32 IDF - [ ] ESP8266 ## Example entry for `config.yaml`: Included `includetest.yaml` to demonstrate the functionality ## Checklist: - [x] The code change is tested and works locally. - [x] Tests have been added to verify that the new code works (under `tests/` folder). If user exposed functionality or configuration variables are added/changed: - [x] Documentation added/updated in [esphome-docs](https://github.com/esphome/esphome-docs).

So try this:

# master_bedroom_closet.yaml

# State machine(s) setup
<<: !include state_machine_light_pir/state_machine_external_components.yaml

state_machine:
  - name: Light Brightness State Machine 1
    id: sm1
    <<: !include {file: state_machine_light_pir/state_machine.yaml, vars:  
        {light_id : 'light1', light-led_id : 'light_led1', sm_id : 'sm1'}  }
  - name: Light Brightness State Machine 2
    id: sm2
    <<: !include {file: state_machine_light_pir/state_machine.yaml, vars: 
        {light_id : 'light2', light-led_id : 'light_led2', sm_id : 'sm2'}  }
  - name: Light Brightness State Machine 3
    id: sm3
    <<: !include {file: state_machine_light_pir/state_machine.yaml, vars:
        {light_id : 'light3', light-led_id : 'light_led3', sm_id : 'sm3'}  }

samburner3 · November 19, 2022, 4:17am

Oh amazing
I’ll try this ASAP.

As apart of a PR should be a requirement to add new features to docs, could have saved me a week

Thanks so much!!

muxa · November 19, 2022, 10:00pm

Thanks for the pointer. I found documentation for it here: Frequently Asked Questions — ESPHome

justone · November 22, 2022, 4:00pm

what makes me wonder … did I miss something or is it still not possible to define substitutions or vars being of type number?

mulcmu · November 28, 2022, 12:32pm

The original PR that added the functionality for !include variable substitutions did add the documentation to the FAQ section. It was just real easy to miss. The substitution section of the esphome.io docs is now updated.