YAML code generation with GO text templates

Share your Projects!
,
1

Sometimes, you have a bunch of repeated YAML code, more than often template code, that is tedious to maintain. Here is what I have put in place to generate this repeated code.

Concepts

  • The idea here is to use GO text templates, a quite common tool to generate text, and more specifically the gomplate project, which provides a CLI tool around GO templates.
  • Each “project” is comprised of a JSON file that contains data and a template file which is the skeleton of what will be generated
  • I’m using a HA Container installation, so I am re-generating the actual HA YAML “code” at each restart through my docker compose file.
  • The generated YAML files is included trough the include_dir_merge_list <the_dir> syntax; typically, in the_dir, I have both manually crafted and generated YAML.

Example

This is a very specific use-case of mine: After my non-HA backups, I publish to a MQTT topic the result of the backup. I want to have this as sensors in HA, but I have more than a dozen different backups, and don’t want to maintain this manually.

gomplate

In a specific /config/gomplate directory, I have 2 files:

  • The “data”: mqtt_backup_entities.json (shortened for readability)
[
  {
    "src": "cloud",
    "dst": "app",
    "dst_path": "/var/data/backups/borg/cloud",
    "bucket": "cloud",
    "typ": "daily",
    "prf": "linx"
  },
  {
    "src": "cloud",
    "dst": "app",
    "dst_path": "/var/data/backups/borg/cloud",
    "bucket": "cloud",
    "typ": "daily",
    "prf": "cloud2-data"
  },
  {
    "src": "cloud",
    "dst": "app",
    "dst_path": "/var/data/backups/borg/cloud",
    "bucket": "cloud",
    "typ": "daily",
    "prf": "cloud2-docker_files"
  },
  {
    "src": "nas",
    "dst": "nas",
    "dst_path": "/srv/dev-disk-by-uuid-c8febff8-2b75-4755-88ed-7d3c12176008/local_backup/omvbackup/",
    "bucket": "nas-local",
    "typ": "daily",
    "prf": "omv-backup"
  }
]
  • The go template: mqtt_backup_entities.tmpl
## gomplate template
## install:
##   go install github.com/hairyhenderson/gomplate/v4/cmd/gomplate@latest
## docker:
##   docker run -v .:/config hairyhenderson/gomplate:stable -d 'data=file:///config/gomplate/mqtt_backup_entities.json' -f /config/gomplate/mqtt_backup_entities.tmpl -o /config/platforms/mqtt/gmp_backups.yaml
## run: 
##   gomplate -d 'data=mqtt_backup_entitites.json' -f mqtt_backup_entitites.tmpl -o platforms/mqtt/mqtt_backups.yaml

{{- range  $b := (ds "data") }}

- name: "Backup {{ $b.typ }} {{ $b.src }} {{ $b.prf }}"
  unique_id: "backup_{{ $b.src }}_{{ $b.prf | strings.ToLower | strings.ReplaceAll "-" "_" }}"
  state_topic: "backups/{{ $b.src }}/{{ $b.typ }}/{{ $b.prf }}"   
  value_template: '{{ "{{" }} value_json["datetime"]  {{ "}}" }}'
  device_class: timestamp
  json_attributes_topic: backups/{{ $b.src }}/{{ $b.typ }}/{{ $b.prf }}
  json_attributes_template: >
    {
      "type": "{{ $b.typ }}",
      "src": "{{ $b.src }}",
      "dst":  "{{ $b.dst }}",
      "dst_path":  "{{ $b.dst_path }}",
      "bucket":  "{{ $b.bucket }}",
      "prefix":  "{{ $b.prf }}",
      "status":  "{{ "{{" }} value_json['status']  {{ "}}" }}"
    }
    
{{- end }}

docker compose

version: '2.1'
services:
    gomplate_mqtt_backups:
        image: hairyhenderson/gomplate:stable
        volumes:
            - ${HA_PATH}/config:/config
        command: -d 'data=file:///config/gomplate/mqtt_backup_entities.json' -f /config/gomplate/mqtt_backup_entities.tmpl -o /config/platforms/mqtt/gmp_backups.yaml
[...]
    homeassistant:
        container_name: home-assistant
        depends_on:
          gomplate_mqtt_backups:
            condition: service_completed_successfully
[...]
  • the result: a /config/platforms/mqtt/gmp_backups.yaml file is generated:
## gomplate template
## install:
##   go install github.com/hairyhenderson/gomplate/v4/cmd/gomplate@latest
## docker:
##   docker run -v .:/config hairyhenderson/gomplate:stable -d 'data=file:///config/gomplate/mqtt_backup_entities.json' -f /config/gomplate/mqtt_backup_entities.tmpl -o /config/platforms/mqtt/gmp_backups.yaml
## run: 
##   gomplate -d 'data=mqtt_backup_entitites.json' -f mqtt_backup_entitites.tmpl -o platforms/mqtt/mqtt_backups.yaml

- name: "Backup daily cloud linx"
  unique_id: "backup_cloud_linx"
  state_topic: "backups/cloud/daily/linx"   
  value_template: '{{ value_json["datetime"]  }}'
  device_class: timestamp
  json_attributes_topic: backups/cloud/daily/linx
  json_attributes_template: >
    {
      "type": "daily",
      "src": "cloud",
      "dst":  "app",
      "dst_path":  "/var/data/backups/borg/cloud",
      "bucket":  "cloud",
      "prefix":  "linx",
      "status":  "{{ value_json['status']  }}"
    }

- name: "Backup daily cloud cloud2-data"
  unique_id: "backup_cloud_cloud2_data"
  state_topic: "backups/cloud/daily/cloud2-data"   
  value_template: '{{ value_json["datetime"]  }}'
  device_class: timestamp
  json_attributes_topic: backups/cloud/daily/cloud2-data
  json_attributes_template: >
    {
      "type": "daily",
      "src": "cloud",
      "dst":  "app",
      "dst_path":  "/var/data/backups/borg/cloud",
      "bucket":  "cloud",
      "prefix":  "cloud2-data",
      "status":  "{{ value_json['status']  }}"
    }

- name: "Backup daily cloud cloud2-docker_files"
  unique_id: "backup_cloud_cloud2_docker_files"
  state_topic: "backups/cloud/daily/cloud2-docker_files"   
  value_template: '{{ value_json["datetime"]  }}'
  device_class: timestamp
  json_attributes_topic: backups/cloud/daily/cloud2-docker_files
  json_attributes_template: >
    {
      "type": "daily",
      "src": "cloud",
      "dst":  "app",
      "dst_path":  "/var/data/backups/borg/cloud",
      "bucket":  "cloud",
      "prefix":  "cloud2-docker_files",
      "status":  "{{ value_json['status']  }}"
    }

- name: "Backup daily nas omv-backup"
  unique_id: "backup_nas_omv_backup"
  state_topic: "backups/nas/daily/omv-backup"   
  value_template: '{{ value_json["datetime"]  }}'
  device_class: timestamp
  json_attributes_topic: backups/nas/daily/omv-backup
  json_attributes_template: >
    {
      "type": "daily",
      "src": "nas",
      "dst":  "nas",
      "dst_path":  "/srv/dev-disk-by-uuid-c8febff8-2b75-4755-88ed-7d3c12176008/local_backup/omvbackup/",
      "bucket":  "nas-local",
      "prefix":  "omv-backup",
      "status":  "{{ value_json['status']  }}"
    }

inclusion

The generated file (and others) is included through configuration.yaml with:

[...]
mqtt: 
  sensor: !include_dir_merge_list platforms/mqtt/
[...]