Help with mqtt templating (from Domoticz)

Good morning,
This interests me a lot
Is it possible to have a little more explanation?
I am new to HA!

Could you possibly be more specific?

Hello,
I have Domoticz which has been running for a few years
Now I’m testing HA
I would like via HA-MQTT and Dz-MQTT to take the data from Dz and transmit it to HA
And if possible also order a lighting switch from HA to Dz via MQTT
For the moment, I don’t want to dismantle all my home automation domoticz without knowing!

I did the same a couple of years ago (and now switched over fully to HA, which is much better!).
You have to use mqtt sensors with templating to catch the the data from Domoticz and update the status of a e.g. a light in HA. And you can also send data/commands back to Domoticz as well of course in a similar way.
It is a steep learning curve however to learn about templating, mqqt-sensors etc in HA.
In the end, just swiching over to HA is much easer by the way :wink:

See my first steps on this in this topic. But also the more recent posts here, that are more advanced ways of doing this.

1 Like

Same here. Synced some sensors in the beginning, but switched step by step completely to HASS and I love it. I was a big Domoticz fan for years, but when you get to know HASS, it’s way more powerful.

Thank you for your opinions.
It doesn’t look obvious to me though!
:wink:

I will try to help you with intercommunication between DZ and HA. I’ve got a fairly large installation, with quite a number of scripts either in bash, LUA, dzVents or python. Based on the fact that communication between DZ and HA is rock solid, I will not migrate everything to HA, even if I’m convinced it is doable, but to me it is not worth the effort. On the contrary, when a device is integrated in HA I keep both installations in sync. Mileage may vary. Currently there are 190 manually configured MQTT entities to which I have added the new entities that were added by HA integration. For those, it is a different approach, if needed I will publish slice of code (there are 36 such entities)

So first you need to set-up an MQTT broker. I already had Mosquitto installed on an RPi, you could very well install HA addon (not to be confused with MQTT integration). Start simple: no user/password, no encryption
Secondly, you need a swiss knife, I mean MQTT explorer that helps greatly to understand who is emitting what.

Now configuration
On DZ side: add MQTT gateway, here is my configuration

On HA side: add MQTT integration (don’t confuse with MQTT broker addon). As you’ll see it is pretty straightforward to set-up

Then add Manually configured entities, there is a configuration slice for each device you want to have in HA
In configuration.yaml add:

mqtt: !include_dir_merge_named mqtts/

and add folders as herebelow
image

It helps to keep each family in a separate folder

Depending on each category of device the configuration is slightly different.
Binary sensors (excerpt)

binary_sensor:
# Etat ouverture fenêtre salle de bains
  - name: "Fenêtre salle de bains"
    object_id: "658"
    unique_id: "658"
    device_class: window
    state_topic: "domoticz/out/658"
    value_template: "{{ value_json.nvalue }}"
    payload_on: 1
    payload_off: 0
    qos: 0
# Etat de toutes les ouvertures
  - name: "Ouvertures général"
    object_id: "530"
    unique_id: "530"
    device_class: window
    state_topic: "domoticz/out/530"
    value_template: "{{ value_json.nvalue }}"
    payload_on: 1
    payload_off: 0
    qos: 0

Buttons (excerpt)

button:
  - name: up
    object_id: "339_up"
    unique_id: "339_up"
    icon: mdi:chevron-up
    command_topic: "domoticz/in"
    command_template: '{"command": "switchlight", "idx": 339, "switchcmd": "Set Level", "level": 40 }'
    qos: 0
    retain: false
  - name: down
    object_id: "339_down"
    unique_id: "339_down"
    icon: mdi:chevron-down
    command_topic: "domoticz/in"
    command_template: '{"command": "switchlight", "idx": 339, "switchcmd": "Set Level", "level": 60 }'
    qos: 0
    retain: false

Covers (excerpt)

cover:
  # Création écran cinéma ouvert/fermé 
  - name: "Ecran cinéma"
    object_id: "29"
    unique_id: "29"
    state_topic: "domoticz/out/29"
    command_topic: "domoticz/in"
    value_template: "{{ value_json.nvalue }}"
    state_closed: "1"
    state_open: "0"
    payload_open:  '{"command": "switchlight", "idx": 29 , "switchcmd": "Off" }'
    payload_close: '{"command": "switchlight", "idx": 29, "switchcmd": "On" }'
    optimistic: false
    qos: 0
    retain: true

Lights (excerpt)

light:
# Création du variateur de la suspension centrale
  - schema: template
    name: "Suspension centrale"
    object_id: "389"
    unique_id: "389"
    state_topic: "domoticz/out/389"
    state_template: "{% if value_json.nvalue == 2 %}on{% else %}off{% endif %}"
    brightness_template: '{{ value_json.svalue1 | multiply (255/100) | round (0)}}'
    command_topic: "domoticz/in"
    command_off_template : '{"command": "switchlight", "idx": 389, "switchcmd": "Off" }'
    command_on_template : >
      {%- if brightness is defined -%} 
        {"command": "switchlight", "idx": 389 , "switchcmd": "Set Level", "level" : {{ brightness | multiply(100/255) | round(0)}} }
      {%- else -%}
        {"command": "switchlight", "idx": 389 , "switchcmd": "Set Level", "level" : {{ states('input_number.idx389') }} }
      {%- endif -%}
    optimistic: false
    qos: 0
    retain: true
# Création du variateur du couloir
  - schema: template
    name: "Couloir"
    object_id: "377"
    unique_id: "377"
    state_topic: "domoticz/out/377"
    state_template: "{% if value_json.nvalue == 2 %}on{% else %}off{% endif %}"
    brightness_template: '{{ value_json.svalue1 | multiply (2.55) | round (0)}}'
    command_topic: "domoticz/in"
    command_off_template : '{"command": "switchlight", "idx": 377, "switchcmd": "Off" }'
    command_on_template : >
      {%- if brightness is defined -%} 
        {"command": "switchlight", "idx": 377 , "switchcmd": "Set Level", "level" : {{ brightness | multiply(100/255) | round(0)}} }
      {%- else -%}
        {"command": "switchlight", "idx": 377 , "switchcmd": "Set Level", "level" : {{ states('input_number.idx377') }} }
      {%- endif -%}
    optimistic: false
    qos: 0
    retain: true

Numbers (excerpt)

number:
# Consigne de température chambre
  - name: "Consigne chambre"
    object_id: "219"
    unique_id: "219" # Consigne température chambre
    icon: "mdi:thermometer-lines"
    unit_of_measurement: "°C"
    min: 10
    max: 22
    step: 0.5
    command_topic: "domoticz/in"
    command_template: '{ "idx": 219, "svalue": "{{value}}" }'
    state_topic: "domoticz/out/219"
    value_template: "{{value_json.svalue1}}"
    optimistic: false
    retain: true
    qos: 0
# Consigne température chambre amis
  - name: "Consigne chambre amis"
    object_id: "342"
    unique_id: "342" # Consigne température chambre amis
    icon: "mdi:thermometer-lines"
    unit_of_measurement: "°C"
    min: 10
    max: 22
    step: 0.5
    command_topic: "domoticz/in"
    command_template: '{ "idx": 342, "svalue": "{{value}}" }'
    state_topic: "domoticz/out/342"
    value_template: "{{value_json.svalue1}}"
    optimistic: false
    retain: true
    qos: 0

Select (excerpt)

# Sélection entrée A/V XMC-1
  - name: "Média"
    object_id: "264"
    unique_id: "264"
    options:
      - "Off"
      - HTPC
      - Shield
      - BR 4K
      - jRiver
      - CD
      - TD
    state_topic: "domoticz/out/264"
    value_template: >
      {% if value_json.svalue1 ==  "0" %} Off
      {% elif value_json.svalue1 == "10" %} HTPC
      {% elif value_json.svalue1 == "20" %} Shield
      {% elif value_json.svalue1 == "30" %} BR 4K
      {% elif value_json.svalue1 == "40" %} jRiver
      {% elif value_json.svalue1 == "50" %} CD
      {% elif value_json.svalue1 == "60" %} TD
      {% endif %}
    command_topic: "domoticz/in"
    command_template: >
      {% set value_map = {
            "Off": 0,
            "HTPC": 10,
            "Shield": 20,
            "BR 4K": 30,
            "jRiver": 40,
            "CD": 50,
            "TD": 60,
          }
      %}
      {"command": "switchlight", "idx": 264, "switchcmd": "Set Level", "level": {{ value_map[value] }}}
    optimistic: false
    qos: 0
    retain: true

Sensors (excerpt)

sensor:
# Température extérieure
  - name: "Température extérieure"
    unique_id: "415_1"
    object_id: "415_1"
    state_topic: "domoticz/out/415"
    unit_of_measurement: "°C"
    device_class: temperature
    value_template: "{{ value_json.svalue1 | round(1) }}"
# Humidité extérieure
  - name: "Humidité"
    unique_id: "415_2"
    object_id: "415_2"
    state_topic: "domoticz/out/415"
    unit_of_measurement: "%"
    device_class: humidity
    value_template: "{{ value_json.svalue2 }}"
type or paste code here

Switch (excerpt)

switch:
# Façade
  - name: "Façade"
    object_id: "468"
    unique_id: "468"
    state_topic: "domoticz/out/468"
    command_topic: "domoticz/in"
    value_template: "{{ value_json.nvalue }}"
    state_on: "1"
    state_off: "0"
    payload_on:  '{"command": "switchlight", "idx": 468 , "switchcmd": "On" }'
    payload_off: '{"command": "switchlight", "idx": 468, "switchcmd": "Off" }'
    optimistic: false
    qos: 0
    retain: true

Hope this helps.

Thank you for the info.
But as I am a beginner with HA, I would need some additional info !
I have two PIs :
One on which Raspberry is installed: 192.168.1.185
The other on which HA is installed: 192.168.1.188
I also have MQTT installed on my PI


And I have Mosquitto installed on HA
image

My question :
How to fetch Json data from 185 to 188?

Hi,

I am having issue to start:

This is my script:

automation domoticz:
  - alias: "Publish to DZ (switch, sensor, input_boolean, counter, proximity)"
    description: "publish"
    variables:
      entities:
        sensor.wilgotnosc_salon: 2841
        switch.grzanie_sypialnia_adam: 2482
      index: "{{ entities.get(trigger.entity_id, 0) }}"
      alert:
        'off': 1
        'on': 4
      alert_level: "{{ alert.get(trigger.to_state.state, 0) }}"
      alert_value:
        'off': Depassement de puissance resolu
        'on': Depassement de puissance
      alert_text: "{{ alert_value.get(trigger.to_state.state, 0) }}"
    trigger:
    - platform: state
      entity_id:
      - sensor.wilgotnosc_salon
      - switch.grzanie_sypialnia_adam
    condition: "{{ index != 0 }}"
    action:
    - service: mqtt.publish
      data:
        topic: domoticz/in
        payload_template: >
          {% if trigger.to_state.domain == 'switch' or trigger.to_state.domain == 'input_boolean'%}
            {"command": "switchlight", "idx": {{ index }}, "switchcmd": "{{ trigger.to_state.state | title }}"}
          {% elif trigger.to_state.domain == 'sensor' %}
            {"command": "udevice", "idx": {{ index }}, "nvalue": 0,  "svalue": "{{ trigger.to_state.state }}"}
          {% elif trigger.to_state.domain == 'proximity' %}
            {"command": "udevice", "idx": {{ index }}, "nvalue": 0,  "svalue": "{{ trigger.to_state.state | multiply(100) }} "}
          {% elif trigger.to_state.domain == 'binary_sensor' %}
            {"command": "udevice", "idx": {{ index }}, "nvalue": {{ alert_level }}, "svalue": "{{ alert_text }}"}
          {% elif trigger.to_state.domain == 'counter' %}
            {"command": "udevice", "idx": {{ index }}, "nvalue": 0, "svalue": "{{trigger.to_state.state | int + state_attr('counter.eau', 'initial')  }}"}                                                   
          {% endif %}
        qos: '0'
        retain: true
    mode: queued
    max: 25

  - alias: "Receive from DZ (switch or input_boolean)"
    description: "receive"
    variables:
      indexes:
        switch.grzanie_sypialnia_adam: 2482
      entity: "{{ indexes.get(trigger.topic.split(''/'') | last | int(0), ''unknown'') }}"
      function: "{{ entity.split('.')[0] }}"
    trigger:
    - platform: mqtt
      topic: domoticz/out/#
    condition: "{{ entity != ''unknown'' }}"
    action:
    - service: "{{function}}.turn_{{ if(trigger.payload_json.nvalue == 0, 'off', 'on') }}"
      target:
        entity_id: "{{ entity }}"
    mode: queued
    max: 25

And I am getting this error:

2023-04-15 00:51:09.980 ERROR (MainThread) [homeassistant.components.automation] Unnamed automation could not be validated and has been disabled: extra keys not allowed @ data['automation domoticz']. Got [{'alias': 'Publish to DZ (switch, sensor, input_boolean, counter, proximity)', 'description': 'publish', 'variables': {'entities': {'sensor.Salon_wilgotnosc': 2841, 'switch.Grzanie_sypialnia_Adam': 2482}, 'index': '{{ entities.get(trigger.entity_id, 0) }}', 'alert': {'off': 1, 'on': 4}, 'alert_level': '{{ alert.get(trigger.to_state.state, 0) }}', 'alert_value': {'off': 'Depassement de puissance resolu', 'on': 'Depassement de puissance'}, 'alert_text': '{{ alert_value.get(trigger.to_state.sta...
required key not provided @ data['action']. Got None
required key not provided @ data['trigger']. Got None

Any hint how to run it?
I am on latest HA 2023.4.4.

Best regards,

Adam

as the log suggests, there is no id at the top of the script. Make sure as well that indentations are correct.
Also what "automation domoticz: " is for? Please have a look at the original code that has been posted.
I would recommend to debug step by step and remove all the code that is not necessary. At of now you’ve got a sensor and a switch, so in payload_template, you may delete everything that is not related to these two domains

I did as you recommended. Added some corrections and it just works!

Adam

No idea for my post?

:wink:

Have you told the MQTT client on 185 that the broker (Mosquitto) is on 188?

This posts is great! Thank you man - it answers almost all my questions about how to push data from domoticz to HA!
One question-> how do you generate this ID:

    unique_id: "415_1"
    object_id: "415_1"

Is it any free number?

Adam

it is indeed free number. I normalized them to DZ idx’s. When an idx reports several values, for instance temperature and humidity sensors in one device, I decided to use _x suffix in order the have a kind of link which is here just for reminding me where does it come from.

Everything is explained in the above post. You should try to test a simple one, and you may also read DZ MQTT and HA MQTT. I did not use auto discover method in order to make sure I know what I’m doing.

Although I have problem with ppm sensor from KNX.
It is declared absolutely the same way as other devices and I normally see it changing in HA interface:

    - name: "Wilgotnosc_salon"
      state_address: "1/5/0"
      type: humidity
    - name: "co_salon"
      state_address: "1/6/0"
      type: ppm
    - name: "temperatura_sypialnia_adam"
      state_address: "1/1/3"
      type: temperature

its declared teh same way like other sensors:

      entities:
        sensor.wilgotnosc_salon: 2545
        sensor.co_salon: 2554
        sensor.temperatura_salon: 2533

      - sensor.wilgotnosc_salon
      - sensor.co_salon
      - switch.grzanie_sypialnia_adam

I even created log to see why I dont see it in mqtt explore:

    - service: system_log.write
      data_template:
        level: error
        logger: homeassistant.components.mylogger
        message: '"udevice", "idx": {{ index }}, "nvalue": 0,  "svalue": "{{ trigger.to_state.state }}"'

But nothing is happening, and sensor values are changing every 5-10 minutes in HA dashboard. Any clue how to solve the problem?

So to make sure I understand correctly. Are these sensors an HA integration that you want to push to DZ? Or are they a DZ integration that you want to send to HA?
Have you looked at the MQTT traffic with MQTT explorer?
You may also listen to a topic straight in MQTT integration. For that go to integration, click on MQTT/Configure.

Its from HA to Domoticz.
It is not and will be not in MQTT explorer as it is not even processed by the script → I dont see it with log i generated, like this sensor change does not trigger the script :frowning:

this is log I have and device 2554 is not there:

2023-04-20 05:32:58.421 ERROR (MainThread) [homeassistant.components.mylogger] "udevice", "idx": 2545, "nvalue": 0,  "svalue": "45.0"
2023-04-20 05:33:17.368 ERROR (MainThread) [homeassistant.components.mylogger] "udevice", "idx": 2517, "nvalue": 0,  "svalue": "off"
2023-04-20 05:33:38.676 ERROR (MainThread) [homeassistant.components.mylogger] "udevice", "idx": 2539, "nvalue": 0,  "svalue": "20.0"

Adam

Could you please post the entire script as it is now?