Manually add MQTT Entities

From MQTT Explorer, it actually looks like everything is working. First, I turned the light off by sending “OFF” to “/set”. And the light did turn off as expected. Then, using the HASS UI, I tried turning the light on. Carefully watching the MQTT Explorer window, I saw “set” go to “ON”. The light then turned on, and I then saw “state” go to { "state" : "ON", "brightness" : 255 }. Then, after about one second, the Lovelace UI switch went back to off, and nothing else changed in MQTT explorer. I never really get a chance to try and turn it off.

Initial conditions (light is currently off)

After tapping the Lovelace UI switch on (light stayed on, but the Lovelace UI switch went back to off):

There is the missing piece of the puzzle.

The device does not publish its current state merely as ON or OFF but as a JSON string containing two key-value pairs:

"state": "ON"
"brightness": 255

We need to use state_value_template to extract the ON value from the state key.

light:
  - platform: mqtt
    command_topic: 'insteon/57.a1.28/set'
    state_topic: 'insteon/57.a1.28/state'
    unique_id: 'diningroomchandelier'
    name: 'Dining Room Chandelier'
    state_value_template: '{{ value_json.state }}'

If this device is a dimmer then we also want to know its brightness level as well. For that we will use the brightness_value_template to extract the brightness value from the brightness key.

light:
  - platform: mqtt
    command_topic: 'insteon/57.a1.28/set'
    state_topic: 'insteon/57.a1.28/state'
    unique_id: 'diningroomchandelier'
    name: 'Dining Room Chandelier'
    state_value_template: '{{ value_json.state }}'
    brightness_value_template: '{{ value_json.brightness }}'

NOTE

If this device not only transmits its status in JSON but also expects to receive commands in JSON, then the simplest way to support it would be use the JSON schema.

2 Likes

IT WORKS!!! :slight_smile:

If I’m understanding this correctly, I’m not using the scripts.yaml that you posted earlier at all. I’m doing this the old-fashioned way of editing configuration.yaml. I’m ok with that. I am just trying to understand.

From here, I will write entries for each of my insteon lights, similar to what we have done here. Then I can add the new light entities to scenes and make it even better! Sweet!

Seriously, big thanks! Sorry for all of the questions, my friend!

1 Like

Correct. This is still the traditional YAML method of configuring an MQTT Light. The challenge here was to understand the format of the payload transmitted by the Insteon-MQTT integration. Once that was known, it simply became a matter of extracting what is needed from the received payload.

You’re welcome!

Please consider marking my post (above) with the Solution tag. It will automatically place a check-mark next to the topic’s title which signals to other users that this topic has an accepted solution. It also places a link below your first post that leads to the solution post. All of this helps other users find answers to similar questions.

2 Likes

Is it possible to please post the full script that created one device with more than one entity. I have an HA switch device (ESP32) that is connected to a secondary serial device (Uno R3) which has additional sensors and switches. My mqtt messaging works but I do not know how to create the multi switch device in HA in a yaml file or script. I am new to HA but old in IT. :wink:

The “full script” is already posted above. If you want to make a device with multiple entities, each entity you create (via MQTT Discovery) should use the same device information.

“each entity you create (via MQTT Discovery) should use the same device information.”

Man it is easy … if you know what you are doing!! THANKS!!!

Amazing fast reply as well!! :clap:

Does that script still work in the current version to manually add mqtt devices. Now when you go to Configuration>Scripts you can only click the scripts wording which looks like it may refresh the script.

Yes.

Thanks for the above, worked great for me. One thing i’m struggling with is applying a state value template, i’m trying to convert the incoming watt hour value to kWh with this…

   "state_value_template": "{{ value.state | multiply(0.001) }}",

but this isnt working so must be formatting it incorrectly, can anyone help? The incoming data is not a json.

Thanks!

   "state_value_template": {% raw %}"{{ value.state | float(0) | multiply(0.001) }}"{% endraw %},
1 Like

Just adding my experience…
I have finally converted my 83 mqtt entities to this script technique (thanks Taras@123).
Mqtt startup time was cut down from 30+ seconds to around 10 seconds.

What I like about this, is that I am able to create devices in HA with this.
Some of my devices have webservers, so by using the configuration_url in the device parameters list, I can now go to HA for that device and click on the URL and it takes me to that device’s web page :slight_smile: .

The one downside for me (and its not a biggy) is that with YAML, I can add comments to various config parameters, whereas with this approach I don’t think there is a way to add comments for JSON structured data. So I’ll leave my YAML file laying around in case I need to refer to it later on.

The hardest part is trying to debug any mistakes in the script’s mqtt payload JSON data. I set the mqtt component logger to debug for starters. If there are no JSON decoder errors, then the debug output should be able to tell you what may be wrong. If the error indicates a JSON decoding error, then it can be quite challenging to figure out. One thing that helped a little was to “snoop” on that particular MQTT discovery topic to see what the payload looks like. If I couldn’t figure it out this way, then worse case was start eliminating lines in the JSON data until the JSON decoding error disappears.

Here are some errors I came across.

  • 00 won’t parse. Must be single digit 0.
  • unique-id/object-id can not contain a “.” . For Insteon users, 26.aa.bb not allowed.
  • For device parameters, “identifiers” or “connections” is required. Otherwise get:
    error - Device must have at least one identifying value in ‘identifiers’
    and/or ‘connections’ for dictionary value @ data[‘device’].
  • For a given JSON key-value pair make sure the value itself doesn’t have a double-quote inside the already double-quoted value. For example: {% raw %}"{% if value == "3" -%}active{%- else -%}inactive{%- endif %}"{% endraw %} will run into errors because the 3 is double-quoted, so change it to single-quotes '3'.
  • Last JSON elements in a list can not have a comma “,” . For the example below (taken from above), "sw_version": "1.X" is the last element in the device list, and device {...} is the last element in the list, and they should not have a comma.
             .....
              "unique_id": "x10_pool_pump_switch",
              "device": {
                  "identifiers": ["x10"],
                  "name": "X10 CM11A",
                  "model": "CM11A",
                  "manufacturer": "X10",
                  "sw_version": "1.X"
              }
            }

Finally, when HA receives a published mqtt discovery message, it does not store all of this data in non-volatile memory. It creates the entities and devices which are stored in one of its .storage/core.xxx files`, but it does not store topics and other parameters. To solve this, one can simply publish the mqtt discovery message with the retain flag set to true as shown earlier in this thread. This way when HA restarts, HA will subscribe to the discovery topic and MQTT will replay all the retained data. An alternative that I have tried out and it is working, is to not retain the data in mqtt, but replay the script(s) at HA startup time using an automation.

Hi
Thank you @123 and @wmaker for your input on the mqtt discovery info.
Im trying to add a second sensor to a device but it keeps replacing the previous added sensor.
Im not sure what Im doing wrong.

service: mqtt.publish
data:
  topic: homeassistant/sensor/12B978/config
  retain: false
  payload: >-
    {  "device_class": "temperature", "name": "Internal Temp", "state_topic":
    "Afterburner12B978/sts/TempBody", "unit_of_measurement": "°C", "unique_id":
    "12B978",  "device": {      "identifiers": ["afterburner"],      "name":
    "AfterBurner",      "model": "4KW",      "manufacturer": "MRJones",
    "configuration_url": "http://192.168.1.31",    "sw_version": "3.4.7"  }}
  qos: '0'

What works for me is to create a new sensor (also new unique_id/topic) but copy-n-paste the same device info from the original to the new one. If that doesn’t work, maybe show what both of your configs look like.

Hi Tommy
That explains it then, I was using the same topic id for both sensors.
Thank you

sorry to bump this, but it comes closest to what I am looking for… please help me get a step further?

Having many existing devices, eg via Zwave, to which I want to add several ‘day_start’ sensors, so they appear in the devices overview.

lets take a Zwave device Audio Gym:

this has a device_id

I learned via Tinkerer in Discord, I can ‘fake’ discovery on those, so I can set the discovery topic and tie the following (now standalone) Mqtt sensors to this device:

    - unique_id: audio_gym_power_daystart
      state_topic: ha_main/sensor/audio_gym/power_daystart
      name: Audio Gym power daystart
      <<: *power

    - unique_id: audio_gym_usage_daystart
      state_topic: ha_main/sensor/audio_gym/usage_daystart
      name: Audio Gym usage daystart
      <<: *energy

these sensors are updated daily (…) via an automation, and I have about 35 of those sets.

  - alias: Set daystart sensors
    id: set_daystart_sensors
    trigger:
      platform: time
      at: '00:00:01'
    action:
      service: python_script.record_daystart

I understand I need to automate a topic publication and retain that, so it needs to be done only once.

Tara’s example above creates a full device, but I only need to add these 2 sensors to the existing device.

what am I supposed to publish exactly… and which of the map options should I use: MQTT Sensor - Home Assistant

Ofc, If I need to change those daystart sensors themselves, please let me know, because they have no other dependencies currently.

I also have another option to create those daystart sensors, and that is without a python script , but use this core Ha script:

script:

  publish_sensor_daystart_values:
    alias: Publish sensor daystart values
    sequence:
      - variables:
          switches: >
            {{expand('switch.z_wave_switches',
                     'switch.dorm_studenten','switch.symfonisk_dining')
                     |map(attribute='object_id')|list}}
      - repeat:
          count: >
            {{switches|count}}
          sequence:
            - variables:
                oid: >
                  {{switches[repeat.index - 1]}}
                actueel: >
                  sensor.{{oid}}_actueel
                totaal: >
                  sensor.{{oid}}_totaal
            - service: mqtt.publish
              data:
                topic: >
                  ha_main/sensor/{{oid}}/power_daystart
                payload_template: >
                  {{states(actueel)}}
                qos: 2
                retain: true
            - service: mqtt.publish
              data:
                topic: >
                  ha_main/sensor/{{oid}}/usage_daystart
                payload_template: >
                  {{states(totaal)}}
                qos: 2
                retain: true

maybe that would make things easier?

the device_registry for this device is:

      {
        "config_entries": [
          "62213aa5a398677a26e1a3354a861183"
        ],
        "connections": [],
        "identifiers": [
          [
            "zwave_js",
            "3967834106-71-271:1538:4099"
          ],
          [
            "zwave_js",
            "3967834106-71"
          ]
        ],
        "manufacturer": "Fibargroup",
        "model": "FGWP102",
        "name": "Metered Wall Plug Switch",
        "sw_version": "3.2",
        "hw_version": null,
        "entry_type": null,
        "id": "93991b3640262c0fa3a7c9b96342513c",
        "via_device_id": null,
        "area_id": "gym",
        "name_by_user": "Audio Gym",
        "disabled_by": null,
        "configuration_url": null
      }

seems the topic/payload should be:

device: ha_main/sensor/audio_gym
payload: >
{"identifiers": ["93991b3640262c0fa3a7c9b96342513c","Audio Gym"],
"name":"Metered Wall Plug Switch",
"manufacturer": "Fibargroup",
"model": "FGWP102"}

in topic publish window:

if I do that, MQTT sees this:

and there is no tie in at all…

I can add the device_id to the key in the entity_registry of these sensors manually, immediate effect:

now how to do that from within the UI, or backend automation…

2 Likes

Did you ever figure this out? Is the only way to tie an entity to an existing (non-MQTT) device by manually editing the entity registry?

sorry, no, I havent explored any further, just create my sensors and they are not tied in to those devices

Hey All

I’m trying to add the MQTT manual but I’m not wining. It’s for our generator.
image

How will I add it to my Configuration file?

You’d be best off starting a new topic. Take a look at the docs here, and in the new topic post the properly-formatted YAML code of what you tried and what error you’re getting.