Manually add MQTT Entities

I think you should be able to configure it following this integration:

There are MQTT-based integrations for lights, switches, locks, covers, climate, etc.

MQTT Integrations

They can be configured manually, in YAML, or as a script that creates them via MQTT Discovery (that’s my preferred method).

Here’s where I am now. I sort of have it working, but there’s a ways to go.

I have added this to my configuration.yaml:

light:
  - platform: mqtt
    command_topic: 'insteon/57.a1.28/set'
    state_topic: 'insteon/57.a1.28/state'
    unique_id: 'diningroomchandelier'
    name: 'Dining Room Chandelier'

After restarting the server, the light shows up in the entity list. But I can’t control it from HASS. When I click on the light, and click on the gear to turn the light on/off, it always says the light is off, and if I try turning the light on, nothing happens. I also tried subscribing to a topic in the MQTT config and sending some commands with mqtt explorer. The light works as expected when I send the commands from mqtt explorer, and this is what was received:

Listen to a topic
 
Listening to
insteon/57.a1.28/#
 
Message 4 received on insteon/57.a1.28/state at 6:50 AM:
{
    "state": "ON",
    "brightness": 255
}
QoS: 0 - Retain: false

Message 3 received on insteon/57.a1.28/set at 6:50 AM:
ON
QoS: 0 - Retain: false

Message 2 received on insteon/57.a1.28/state at 6:50 AM:
{
    "state": "OFF",
    "brightness": 0
}
QoS: 0 - Retain: false

Message 1 received on insteon/57.a1.28/set at 6:50 AM:
OFF
QoS: 0 - Retain: false

Message 0 received on insteon/57.a1.28/state at 6:50 AM:
{
    "state": "ON",
    "brightness": 255
}
QoS: 0 - Retain: true

I guess my question is, how can I control this light from within HASS?

@123 I’d love to use MQTT Discovery, but insteon-mqtt doesn’t fully support discovery yet. I’m ok with manually doing it for now though.

That’s exactly what Taras uses his script for (if I understand correctly). His script sends out the message needed to make the devices discovered automatically.

That would be nice. I’d like to hear more about that.

1. Create an MQTT Light using a script

Assuming you have an existing scripts.yaml file, paste this into it and save the file:

Click to reveal scripts
## MQTT Discovery Test
  example_create:
    alias: 'Example MQTT Light - Create'
    sequence:
      service: mqtt.publish
      data:
        topic: 'homeassistant/light/example/config'
        retain: true
        payload: >
          {
            "name": "Example",
            "unique_id": "ABC123",
            "stat_t": "home/example",
            "cmd_t": "home/example/set",
            "bri_stat_t": "home/example/brightness",
            "bri_cmd_t": "home/example/brightness/set"
          }

  example_delete:
    alias: 'Example MQTT Light - Delete'
    sequence:
      service: mqtt.publish
      data:
        topic: 'homeassistant/light/example/config'
        retain: true
        payload: ''

Then carry out the following simple steps

  1. Execute Configuration > Server Controls > Check Configuration and confirm it reports no errors.
  2. Execute Configuration > Server Controls > Reload Scripts
  3. Execute Configuration > Scripts > Example MQTT Light - Create

Congratulations! You have used a script to create an MQTT Light via MQTT Discovery.

2. Control/remove the light

If you check Developer tools > States, you will find a new entity named light.example and its state will be off.

  • To turn on light.example, publish ON to home/example.
  • To change its brightness, publish 125 to home/example/brightness
  • To remove light.example from your system, execute:
    Configuration > Scripts > Example MQTT Light - Delete

3. Advantages of this method

One of the advantages (compared to manually defining MQTT entities in YAML) is that you don’t need to restart Home Assistant to add/remove MQTT entities. Another advantage is that you can define Devices using MQTT Discovery but cannot do that via YAML.

Here’s an example of what I mean. The following script defines a switch. However, I want the switch to be represented as an entity of a device called “X10 CM11A”. Notice the additional “device” information included in its discovery payload (you can’t do that if you were to define the switch in YAML).

      - service: mqtt.publish
        data:
          topic: homeassistant/switch/pool_pump/config
          retain: true
          payload: >
            {
              "name": "Pool Pump",
              "state_topic": "home/pump/powerstate",
              "command_topic": "command/pump/powerstate",
              "payload_on": "1",
              "payload_off": "0",
              "state_on": "1",
              "state_off": "0",
              "unique_id": "x10_pool_pump_switch",
              "device": {
                  "identifiers": ["x10"],
                  "name": "X10 CM11A",
                  "model": "CM11A",
                  "manufacturer": "X10",
                  "sw_version": "1.X"
              }
            }

The result will be the creation of a Device called “X10 CM11A” and “Pool Pump” is one of the several entities belonging to it.

7 Likes

@123 OK, I’ve done everything, and it works. But I am struggling now figuring out how to make my actual lights work with this. Maybe I should mention that I have only been using HASS for a month or two. All of my automations are in node-red, and all of my devices and entities so far have been auto-discovered. I know how to edit YAML files and JSON, but as for understanding what scripts in HASS do, and templates (I keep seeing that word thrown around), and everything else, I’m very much a newbie.

So, I have installed insteon-mqtt. From their documentation here:

Here is a sample configuration that accepts and publishes messages using upper case ON an OFF payloads.

switch:
  # Output state change:
  state_topic: 'insteon/{{address}}/state'
  state_payload: '{{on_str}}'

  manual_state_topic: 'insteon/{{address}}/manual_state'
  manual_state_payload: '{{manual_str.upper()}}'

  # Direct change only changes the load:
  on_off_topic: 'insteon/{{address}}/set'
  on_off_payload: '{ "cmd" : "{{value.lower()}}" }'

  # Scene change simulates clicking the switch:
  scene_topic: 'insteon/{{address}}/scene'
  scene_payload: '{ "cmd" : "{{value.lower()}}" }'

Do I use that information somehow and turn it into a script that will operate my real lights? A few posts up, I posted my configuration.yaml file for the lights, if that helps. I guess what I don’t know is, does this new way replace insteon-mqtt? Does it replace the stuff in my configuration.yaml? Or does it work with them? Do I need to write some scripts like your pool pump example for each of my switches?

Really sorry for all of the newbie questions, but seriously thanks for the help!!!

Your last paragraph is a pretty good simulation of a Gish Gallup.

Pick one of the five questions you asked and I’ll answer it.

Sorry, I guess that was my feeble attempt to illustrate the level of my confusion. I’ve been reading up on these things, and I’m not seeing what ties it all together. I can’t tell you how much I really appreciate the help here though!

My one question is: How do I make this work with one of my insteon lights?

When you use MQTT Explorer to publish ON to insteon/57.a1.28/state does the state of light.dining_room_chandelier change from off to on?

No. Sending ON and OFF commands to insteon/57.a1.28/state does not work. It does work, however, when I send them to insteon/57.a1.28/set.

Overview of the two topics:

The state_topic is the MQTT topic that represents the device’s current state. Home Assistant subscribes to this topic and uses whatever value it receives to represent the entity’s current state.

In other words, if you publish ON to the state_topic, the entity’s state will be on.

The command_topic is the MQTT topic used to control the device. Home Assistant publishes to this topic.

In other words, if you set the light entity to on, Home Assistant will publish ON to the command_topic.

What exactly “does not work”?

What it should do is set the state of light.dining_room_chandelierto on (you can check its state in Developer tools > States). What it will not do is turn on the physical device.

The exact thing that isn’t working is when I go to Lovelace, and click on Dining Room Chandelier light, I can’t turn the light on or off. It always shows OFF. If I try to turn it on, it clicks over to on and then right back to off (but the actual light does not actually turn on).

What does work is when I use MQTT explorer, when I publish ON and OFF to insteon/57.a1.28/state, the state of light.dining_room_chandelier does follow, as you suggested. Like I said though, the light itself is not responding.

Conversely, when I use MQTT explorer and publish ON and OFF commands to insteon/57.a1.28/set, the actual light turns on and off as expected, but nothing happens to light.dining_room_chandelier.

All of this adds up to I cannot control the light with HASS. I can by sending commands with MQTT Explorer, but the changes aren’t showing up in HASS.

OK that’s good and clarifies your previous report (which implied this operation does not work).

This is useful. If you use the UI to turn on the light and it almost immediately reverts back to off, it means Home Assistant never received acknowledgement from the device via the state_topic. In this particular case, it’s probably because the command Home Assistant published (via command_topic) was either the wrong command or was never received by the light. The light never responded to the command (via state_topic) so Home Assistant dutifully reverted the toggle button (in the UI) from on back to off.

You have confirmed that the device is indeed subscribed to insteon/57.a1.28/set and can be controlled with ON and OFF payloads.

The entity light.dining_room_chandelier is publishing to the same topic (insteon/57.a1.28/set) but isn’t getting the same results. That’s the puzzle that needs to be solved.

Although it’s possible to use payload_on and payload_off options to specify what to publish to represent on and off, the default values are precisely what you want, namely ON and OFF (all uppercase). In other words, Home Assistant should be sending the correct payloads to control the physical light. However, I suggest you confirm it. Simply use MQTT Explorer to monitor insteon/57.a1.28/set as you flip the UI’s toggle button from off to on to off.

From MQTT Explorer, it actually looks like everything is working. First, I turned the light off by sending “OFF” to “/set”. And the light did turn off as expected. Then, using the HASS UI, I tried turning the light on. Carefully watching the MQTT Explorer window, I saw “set” go to “ON”. The light then turned on, and I then saw “state” go to { "state" : "ON", "brightness" : 255 }. Then, after about one second, the Lovelace UI switch went back to off, and nothing else changed in MQTT explorer. I never really get a chance to try and turn it off.

Initial conditions (light is currently off)

After tapping the Lovelace UI switch on (light stayed on, but the Lovelace UI switch went back to off):

There is the missing piece of the puzzle.

The device does not publish its current state merely as ON or OFF but as a JSON string containing two key-value pairs:

"state": "ON"
"brightness": 255

We need to use state_value_template to extract the ON value from the state key.

light:
  - platform: mqtt
    command_topic: 'insteon/57.a1.28/set'
    state_topic: 'insteon/57.a1.28/state'
    unique_id: 'diningroomchandelier'
    name: 'Dining Room Chandelier'
    state_value_template: '{{ value_json.state }}'

If this device is a dimmer then we also want to know its brightness level as well. For that we will use the brightness_value_template to extract the brightness value from the brightness key.

light:
  - platform: mqtt
    command_topic: 'insteon/57.a1.28/set'
    state_topic: 'insteon/57.a1.28/state'
    unique_id: 'diningroomchandelier'
    name: 'Dining Room Chandelier'
    state_value_template: '{{ value_json.state }}'
    brightness_value_template: '{{ value_json.brightness }}'

NOTE

If this device not only transmits its status in JSON but also expects to receive commands in JSON, then the simplest way to support it would be use the JSON schema.

2 Likes

IT WORKS!!! :slight_smile:

If I’m understanding this correctly, I’m not using the scripts.yaml that you posted earlier at all. I’m doing this the old-fashioned way of editing configuration.yaml. I’m ok with that. I am just trying to understand.

From here, I will write entries for each of my insteon lights, similar to what we have done here. Then I can add the new light entities to scenes and make it even better! Sweet!

Seriously, big thanks! Sorry for all of the questions, my friend!

1 Like

Correct. This is still the traditional YAML method of configuring an MQTT Light. The challenge here was to understand the format of the payload transmitted by the Insteon-MQTT integration. Once that was known, it simply became a matter of extracting what is needed from the received payload.

You’re welcome!

Please consider marking my post (above) with the Solution tag. It will automatically place a check-mark next to the topic’s title which signals to other users that this topic has an accepted solution. It also places a link below your first post that leads to the solution post. All of this helps other users find answers to similar questions.

2 Likes

Is it possible to please post the full script that created one device with more than one entity. I have an HA switch device (ESP32) that is connected to a secondary serial device (Uno R3) which has additional sensors and switches. My mqtt messaging works but I do not know how to create the multi switch device in HA in a yaml file or script. I am new to HA but old in IT. :wink:

The “full script” is already posted above. If you want to make a device with multiple entities, each entity you create (via MQTT Discovery) should use the same device information.