Shelly Dimmer with MQTT

I am using the integration for the dimmer and for an RGBWW. However the response time on my shelly25s and Shelly1s was way too long to use them in automations (eg. Switch a Shelly and turn on a light on a smart plug as well). Acc. my understanding the integration is polling (not adjustable) and mqtt is using push. At this point there is no other way as using mqtt for the relays, but in general I would like to have them setup all in the same way if possible.

Here is my configuration of the dimmer.

  mqtt:
    light:
      - name: "Bedroom 1"
        schema: template
        state_topic: "shellies/light-bedroom-ceiling/light/0/status"
        availability_topic: "shellies/light-bedroom-ceiling/online"
        command_topic: "shellies/light-bedroom-ceiling/light/0/set"
        state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
        command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
        command_off_template: '{"turn": "off"}'
        brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
        payload_available: "true"
        payload_not_available: "false"
        qos: 1
        retain: false
        optimistic: false

I can see a few differences. But Cannot say those are crucial

  1. state_template and brightness_template are enclosed with double quotes, while you are using single quotes.
  2. You are using unique_id. Check others entities declaration or logs. Maybe you did copy&paste error, having 2 entities with the same identifier?
  3. Optimistic setting (you are using default which equals true). For devices responding immediately false is better IMO. Also maybe there is a glitch in implementation. Worth to check.

Other than that it looks pretty the same (or differences are not important).

3 keywords: stability, reliability, and transparency.
MQTT is (potentially or not) way better in all those aspects compared to native integration.

1 Like

Thanks a lot … I will try that and give feedback. One question:
How your topics are linking into the correct device or in other words, how does mqtt know what Shelly ID to “talk” to?

You have:

        state_topic: "shellies/light-bedroom-ceiling/light/0/status"
        availability_topic: "shellies/light-bedroom-ceiling/online"
        command_topic: "shellies/light-bedroom-ceiling/light/0/set"

and I have (which includes the device ID:

      state_topic: "shellies/shellydimmer-<my_ID>/light/0/status"
      command_topic: "shellies/shellydimmer-<my_ID>/light/0/set"

Sorry if that is a rookie question :slight_smile:

Thanks again

Ok … I changed to:

    - name: "My Not Working Dimmer"
      schema: template
      state_topic: "shellies/shellydimmer-3C6105E42C4D/light/0/status"
      availability_topic: "shellies/shellydimmer-3C6105E42C4D/online"
      command_topic: "shellies/shellydimmer-3C6105E42C4D/light/0/set"
      state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
      command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
      command_off_template: '{"turn": "off"}'
      brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
      payload_available: "true"
      payload_not_available: "false"
      qos: 1
      retain: false
      optimistic: false

I have changed the name, to make sure to have a new entity (shouldn’t matter but just in case). Still not working. Communication seems to work, so I wouldn’t expect a settings issue?

According to the integration page, it’s local push, not poll.

Can you elaborate? I don’t have any stability or reliability problems with the integration (not entirely sure what you mean by transparency), and I believe that in most cases adding another layer would make these worse, not better.

Ok I found it. Little error in my topic. I needed to put in shellies/shellydimmer**2**…
following working code:

  light:
    - unique_id: mq_kitchen_light_dimmer
      name: "mq Kitchen Light Dimmer"
      icon: mdi:light-recessed
      schema: template
      state_topic: "shellies/shellydimmer2-xxxxxx/light/0/status"
      availability_topic: "shellies/shellydimmer2-xxxxxx/online"
      command_topic: "shellies/shellydimmer2-xxxxxx/light/0/set"
      state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
      command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
      command_off_template: '{"turn": "off"}'
      brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
      payload_available: "true"
      payload_not_available: "false"
      qos: 1
#      retain: false #came back with error so I took it out, will investigate why
      optimistic: false

Thanks everyone

You might be right here. Anyway, the Shellies respond way faster via mqtt. I built a 2 way switch automation using a shelly to turn a smart plug on. Worked fine at the beginning with the integration but was slow (up to 5 sec) recently. So I assume something changed on the integration itself.
At least I can say I’m happy with the results now :slight_smile:
And I also experienced certain stability issues here. I will observe if it is better now.

Generally, in complex systems, components providing single responsibility is a better strategy/design pattern than a single big component which do everything. Even for the price of communication overhead. Such components are easier to maintain (less complex code, fewer internal dependencies, communication via strictly defined APIs etc). The result is more mature, stable, system with fewer potential issues with possibly faster development pace overall.
Please note, that thanks to strictly defined interfaces (ie MQTT protocol) single components (incl devices) might evolve independently. (see below)

It’s valid for HA. The system is very complex, with internally shared objects, thus creating a pileup of internal dependencies. Constant evolution makes this even worse.
You can see the result after each release when things get broken (even those that - based on the change log - weren’t touched). It was happening several times for Shelly’s integration in the past.
Not speaking about time needed to fix those issues. Sometimes those are days, other times - months. As a end-user you don’t want to experience that.

MQTT is matured component, an industry standard. Something you can rely on more than on HA. It doesn’t add to instability or unreliability. It’s the opposite.

Also note, that restarting HA (which you do possibly often upgrading it) doesn’t cause devices disconnection from MQTT (MQTT is another docker container or might run on on separate server). This separation also adds to the overall health of the system (not speaking about security, which is increased since HA and devices must not see each other).

By transparency, I meant that mqtt gives you insight into communication going on between a device and the HA. Also, you are a creator of the entity declaration. So you know exactly where to look at, in case of problems. Moreover, you can access mqtt and control devices through mqtt, skipping HA. It eventually allows you to use features provided by a manufacturer, not supported by HA.
Obviously, HA integration is the opposite.

For me, the decision to go for MQTT-based solution is a natural one (I’m IT architect, and programmer). For the same reasons, I opted-in for Zigbee2MQTT instead of ZHA.
A lot of people probably think about such architecture as another component to maintain, which adds to complexity, instability etc. But I’m confident to say, it’s the opposite.

I hope it answers your question.

Thanks for the detailed response. I’m well versed in software architecture, but as the saying goes “You can solve every problem with another level of indirection, except for the problem of too many levels of indirection”.
Yes, the Shelly integration did have breaking changes in its infancy, but so does the MQTT integration, and from a very cursory look, it has more PRs in minor releases than the Shelly integration, indicating it might break more often. Additionally, you also have to worry about maintaining your MQTT server, and keeping up to date with security patches for your MQTT server.

I’m not aware of any case in which a fix took more than a few days.

Shelly components communicate with MQTT over plain text. I don’t think you can make any serious claims about security.

I definitely could see the case for Zigbee and Z-Wave. These are complicated stacks that are better off maintained independently. Shelly components have a simple interface.

gen1 seems stable for some time, but gen2 are still evolving wich renders in remarkable delays in integration support. Year and half ago, new fw broke integration and it took more than month to update integration (some devices had issues even half year later). not that it cannot happen with mqtt but you can update settings immediately not waiting for devs. (and tbh gen1 mqtt didn’t changed for 3 years afaik in a way it required changes in settings)

I’m aware mqtt integration might change (recently have changed), a bug might be introduced to mqtt addon (which I use) but at first those are not so many updates, at second user base of mqtt is way wider than shelly alone, so probably those components get more attention from devs. Anyway number of issues I experienced with mqtt differs by orders of magnitude comparing to issues reported by users of the Integration.

anyway with the integration you still cannot:

  • change light attributes (brightness, color) having shelly entity turned off (without turning it on)
  • configure switch as light
  • replace one device with another without changing name/id especially when changing its type
  • change shelly device ip address
  • there are problems with removal of shelly devices from the system (see How to delete a device / entity in ha - #10 by petro)

Probably lot more issues related to limited flexibility.

I’m not here to convince any one (or maybe I am :wink: ) or criticise devs work. Just giving idea how advantageous is using mqtt integration comparing to native one. Especially if the only argument against is presumed higher complexity.

Same here. Just trying to understand the pros and cons, and see whether they relate to my setup. I think the main difference between us is that I only use gen1 devices, so I haven’t seen any of the problems with the native integration that you have seen.

I’m not sure these are true. The Shelly integration is getting a lot of love from developers, and is also considered a platinum integration (vs gold for MQTT), meaning it adheres to all of the HA best practices. As I mentioned in my previous post, a cursory look for minor releases shows more MQTT PRs than Shelly PRs, indicating that the MQTT integration needs hotfixes more often.

You could also say about all of these that they are presumed higher stability, to paraphrase your presumed higher complexity remark :slight_smile:.

And we haven’t even discussed the configuration - getting MQTT configured on all of the modules, setting all of the devices in HA (or using some script for auto discovery).

In any case, maybe once I get a gen2 device I’ll experience what you’re experiencing and want to switch. In the meantime, I remain unconvinced.

So I am. I was using Shelly devices way before the integration has been added. And I saw the whole process of incremental improvements. And read tons of reports from users that something doesn’t work, or stopped to work etc. If you jumped into this train recently, then likely gen1 devices are covered pretty well (excluding features that are not supported because HA does not support them natively, but you could use them through automation)

Fair enough :wink: Every architecture works until one find out its limitations. If someone asks for that he should get a fair answer.
It’s not about forcing someone to change his approach if it does work for him. But rather to agree that some patterns are stronger than others. The way HA and devices connect to MQTT (not another way around) and not seeing each other is a very strong pattern.

BTW can you change the IP address of Shelly registered in the Integration without the need of doing any changes to HA? Someone recently wrote that Shelly devices are identified by IP address.

I’ve been using Shelly way before the integration, and even attempted to create an official MQTT based integration before the current one was developed (the idea was to add the discovery functionality but delegate everything else to the regular MQTT integration, but there was simply no way to do that).
Been using the native integration since the first version is came out (even contributed a PR or two), and never encountered any issue. I still have a couple of devices that I haven’t transitioned and are still on MQTT, and they’re actually less reliable (although there are other differences, so it isn’t necessarily related to MQTT).

All of my Shellys are on DHCP, and IP address changes are handled seamlessly.

I was asking about how HA copes with a change of IP address of already registered Shelly devices.
DHCP has nothing to do with that.

Considering that changing the firmware of Shelly a year and half ago broke the shelly integration (and it is only one out of many examples) how is it possible you have encountered none of them? Maybe you are using those devices in a specific way. But then taking it as a common example for advice might be misleading to other people.

As I wrote, my Shelly modules are on DHCP (meaning their IP address can change when their lease expires), and those changes are dealt with seamlessly by HA.

I upgrade HA more frequently than I upgrade Shelly firmware, so it is possible I missed the specific update, but it can’t have been that long. Can you share more information on this incident?

As far as I remember it was with fw 1.8. Shelly changelog mentions rewriting CoAP protocol from scratch. It wasn’t 1.5 ago. It was August 2020.

The 1.8 fw came out in Aug 2020. The official Shelly integration came out in September 2020. Whatever 1.8 broke wasn’t the official integration.

ok. My fault. So back again to my the first suggestion (1.5y ago).
It was the integration update (I think 2021.5 ?) that brought limitations to firmware 1.10 and newer. It caused a lot of reports: first, updating HA broke communication with shellies for lot of people who didn’t updated fw yet. Second, those changes introduced issues that were fixed in subsequent HA releases.
When looking at HA changelogs, I found that battery-powered Shelly devices were added later on… so more than 2 years after I use them with mqtt.

Anyway, regardless of how you see the Shelly integration, it adds a dependency on HA developers to the pipeline. And this is what is suboptimal comparing to usage of MQTT+MQTT integration.

To be fair I registered following breaking “issues” with mqtt:

  1. In Feb’21 MQTT add-on has been released with the bug, which caused MQTT to lag (up to 5 secs). It took about 3 months to fix that. It was issue with the add-on, not with MQTT
  2. After introducing manual reload of mqtt entities, the expiration time for mqtt configured entities was broken (it turns devices into unavailable after some time after reload). It took about half a year to be fixed (since my first post/report about that)
  3. recently HA changed the format of yaml configuration of mqtt entities.

Notice, how few issues over 3 years. Also, no issue comes from MQTT server itself. If we jump back to the beginning of this discussion: people are afraid of this dependency. But the reality is the opposite. MQTT is the most stable component in this party. If you combine it with stable MQTT implementation on Shelly side (Coap has been rewritten/changed making the integration not backward compatible) - it’s really hard to say other option is better.

But we’ve already established that so does MQTT, and I’ve shown evidence that suggests the Shelly integration might be better maintained than the MQTT one.

Hmm, you showed even fewer than that for the Shelly integration…

Irrelevant. Using MQTT introduces additional dependencies. Whether it’s the add-on that breaks, the MQTT server itself, or some other component along the way does not affect the argument.

IMO 3 issues with mqtt (all non-breaking or resolvable by the user) compared to a number of issues with the Shelly Integration incl waiting for subsequent HA releases to fix is a big difference (not speaking about a delay in providing support for some Shelly features). If you want me to list all issues with Shelly integration over here, then you knocking the wrong door. The forum is full of them - it’s proving enough what I say. At that time I was only a reader of those issues. Just passing the information those have been reported and discussed on this forum.

Not sure what you mean by “better maintained”. I don’t care about medals. From the user’s perspective, it’s important to how many issues he has to face over a period of time. And how much time he has to wait for a fix for a potential issue. IMO MQTT way is unbeatable compared to the integration one in those areas.

BTW not once not twice has happened, that a new release did break integration that wasn’t listed in the changelog (it happened also to Shelly integration AFAIK). Fortunately, it never happened to mqtt integration (at least not breaking it, if we consider the issue with expiration). And this is the biggest income from choosing mqtt: the separation. I’m not afraid of updating HA because it is highly unlikely that it breaks MQTT.