Shelly Dimmer with MQTT

then it seems to be vsc issue only, as I said before

Yes thanks anyway!

Hi everyone,

I started over the weekend to move all my Shellies to MQTT. So far no issues, but I definitely have trouble to get my Dimmer to work. The Dimmer itself works fine since month with the integration.

image

What am I doing wrong? (I’m not a hack in Yaml, btw)

Cheers

mqtt:
  switch: # including all other relays such as Shelly25 and Shelly1

  light:
    - unique_id: mq_kitchen_light_dimmer
      name: mq Kitchen Light Dimmer
      schema: template
      state_topic: "shellies/shellydimmer-<my_ID>/light/0/status"
      state_template: '{% if value_json.ison %} on {% else %} off {% endif %}'
      command_topic: "shellies/shellydimmer-<my_ID>/light/0/set"
      command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
      command_off_template: '{"turn": "off"}'
      brightness_template: '{{ value_json.brightness | float | multiply(2.55) | round(0) }}'

Now that there’s an official Shelly integration, what’s the motivation to use MQTT?

I am using the integration for the dimmer and for an RGBWW. However the response time on my shelly25s and Shelly1s was way too long to use them in automations (eg. Switch a Shelly and turn on a light on a smart plug as well). Acc. my understanding the integration is polling (not adjustable) and mqtt is using push. At this point there is no other way as using mqtt for the relays, but in general I would like to have them setup all in the same way if possible.

Here is my configuration of the dimmer.

  mqtt:
    light:
      - name: "Bedroom 1"
        schema: template
        state_topic: "shellies/light-bedroom-ceiling/light/0/status"
        availability_topic: "shellies/light-bedroom-ceiling/online"
        command_topic: "shellies/light-bedroom-ceiling/light/0/set"
        state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
        command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
        command_off_template: '{"turn": "off"}'
        brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
        payload_available: "true"
        payload_not_available: "false"
        qos: 1
        retain: false
        optimistic: false

I can see a few differences. But Cannot say those are crucial

  1. state_template and brightness_template are enclosed with double quotes, while you are using single quotes.
  2. You are using unique_id. Check others entities declaration or logs. Maybe you did copy&paste error, having 2 entities with the same identifier?
  3. Optimistic setting (you are using default which equals true). For devices responding immediately false is better IMO. Also maybe there is a glitch in implementation. Worth to check.

Other than that it looks pretty the same (or differences are not important).

3 keywords: stability, reliability, and transparency.
MQTT is (potentially or not) way better in all those aspects compared to native integration.

1 Like

Thanks a lot … I will try that and give feedback. One question:
How your topics are linking into the correct device or in other words, how does mqtt know what Shelly ID to “talk” to?

You have:

        state_topic: "shellies/light-bedroom-ceiling/light/0/status"
        availability_topic: "shellies/light-bedroom-ceiling/online"
        command_topic: "shellies/light-bedroom-ceiling/light/0/set"

and I have (which includes the device ID:

      state_topic: "shellies/shellydimmer-<my_ID>/light/0/status"
      command_topic: "shellies/shellydimmer-<my_ID>/light/0/set"

Sorry if that is a rookie question :slight_smile:

Thanks again

Ok … I changed to:

    - name: "My Not Working Dimmer"
      schema: template
      state_topic: "shellies/shellydimmer-3C6105E42C4D/light/0/status"
      availability_topic: "shellies/shellydimmer-3C6105E42C4D/online"
      command_topic: "shellies/shellydimmer-3C6105E42C4D/light/0/set"
      state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
      command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
      command_off_template: '{"turn": "off"}'
      brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
      payload_available: "true"
      payload_not_available: "false"
      qos: 1
      retain: false
      optimistic: false

I have changed the name, to make sure to have a new entity (shouldn’t matter but just in case). Still not working. Communication seems to work, so I wouldn’t expect a settings issue?

According to the integration page, it’s local push, not poll.

Can you elaborate? I don’t have any stability or reliability problems with the integration (not entirely sure what you mean by transparency), and I believe that in most cases adding another layer would make these worse, not better.

Ok I found it. Little error in my topic. I needed to put in shellies/shellydimmer**2**…
following working code:

  light:
    - unique_id: mq_kitchen_light_dimmer
      name: "mq Kitchen Light Dimmer"
      icon: mdi:light-recessed
      schema: template
      state_topic: "shellies/shellydimmer2-xxxxxx/light/0/status"
      availability_topic: "shellies/shellydimmer2-xxxxxx/online"
      command_topic: "shellies/shellydimmer2-xxxxxx/light/0/set"
      state_template: "{% if value_json.ison %} on {% else %} off {% endif %}"
      command_on_template: '{"turn": "on"{% if brightness is defined %}, "brightness": {{(brightness | float * 0.3882 + 1) | round(0)}}{% endif %}}'
      command_off_template: '{"turn": "off"}'
      brightness_template: "{{ (value_json.brightness | float * 2.55) | round(0) }}"
      payload_available: "true"
      payload_not_available: "false"
      qos: 1
#      retain: false #came back with error so I took it out, will investigate why
      optimistic: false

Thanks everyone

You might be right here. Anyway, the Shellies respond way faster via mqtt. I built a 2 way switch automation using a shelly to turn a smart plug on. Worked fine at the beginning with the integration but was slow (up to 5 sec) recently. So I assume something changed on the integration itself.
At least I can say I’m happy with the results now :slight_smile:
And I also experienced certain stability issues here. I will observe if it is better now.

Generally, in complex systems, components providing single responsibility is a better strategy/design pattern than a single big component which do everything. Even for the price of communication overhead. Such components are easier to maintain (less complex code, fewer internal dependencies, communication via strictly defined APIs etc). The result is more mature, stable, system with fewer potential issues with possibly faster development pace overall.
Please note, that thanks to strictly defined interfaces (ie MQTT protocol) single components (incl devices) might evolve independently. (see below)

It’s valid for HA. The system is very complex, with internally shared objects, thus creating a pileup of internal dependencies. Constant evolution makes this even worse.
You can see the result after each release when things get broken (even those that - based on the change log - weren’t touched). It was happening several times for Shelly’s integration in the past.
Not speaking about time needed to fix those issues. Sometimes those are days, other times - months. As a end-user you don’t want to experience that.

MQTT is matured component, an industry standard. Something you can rely on more than on HA. It doesn’t add to instability or unreliability. It’s the opposite.

Also note, that restarting HA (which you do possibly often upgrading it) doesn’t cause devices disconnection from MQTT (MQTT is another docker container or might run on on separate server). This separation also adds to the overall health of the system (not speaking about security, which is increased since HA and devices must not see each other).

By transparency, I meant that mqtt gives you insight into communication going on between a device and the HA. Also, you are a creator of the entity declaration. So you know exactly where to look at, in case of problems. Moreover, you can access mqtt and control devices through mqtt, skipping HA. It eventually allows you to use features provided by a manufacturer, not supported by HA.
Obviously, HA integration is the opposite.

For me, the decision to go for MQTT-based solution is a natural one (I’m IT architect, and programmer). For the same reasons, I opted-in for Zigbee2MQTT instead of ZHA.
A lot of people probably think about such architecture as another component to maintain, which adds to complexity, instability etc. But I’m confident to say, it’s the opposite.

I hope it answers your question.

1 Like

Thanks for the detailed response. I’m well versed in software architecture, but as the saying goes “You can solve every problem with another level of indirection, except for the problem of too many levels of indirection”.
Yes, the Shelly integration did have breaking changes in its infancy, but so does the MQTT integration, and from a very cursory look, it has more PRs in minor releases than the Shelly integration, indicating it might break more often. Additionally, you also have to worry about maintaining your MQTT server, and keeping up to date with security patches for your MQTT server.

I’m not aware of any case in which a fix took more than a few days.

Shelly components communicate with MQTT over plain text. I don’t think you can make any serious claims about security.

I definitely could see the case for Zigbee and Z-Wave. These are complicated stacks that are better off maintained independently. Shelly components have a simple interface.

gen1 seems stable for some time, but gen2 are still evolving wich renders in remarkable delays in integration support. Year and half ago, new fw broke integration and it took more than month to update integration (some devices had issues even half year later). not that it cannot happen with mqtt but you can update settings immediately not waiting for devs. (and tbh gen1 mqtt didn’t changed for 3 years afaik in a way it required changes in settings)

I’m aware mqtt integration might change (recently have changed), a bug might be introduced to mqtt addon (which I use) but at first those are not so many updates, at second user base of mqtt is way wider than shelly alone, so probably those components get more attention from devs. Anyway number of issues I experienced with mqtt differs by orders of magnitude comparing to issues reported by users of the Integration.

anyway with the integration you still cannot:

  • change light attributes (brightness, color) having shelly entity turned off (without turning it on)
  • configure switch as light
  • replace one device with another without changing name/id especially when changing its type
  • change shelly device ip address
  • there are problems with removal of shelly devices from the system (see How to delete a device / entity in ha - #10 by petro)

Probably lot more issues related to limited flexibility.

I’m not here to convince any one (or maybe I am :wink: ) or criticise devs work. Just giving idea how advantageous is using mqtt integration comparing to native one. Especially if the only argument against is presumed higher complexity.

Same here. Just trying to understand the pros and cons, and see whether they relate to my setup. I think the main difference between us is that I only use gen1 devices, so I haven’t seen any of the problems with the native integration that you have seen.

I’m not sure these are true. The Shelly integration is getting a lot of love from developers, and is also considered a platinum integration (vs gold for MQTT), meaning it adheres to all of the HA best practices. As I mentioned in my previous post, a cursory look for minor releases shows more MQTT PRs than Shelly PRs, indicating that the MQTT integration needs hotfixes more often.

You could also say about all of these that they are presumed higher stability, to paraphrase your presumed higher complexity remark :slight_smile:.

And we haven’t even discussed the configuration - getting MQTT configured on all of the modules, setting all of the devices in HA (or using some script for auto discovery).

In any case, maybe once I get a gen2 device I’ll experience what you’re experiencing and want to switch. In the meantime, I remain unconvinced.

So I am. I was using Shelly devices way before the integration has been added. And I saw the whole process of incremental improvements. And read tons of reports from users that something doesn’t work, or stopped to work etc. If you jumped into this train recently, then likely gen1 devices are covered pretty well (excluding features that are not supported because HA does not support them natively, but you could use them through automation)

Fair enough :wink: Every architecture works until one find out its limitations. If someone asks for that he should get a fair answer.
It’s not about forcing someone to change his approach if it does work for him. But rather to agree that some patterns are stronger than others. The way HA and devices connect to MQTT (not another way around) and not seeing each other is a very strong pattern.

BTW can you change the IP address of Shelly registered in the Integration without the need of doing any changes to HA? Someone recently wrote that Shelly devices are identified by IP address.

I’ve been using Shelly way before the integration, and even attempted to create an official MQTT based integration before the current one was developed (the idea was to add the discovery functionality but delegate everything else to the regular MQTT integration, but there was simply no way to do that).
Been using the native integration since the first version is came out (even contributed a PR or two), and never encountered any issue. I still have a couple of devices that I haven’t transitioned and are still on MQTT, and they’re actually less reliable (although there are other differences, so it isn’t necessarily related to MQTT).

All of my Shellys are on DHCP, and IP address changes are handled seamlessly.

I was asking about how HA copes with a change of IP address of already registered Shelly devices.
DHCP has nothing to do with that.

Considering that changing the firmware of Shelly a year and half ago broke the shelly integration (and it is only one out of many examples) how is it possible you have encountered none of them? Maybe you are using those devices in a specific way. But then taking it as a common example for advice might be misleading to other people.

As I wrote, my Shelly modules are on DHCP (meaning their IP address can change when their lease expires), and those changes are dealt with seamlessly by HA.

I upgrade HA more frequently than I upgrade Shelly firmware, so it is possible I missed the specific update, but it can’t have been that long. Can you share more information on this incident?

As far as I remember it was with fw 1.8. Shelly changelog mentions rewriting CoAP protocol from scratch. It wasn’t 1.5 ago. It was August 2020.