Why the push for the new ZwaveJS integration vs MQTT?

Configuration Z-Wave
1

I’ve been sitting on the sidelines and watching the Zwave Saga on HA unfold as I prepare to upgrade my network from the old OZW integration. I managed to dodge a bullet by not jumping on the last years Beta Integration bunny trail and want to make sure things are solid and stable before making a move.

I’m left with one burning question as I watch all this - Why is the HA Technical Leadership pushing for the tight-integration websocket approach vs the more loosely coupled MQTT?

It is really bugging me because almost everything about the MQTT approach seems to be better. Its just as fast (if not faster). If you want tools to manage the Zwave network you have to be running the mqtt version anyway - at least for now. Even after HA develops all of the controls they need internally to their integration it will always be difficult to update/keep up as the underlying ZwaveJS evolves. As we progress through the Beta process there are constant version conflicts killing peoples devices (e.g., the recent update to ZwaveJS broke thermostats because the corresponding HA update isn’t done yet) and this is unlikely to change when you have two applications that must maintain tight synchronization in their update path.

OTOH, The loosely coupled MQTT approach appears to give you clean naming for devices and entities that is always consistent while the WS approach requires you to go in after-the-fact to rename devices, etc. (not only easier to configure but MUCH easier troubleshooting!). All the tools to manage the network are embedded and kept in-sync with each release (no dependency on updating HA at the same time). Its just cleaner. And the underlying ZwaveJS devs are the same either way…

I could go on and on.

From an engineering perspective, the tight coupling approach will always be subject to upgrade timing for the integration and HA. This is unavoidable with this approach and requires a discipline to maintain forward/backward compatibility (more work for the devs, and a disaster for the users when they get it wrong). It is a provably more complex approach to manage, carries more technical risk and is the antithesis of modern systems design approaches.

It seems to me that the ONLY advantage to the more tightly coupled WS approach is the HA team’s desire to control all aspects of the UI. Their argument is that this is “simpler” for “less sophisticated” users - but that is a complete red-herring. Every one of the side effects of this approach - device naming difficulties, version control risks, etc. - actually increase complexity and confusion for end users. And HA could achieve their simplicity desires by offering an embedded MQTT broker (oh -wait - they already do!) and linking the zwave2mqtt control panel from the UI - job done. And for those of us with more complex setups we already run an MQTT broker and have no problems with a second website for managing a complex integration - again, job done.

So for me I think I am decided and will go with pure MQTT. Anyone else can still make that choice no matter what HA does with the integration, no worries.

But what I am left is a last self-doubt, a question of the HA team’s motivation…what am I missing? Why are they choosing the more difficult path, the one that that actually adds un-needed complexity, technical risk and end-user confusion over the simpler approach?

5 Likes
2

I tend to agree with your view, also keen on “an answer”. Although I must admit that I wonder what the "the question is…?

3

The question is “what am I missing here”? There must be some more to the story because it is not logical that the HA Leadership would intentionally choose the more difficult development path, the one that carries more technical risk, the one with more end-user complexity, etc., without a good reason. And yet no reason is evident - good reason or poor one, I can find no reason at all. EXCEPT, perhaps, a desire on the HA team to control all aspects of the UI. And I would consider that a terrible reason…

At least for me - before I make a final commitment to the MQTT path - I’d like to give the community one more chance to speak out about why the WS approach is the “better” path.

4

It was brought up a couple of times in the thread that was initially asking about OZW (that thankfully actually prompted some real action) as well as the HA architecture issues on GitHub. Basically, the HA core team would prefer to move away from requiring MQTT for any core addons like this as it creates both the indirection (another potential point of failure) as well as an extra requirement for users. While you may find MQTT easy to use, many people are completely lost when looking at it. The concepts it provides are actually not simple for those that don’t have experience with message queue systems.

3 Likes
5

On top of what @paulstronaut is saying, MQTT discovery in home assistant does not cover all the cases that zwave has. People may have to manually configure zwave entities that do not fall into the discovered MQTT devices.

As a history lesson: Many of the game breaking issues that OZW beta have/has are related to an integrated MQTT server. At first MQTT was assumed to be added by the user. This proved to be too difficult for the majority of users (who posted on the forums and asked questions in discord). So the team added the MQTT server into the OpenZwave beta addon to make it easier, which lead to all these problems. To avoid this again with the zwave_js, MQTT was nixed pretty early on.

Also, websockets give the devs the ability to tailor a frontend similar to the existing Zwave (deprecated) instead of using an outside package (like ZwaveJS2MQTT). The current interface for the integration is lacking because it’s being developed.

2 Likes
6

As somebody coming from Smarthings then Hubitat. Requiring a Add-On container for the z-wave then another for the MQTT, then it’s all got to connect to HA itself.
Jumping into HA I knew it was going to be more work than ST and Hubitat but to a lot of new users that’s so much more effort than they’re used to what is probably their main interface, Z-Wave.
They expect it to just work out of the box.

I’m on a crusade to eliminate MQTT.
It will fail.

2 Likes
7

Interesting discussion here. I migrated from OpenHAB to HA because of their tight and problematic coupling of my Zwave controller. I always had problems with it, but MQTT was no fun with OpenHAB.
Now I’m running HA with everything interfaced through MQTT and am quiet happy, with the exception of Zwave2mqtt. My problems are almost always Zwave related…

So, for me it’s not that much a question about MQTT, WS or REST. It’s about the architectural design decision how to deal with all those low-level protocols and their respective controllers. And when it comes to those incompatibility issues: Every kind of API should make use of versioning (which ZwaveJS according to their documentation does not). Same for Zwave2Mqtt (or ZwaveJS2Mqtt by the same developers): No versioning.

Which brings me to another problem: API design constraints. Problem with MQTT is the lack of any design constraints like we have for example for REST. Same for any WS based API. For MQTT at best we have the Homie convention (quiet dead) and the proprietary HA way. For WS we have nothing. For REST we have a 20 years old dissertation with the mentioned architectural design constraints.

And that’s pretty much the same dilemma for every IoT based solution dealing with multiple low level protocols: How the hack do we unify that manifold for controling and monitoring our components AND configure every single parameter of that mess?

The discussion here seem to reflect the desire to create a proxy in HA for everything possible in a Zwave controller. Maybe they want to go the same route for all the other protocols? Question is: Is that still usable and useful? My experience says: No. Use another interface for such detailed configuration. OpenHAB and HA invest great efforts to NOT following the simple rule of “separation of concern”.

BTW: What part of ZwaveJS is using websockets as it is mentioned here? All I can see is a native JavaScript API.

8

As a new user struggling to learn HA the zwave issue was a nightmare. The one thing I didn’t want was choice, zwave gave me three. I ended up with the js version but only because that was recommended. It seems incomplete; some critical features are missing (heal network, re-interview a node, plot network diagram etc etc).

9

I am one of those new users who failed to install MQTT. I didn’t understand what it was nor why it was required.

1 Like
10

I am missing the heal network functionality as well.
I’ve been told you can install the full z-wave JS to MQTT addon and just disable MQTT and you’ll get that functionality, but I’m just waiting a bit to see if it pops up on the configuration page for a little bit.

If you need help because you need MQTT for something else let me know, I can lend a hand.
It wasn’t easy for me to understand either coming from SmartThings/Hubitat.
There are some free tools that help.

11

I decided to go with zwjs2mqtt for the zwave config ui, but I am looking forward to when the official zwjs gets similar features. The thing I am not looking forward to… having to rename all the entities a 2nd time when I switch from zwjs2mqtt to the official zwjs addon (just finished migrating from 1.4 today).

12

There is a process to not have to do that again.

13

What is this process you speak of? Is there any info on it… or maybe devs are cooking up some migration automagical tool?

14

Petro wrote up a way: Switching from OpenZwave (Beta) to Zwave JS

He also discusses how to go from ZwaveJS to ZwaveJS2MQTT

2 Likes
15

That is the guide I used to migrate; worked well for me. johnboiles shared how he used a root ssh session to copy the files. I didn’t realize I could enable root ssh to hassos. That is good info; they should put that in the regular users manual. Now I’m curious how to disable ssh after I’m done fiddling around.

2 Likes
16

I wondered that too! Please let us know if you figure it’s possible

17

You wouldn’t have to rename things if you’re using the front end and js server integration, and not mqtt discovery. You’d only have to rename if your devices were setup by the mqtt integration and not the zwave-js integration.