The Future of Z-Wave in HA - QT-OpenZWave

blhoward2 · January 6, 2021, 2:16am

You all see this?

Move Z-Wave support from OZW to zwave-js

opened 11:14PM - 05 Jan 21 UTC

closed 03:05PM - 11 Jan 21 UTC

## Context Currently a new Z-Wave integration is in the making, based on Open…Zwave 1.6, using the "OpenZwave daemon" as intermediate bridge with mqtt as communication protocol. While development had a flying start we're at a point there are multiple issues why I'm proposing to revisit the choice for OpenZwave(QT): - We ran into several mysterious issues because of the fact that mqtt is used as protocol, mainly unpredictable results in fetching the (startup) state from retained messages. - Added complexity of an MQTT broker and the OZW daemon for both users and developers. - Stability issues with the OZW daemon, not starting at all or sudden restarts, not so pleasant for user experience. - Device/Z-wave compatibility issues that need to be fixed upstream (meaning: OZW). - While OpenZwave is a very popular opensource project and many people rely on it, it's main development is driven by one developer. In all the past years he did not have much help from other contributors to the project. One reason is that it's written in the programming language C++ which is just not so popular by nowadays young developers in comparison to e.g. python and javascript. Because this programming knowledge is also missing at the Home Assistant devs, there is a long queue of getting issues fixed. - The above mentioned core developer of OZW has given a statement just recently that he's taking a break from OZW and all it's demanding users which gives a bit of a dark future at this time for OZW unless (finally) some people step up and help out coding C++ with him. There's a lot of reason to doubt if we're betting on the right horse with OpenZwave (daemon) for what's supposed to be the "shiny new" Z-wave integration for Home Assistant that should finally make the Z-Wave experience rock-solid. This leads to frustration both with the devs actively working on it and the community. There must be an evaluation of alternatives and that's where this proposal is all about. ## Proposal For implementing Z-wave there are unfortunately limited options. The only opensource library that ever existed was/is OpenZWave, except that it's python wrapper has gone stale. Really the best option would be to create a Z-wave driver completely written in python but that will require a busload of developers and time to pull off. Recently a new opensource Z-wave implementation has seen the light: Zwave JS. A driver written from the ground up in javascript/typescript gaining a lot of traction: - It has good documentation and a good codebase. - As the language is more popular, code contributions are more likely. - It has given thought about easily exchanging device-specific configs. While this project is (just like OZW) also primary developed and maintained by a single developer (Al Calzone) it is more likely that others will help out. Even most of the Home Assistant devs/contributors know their way in some javascript/typescript. A few members of the small team that was/is actively developing the new OZW integration for Home Assistant has explored options the last couple of days and exchanged thoughts with the developers of Zwave-JS. Even though it means that we will have to smash 70% of our work from last year in the trash, we still think we should make the switch (now we still can) to Zwave-JS instead of OpenZwave, how harsh this may be for both Fishwaldo (developer of OZW) and everyone contributing to the new integration. ## Consequences PRO: The actual interaction with the Z-wave stick/device will still be detached from Home Assistant but this is a good thing as this will not restart the Z-wave mesh while Home Assistant is restarted, so that will stay. PRO: Instead of talking over MQTT to the OpenZwave MQTT daemon, a "ZWave server" will be written which can be fired up in it's own process (as docker image, addon etc.) and it will be the bridge between Z-Wave hardware and Home Assistant. In fact this bridge will serve any client interested in utilizing the Zwave mesh. For example Zwave-to-mqtt and Home Assistant can be used **side by side**. PRO: because of the above mentioned an MQTT broker is no longer a hard requirement which reduces the complexity, especially for new (less tech) users. we can manage the entire path automatically. PRO: As we have the ability to actively work with the developers of Zwave JS and we have knowledge of the upstream code we can actually contribute in fixing stuff upstream. PRO: Zwave JS has some features that are not supported in OZW such as firmware updates. Also from a first impression it seems to be a bit faster and stable. PRO: No more instability issues with the zwave daemon that was haunting many users. CON: Zwave JS is still missing support for a few command classes/devices such as garage doors but this could be easily added by us or any contributor. CON: A fully automatic migration of legacy/current ZWave in HA to this new implementation will be more complicated because of the fact that the internals are handled differently. I'm for sure missing a few points here but the bottomline is that the overall feeling by me and a few others is very solid to make the switch. read all about Zwave JS here: https://zwave-js.github.io/node-zwave-js/#/README

daphatty · January 6, 2021, 4:21am

This thread’s title needs to be renamed. Clearly, OZW is not the future of Z-wave in HA.

murphys_law · January 6, 2021, 3:50pm

Maybe good to keep it for historical sake. Haha

joachimmg · January 6, 2021, 6:01pm

Very exciting news! Thanks for sharing!

panteragstk · January 6, 2021, 11:43pm

Well, I guess that’s that then.

Guff666 · January 7, 2021, 1:29pm

A while back I posted on here about how maybe it was better to use a dedicated ZWave hub to separate the ZWave domain. At the time, I changed my mind and chose to implement ozw on a separate HA instance located in a better place (RF-wise).

I love MQTT and use it for a stack of other tasks in the house. It IS my communications platform of choice. Consequently, I have a resilient MQTT infrastructure to cope with the occasional vagaries of the network.

The only problems I have are caused by ozw’s restriction to use qos=0 (which I asked about in a separate thread here.

I’m not convinced that changing the ZWave implementaion again is a good choice for me. The Pros/Cons for changing highlight the single-developer issue that has affected ozw development (no reflection on @Fishwaldo), and there’s no guarantee this won’t re-occur. Given the critical status of ZWave in my house, therefore I am thinking again of swapping to a commercial ZWave hub

petro · January 7, 2021, 1:32pm

The likely hood of this occuring is much lower because the codebase is more widely known. OpenZwave is written in C++ where as Zwave2jsmqtt is written in JS. Pretty much anyone who contributes to HA will be able to contribute to Zwave2jsmqtt because the frontend is written in JS/TS.

Guff666 · January 7, 2021, 2:03pm

I hope you’re right.

petro · January 7, 2021, 2:04pm

We already have 7 collaborators.

joachimmg · January 7, 2021, 2:24pm

HA is using the driver ZwaveJS directly to make their own implementation and not Zwavejs2mqtt. However, zjs2mqtt would be a good alternative to @Guff666

I’m excited about two things, thats on the roadmap: OTA updates and S2 security (which should speed up communication?).

petro · January 7, 2021, 2:25pm

Yes, I was simplifying it for people who don’t understand the difference. When official name comes out, then we’ll call it that

joachimmg · January 7, 2021, 2:25pm

Ah, ok, but might also confuse as the HA implementation won’t use MQTT.

joachimmg · January 7, 2021, 2:26pm

Haha okei, but i guess that’s zwave-js-server

petro · January 7, 2021, 9:20pm

I believe it’s going to be called “Z-Wave JS”

joachimmg · January 7, 2021, 10:18pm

Whatever is good, as long as it works.

petro · January 7, 2021, 10:19pm

it’s programmed to turn your lights on at midnight or when sleeping

joelp · January 9, 2021, 2:49am

It amazes me how quickly… the HA community has adapted to the feedback and produced an alternative to its Z-Wave integration. More importantly I think what I see in the HA community is a willingness to look at improvements instead of dwelling on “flaws”.

But okay. I guess HA is over… this thread as quickly as it began and is moving forward.

Awesome!

blhoward2 · January 21, 2021, 7:18am

For those of you not following the other thread about switching paths:

Just an update for everyone. I’m in the process of importing every single certified zwave device into Zwavejs2mqtt and node-zwave-js (the new server backend). I’m about 98% finished and working on bugs now so confident it’ll work. It’s a few hundred additions and revisions to around 1,000 device files representing devices from all over the world (after accounting for model changes…its 4,700 entries total). In talking with the maintainer, our goal is to have it auto-PR newly added devices each week or so. At the moment the database even includes pre-release devices so hopefully device file issues become a thing of the past.

From switches to bulbs to thermostats to mousers and graters (whatever the hell they are…the Russians have some interesting devices), we’ll have device files for it all.

FreelancerJ · January 21, 2021, 9:49am

Are you telling me the Russians have worked out automating their cheese graters? And here I am, still grating my cheeses with my hands, like a peasant!

FreelancerJ · January 21, 2021, 10:08am

This is also amazing news, really looking forward to testing it out!

Given you’ve been going through config so much, maybe you could answer a question for me? When devices are added with security, does it change what config is used to identify them?

I’ve just installed a Aeotec Dual Nano Switch and two single Nano Switches, and when I securely add them to my dockerised Z-Wave JS controller (using a Z-Stick gen5+) once the interview stages have completed, they show up as “unknown device”, but they work fine with all features when insecurely added. On the other hand, my Multisensor 6 adds securely and still shows up all properly identified…
Would that be resolved once the config database is complete, or am I barking up the wrong tree?