Can we revisit the move to qt-openzwave?

Configuration Z-Wave
107

I think the idea is that we don’t just depend on one way that in turn is dependant on a single person. The point is that the way it currently stands with OZW is that it increasingly doesn’t work great for many people today, myself included and those aren’t just bugs for us, they’re complete show stoppers that means we will have to throw it (HA) all away.

I’m very new to HA, I’ve started out in the last few weeks, I’ve selected the most ‘supported’ hardware (RPi+GoControl HUSBZB-1 USB stick) and software (official HAOS images for RPi, official add-ons such as OZW) to provide the best possible experience annnnnd… my Z-Wave fails 1-3 times a day. I stopped adding Z-Wave devices at 74 because it was just getting more and more unreliable so I haven’t bothered adding the rest and I haven’t bothered going further with the rest of HA.

Since 90%+ of my ‘smart home’ is based on Z-Wave devices, if I cannot get it working reliably, I cannot get HA working reliably, no matter how good the rest of it is and how good the rest of it is actually working for me. It doesn’t matter how good/better/faster HA is over SmartThings if the official path HA has taken for Z-Wave doesn’t work for me currently (and seemingly an increasing number of others), I’ll have to go back to SmartThings.

No HA means no Nabu Casa revenue, this should be a concern for the HA project. We’re not trying to piss anyone off but we are pissed off, we’re not trying to make anyone leave, we’re trying to get help on how to get the necessary information and who to give it to so that hopefully we can fix that and at the same time voicing concerns and asking questions over essentially hinging the entire future of HA (for those of us that heavily depend on Z-Wave) on one person.

It’s all very well and good saying “well step up and code it / contribute to it yourself” when people have been doing just that but PRs for huge show stopper bugs are left hanging in the breeze. As it stands, HA and my dependency on Z-Wave means that HA is completely unusable to me if OZW is to be the official direction and while my GF is extremely forgiving, I can’t expect her to use this so I’m going to have to go back to SmartThings no matter how much I wanted to migrate to HA.

It’s okay to ask people to “be patient” and “just wait for the fix” but it’s a problem when that ask is indefinite on a show stopper. As a new user, this hasn’t been a good experience, not the bug seemingly with OZW, it’s the handling of it.

8 Likes
108

Just to add in my 2 cents - I would broadly agree with a lot of comments by Paul and others in this thread.

I jumped to the new OZW approach quite a while back after having issues with a few z-wave devices, hoping that development would kick on at a pace (yes I was happy to take that risk). Pretty sure I initially did so after a post from one of the main HA update threads heaping praise and a golden future on OZW. I think the key here is that the OZW currently stands in contrast to the rapid and fantastic pace of development in the rest of the HA environment. My fairly simple setup runs, but is definitely not without issues.

I’m not a developer, and sometimes struggle understanding the development process and interelations between the various add-ons, integrations and core - it seems to me that I am not alone in this based on a few other comments about responsibility for direction.
I would additionally add that I am extremely grateful to all those who contribute their time to this project. I would help more myself if I had the time and skillset!

3 Likes
109

Perhaps if you could document your system and errors in the forum, the community might be able to help with diagnosis of what’s going on. Clearly your experience is very different from mine, and it would be beneficial for everyone to understand why.

2 Likes
110

I agree - and with progress apparently having come to almost a standstill for many months, it’s clearly making people nervous. And while this naturally leads people to question the future of this project, I am prepared to trust Paulus Schoutsen when he says that there are volunteers available who are working on OZW via MQTT, and that they’re collaborating and doing their best to make that as great as possible.

I realise that this won’t come as much comfort to those whose ZWave networks are not performing properly.

2 Likes
111

I would love to but forgive me as I’m not entirely sure how to give the community the information they probably need nor do I fully understand the things I’m working on.

From an outside, with basic understanding the problem from my perspective seems to be squarely OZW, nothing else seems to be experiencing issues. All I know is that periodically and randomly, my entire Z-Wave mesh goes completely offline or ‘unavailable’ to HA. The Zigbee portion to the GoControl HUSBZB-1 Zigbee/Z-Wave USB stick remains completely responsive, the Z-Wave portion I believe still shows up on the hardware list under the supervisor section.

Trying to reach the OZW Admin GUI through HA’s GUI is unresponsive. If I restart the OZW add-on I can usually get back to the OZW Admin GUI but when I attempt to connect it just hangs on ‘Connecting - 0%’, some time later the mesh will come back online, sometimes I have to restart the OZW add-on a few times, sometimes I have to entirely restart HA itself before it will come back.

Then it’ll last a few hours, sometimes as long as 24 hours before it hangs again. I’ll happily delve into any logs or whatever people need.

1 Like
112

A good starting point would be to read this post first, and use it to create a fresh post describing the issue in the ZWave category of the forum.

1 Like
113

I’m going to add another $0.02 on this topic, which with everyone else that added $0.02 is quickly becoming the core financing for HA :smiley:

I came from a commercial home automation platform and have been sandboxing HA for a couple of weeks, about ready to switch completely over.

In a commercial platform where I needed a piece of software for $250 + $80/year plus a dedicated computer to run it on (if I wanted the performance), I expected a certain level of stability and response to issues - and I got them. I went into HA knowing and understanding that it is open-source and, as such, meant I lost that safety net. I knew I would be at the mercy of community support and open source developers who do this in their spare time.

My switch from accountability to open-source was intentional. As a developer the open source appeals to me because I know I can jump in and easily contribute (and did so on my paid software for many add-ons I wrote for them). I know that if I don’t like something or want to add functionality then I can write it myself, albeit with a bit of a learning curve.

This is to say that you have two options: pay for software and the support that comes with it and be at the mercy of the development cycle of that product or get a free system with hundreds of contributors where the development cycle is not as “structured” but gives you tons more options.

I can say that my last system was fantastic, it was well vetted and tested and was solid as a rock - however their integrations were about half what HA has and I wanted that extra integration. The fact that you have two Z-Wave options is not a bad thing, it’s a good thing because it gives people different options depending on their needs. And if you don’t like it, you can roll up your sleeves and contribute your own version.

Is either system perfect? No, but neither is commercially developed software. Almost all of my Z-Wave devices have been obtained because the software devs wrote the code to support them, not because it was my first choice. It worked out well enough, but there have been cases where I had an unsupported device that I waited six months or a year to get full support for. While they gave you some options to tweak your own Z-Wave commands, HA allows me to actually tweak the source if I want.

There’s a few devices I want in the HomeKit integration, having written the entire HomeKit integration for my last software I’m familiar with the protocols and intend to contribute to that project once I figure out the HA structure.

The point is, open source is a good thing if it’s maintained and while Z-Wave might be idle at the moment, anyone at any time can jump in and kickstart it again and from what I can see, HA is maintained open source, not some pet project that lays dormant until the person who started it decides they want a new feature for themselves. That is why, out of all the various open source home automation platforms, I chose HA.

11 Likes
114

Personally I started with the old integration and moved over to the beta shortly after it was announced. Yeah, I’ve had issues with the beta but that’s because it’s a beta. It’s expected and I knew that going in. I tend to like being on the bleeding edge with most things I use. While I’ve had issues with it, I find it works much better than the old integration in terms of device support and when it hasn’t, I’ve written and contributed the device configs. Overall it’s been pretty stable for me. Do I wish it worked better and some of the bugs were fixed? Yes. But that will come in time. I get why people are upset but I don’t get why we need to jump the gun and rush to judgement when someone takes a break. Personally I’m amazed that this project is free. I came from smartthings and continue to be amazed by this project.

4 Likes
115

Just as an FYI for this thread, I emailed the maintainer of zwavejs2mqtt offering to help develop at least a bare bones integration to get it going, but apparently someone else already offered to do so this morning. So, whoever you are, thank you and please let me know how I can help.

I appreciate you confirming that another approach would not be shut out by the project. That has been my concern all along. Much of the messaging has made it seem like qt-openzwave was THE approach and the responses to questions have made it seem very much like no other option would be welcomed if one were developed. Talks about paths forward and such can be taken as very exclusionary. Obviously people are free to work on whichever project they wish to work on. As you say, that is the nature of open source.

6 Likes
116

If the zwave2jsmqtt project was API/MQTT compatible with qt-openzwave, then you wouldn’t need a different HA integration. Both the qt-openzwave daemon and the zwave2jsmqtt daemon could both piggy-back on the same ozw HA integration. Since the zwave2jsmqtt project is so new, maybe that’s something they attempt to do, since it would mean that your choice of daemon would be transparent to HA. :man_shrugging:

3 Likes
117

There are many difficulties with this. The ozw daemon hardcodes the messages under the openzwave topic. And zwavejs2mqtt supports a bunch of things that ozw does not, like some sort of controller migrations and secondary controllers. You can also modify or create custom sensors which may or may not carry through. (The new version is new but they started with the very stable zwave2mqtt. It’s actually further along development wise than qt-openzwave in both features and stability.) Additionally ozw 1.6 and zwave2js treat some things like double taps differently at a more basic level in the command classes. It would also be a nightmare to manage. They tweak the ozw integration at times to fix things to my understanding.

Because qt-openzwave puts part of the zwave on the MQTT topic this isn’t as straightforward as saying you have to call everything the same thing or tell the integration the values like with regular MQTT. It’s a bit like saying if only Honda would make their engine more like Tesla’s, they could both use the same engine since they both turn a crankshaft.

You don’t actually need an integration with zwavejs2mqtt. It works fine with the stock MQTT integration unlike ozw. But then you have to publish MQTT messages for some things or to trigger add nodes (or use the web ui) and such versus having an integration that would act as a middleman.

I get why you and others would want it but even skimming the code for the ozw integration makes it clear it would never work without a rewrite of qt-openzwave to make it more of an MQTT-complaint client that posts ordinary MQTT topics.

118

I haven’t run zwave2jsmqtt, so I’ll defer to your knowledge, but my point is that the ozw integration is just simply representing the zwave devices as HA devices, and then creating some device-specific entities. It then also will be providing a nice UI to execute some of the common zwave commands (add/remove node, etc). It doesn’t know anything about the daemon-level implementation.

If it’s not possible, then that’s fine.

119

I’m not an expert but my understanding of ozw is that it does actually know a lot about the qt-openzwave daemon and is doing some of the processing. It’s not purely acting as an MQTT client.

The zwavejs2mqtt integration would do as you say. That’s why it does work out of the box with the MQTT integration. Zwavejs2mqtt creates all of the entities and puts them on topics. My understanding (which could be wrong) is that Qt-openzwave puts the entire zwave message on the topic and then the integration breaks it down and creates entities. That’s not MQTT-compliant. And it means that any api change by one would have to be matched by the other.

Also, zwavejs2mqtt already has a nice (and complete) ui. It feels years ahead of qt-openzwave because the backend is new but the daemon itself is not.

While it may be possible to create a bandaid layer that translates everything to ozw’s custom formatting, it would be just that. Zwavejs2mqtt would be forever stuck to matching ozw’s api changes going forward.

120

I guess I’m confused then. What would the “bare-bones” integration look like that you mentioned earlier? What’s the point?

121

Some people have had trouble using it because there isn’t a service call to set lock codes, as an example. Or to send zwave parameters. You have to publish an MQTT topic. My vision was to make an integration for now that did that and allowed access to the web ui from within HA as it is already fully functional, and maybe later it could handle its own discovery so changes to the MQTT integration don’t break it. For now it would continue to rely on the MQTT integration to create the entities as that works fine. That way api changes and such to the control infrastructure wouldn’t require changes as it is developed. I have no idea what the vision of the other person is.

That wield also be super quick to do. All the service call does it translate it into an MQTT message.

This is similar to part of what the ozw integration does or will do. You can currently publish an MQTT message with ozw to set a zwave parameter. But the integration makes it easier and more native. The same with refreshing nodes, etc.

122

Ok, cool. That’s the same stuff that the ozw is working on right now (writing service calls, abstracting the MQTT interaction, etc), but you’re saying that it needs to interface with MQTT differently. I’m guessing at least some of the code and/or look and feel could be replicated, but with the MQTT implementation being different for the different daemon backend. Probably wouldn’t be too hard.

The UI panel would be handled via the addon, not the integration

123

Only people running supervised can use the add on. The ozw integration handles the ui, not the other way. (Right now it can only do a few things.) The difference is that there is already a fully functional control center here. The service calls are just to make automations and the like more easily able to control those things.

124

I was talking about the ozw-daemon vnc ui that’s provided by the addon, which is what I thought you were referring to (the zwavejs2mqtt equivalent). The UI from the integrations page would be handled by the integration, I agree.

125

The equivalent of the VNC part is done and is already a website. Since it is already a website there is no need for two separate controls. That’s only necessary with ozw because of the odd choice to require a qt interface. It looks much like what the built-in one looked like but with more control/features.

Screen Shot 2021-01-02 at 9.05.49 PM
Screen Shot 2021-01-02 at 9.05.49 PM1582×644 73.1 KB

This is already running fine in Lovelace. I have it setup as a panel. And unlike the current vnc ui, you can change association groups, modify sensors, create custom sensors, hard and soft reset the stick, create and change scenes, etc.

2 Likes
126

It seems that, in a perfect world, HA developers would define what they want the MQTT API to look like, and then zwave backend developers would implement it.

But I take it that horse is already out of the barn, and the approach was “we’ll work with whatever qt-openzwave is sending us,” which you are suggesting is not a particularly implementation agnostic protocol. Oh well.

3 Likes