Cannot add ZWave Integration (also ZWave integration vs. ZWaveJS add on)

[ImaginaryTango (Hal)]

I have a Pi that is the 2nd HA system on my LAN. (We have a house and barn, and the barn is far enough away it can’t be reached with wifi and ZWave or Insteon signals won’t go that far, thus a 2nd instance of HA. The two buildings are connected via fiber.) I’m trying to hook both systems together through Remote Home Assistant. Not having a lot of luck with that, but it’s a factor in what’s going on.

I wrote the latest image for RPi (8.5) to an SD card a few days ago and put it in this Pi, which has an Aeotec Z-Stick 7 plugged into it. When I turned on the Pi and waited, HA installed then I added the ZWave integration and it found the Z-Stick and all the Z Wave devices that had been registered on that stick previously, under another system. So that worked just fine.

After that, I was trying to get Remote Home Assistant set up and had problems with it. Since RHA is a custom component, there are issues. Due to the errors and problems I had, I burned a new image of the latest HA (again, 8.5) to an SD card and plugged it in. I went through several images like this before stopping that. Currently I have an image of HA on an SD card. I’ve messed around with adding HACS and trying to add Remote Home Assistant. I’ve added a terminal/ssh add on and that’s about all I’ve added.

When I add the ZWave integration, I get this dialog:

From there, I click on “SUBMIT.” Then I get this:

I’ve seen this come up over and over, since I’ve been testing with a number of freshly written instances of HA on SD cards. It happened on a brand new copy of HA and it’s happened on every instance I’ve tried on it tonight. It was only a few days ago that I had put an instance of HA on this very Pi and I could set up the ZWave integration easily and HA quickly found all the devices on it. Now, with a new copy of the same version of HA, on the same Pi, I keep getting an unknown error.

Also, on a side note, I see the ZWaveJS Add-on is on this system. I take it that the ZWave integration needs the ZWaveJS add-on to work properly?

I can’t find anything useful in the logs that shows any errors. Is there a particular log I should be looking at for information on this?

I wondered if this could be related to a permissions issue for the device nodes in /dev, since I’ve seen that before, but I can’t change permissions since it’s a read-only FS. (I wouldn’t have done that long term, but I was hoping that would tell me if that’s the issue.)

How do I find out what’s going wrong on this (and other test installs) tonight that didn’t go wrong a few days ago with a new HA install?

[tmjpugh]

Is barn zwave only or a mix, Wi-Fi and zwave?

If zwave only, you should only add RasPi running zwavejsmqtt. You don’t need HA for zwavejsmqtt.
If Wi-Fi also, just add an AP and connect Wi-Fi devices to main install since you said you have fiber between barn/house. This will also extend Wi-Fi to barn. Wondering why you need 2 separated HA installs and thinking you made it to complex?

@ImaginaryTango

Also, on a side note, I see the ZWaveJS Add-on is on this system. I take it that the ZWave integration needs the ZWaveJS add-on to work properly?

If using adddon you don’t use integration. Likely you need to remove integration and setup and use addon.

@ImaginaryTango

After that, I was trying to get Remote Home Assistant set up and had problems with it.

I forget how RHA work but maybe this is better? Also, still not sure you need Full HA if only for zwave or Wi-Fi. Believe connecting all back to main instance using lan connection over the fiber is best.

Just in case I was wrong about fiber I also suggest using a pair of Ubiquiti Nanostation Loco to bridge network. They’re about $50USD each and work without fail in all weather.

[ImaginaryTango (Hal)]

@tmjpugh

If zwave only, you should only add RasPi running zwavejsmqtt. You don’t need HA for zwavejsmqtt.

Short answer: Insteon. I know it works via power lines and radio, but for some reason, it doesn’t work from house to barn. (Most likely because the barn is on a different circuit than the house.)

TLDR background (I try to avoid all this, but it impacts a lot of decisions and planning for home automation): We custom built this house and it has a storybook feel to it. That means push button light switches in some places. (The barn is not as custom done, though.) When building, I bought a lot of Insteon Micro Module Dimmers to go in light switch wall boxes so they could work with our switches. The idea was that the electricians could put them in while doing the wiring, but they didn’t understand them (even with simple and clear wiring directions!), so I’ve put them in later and have some leftover that will be used in the barn. Also, with this being a custom design house, and us starting construction in our 50s and wanting to live here forever, everything is planned out with the thought of, “What will happen if we end up in a wheelchair or when there’s only one of us left?” So a single interface works better and is also easier for friends and family who visit us often enough for me to setup as users on the HA system. (I tried using 2 servers on the HA iPhone app - I can sign into one, then sign out. Even though it finds both systems, I can only use one at a time and have to sign in, then out, then into the other - switching servers might work IF I could see an easy way to include both that are on the LAN.)

I’m including all the detail because I find, in discussions, it helps to explain why I’m doing things in what’s not always the simplest or best way for most situations.

@tmjpugh

If Wi-Fi also, just add an AP and connect Wi-Fi devices to main install since you said you have fiber between barn/house. This will also extend Wi-Fi to barn. Wondering why you need 2 separated HA installs and thinking you made it to complex?

Here’s the problem:

The barn is 350’ from the house and from before May until after September, it’s a thick, dense forest. Overall, since I had to run a trench down to the barn for power and water (and return line for sewage), it was more reliable to run a conduit and run fiber through it. I went over it thoroughly with my cousin, a networking engineer, when I was designing the house and he said that with the trees, don’t even try radio waves, due to the distance and the denseness of the growth.

As an aside, and not really related to these issues, that’s the SHORT run. To finally get good internet, I already had at trench from the house, under the driveway, to our garden shed, during construction, for when we could finally get an ISP to run lines to us, 1/3 of a mile off the roadway. They won’t, but Starlink came along. Due to the trees around the house, even with 1 acre cleared, I had to run fiber through that 100’ conduit to the shed, then make another 800’ trench out to a post in our front field for the Starlink dish. I have 1/0 wire (2 conductors and ground) and conduit with fiber in it running that full 800’, to a weatherproof box from Altelix, where I can plug in the Starlink router and connect it to the fiber going to our house.

@tmjpugh

If using adddon you don’t use integration. Likely you need to remove integration and setup and use addon.

Here’s what I don’t get: I never added the add-on! I was wondering how it got there and if, maybe, the integration automatically adds it. I do have both on the main system, but don’t remember adding the add-on. It’s running, but it’s not set to start at boot for for the watchdog to restart it, so something else (definitely not me) must have started it. So this is a mystery and I admit lack fo experience and ignorance in this issue.

@tmjpugh

I forget how RHA work but maybe this is better? Also, still not sure you need Full HA if only for zwave or Wi-Fi. Believe connecting all back to main instance using lan connection over the fiber is best.

I’ll look into that. I asked, in this post, about what to use for this situation and the only answer was RHA. I’m open to other ideas. (While this may be best discussed on that thread, I’ve found that sometimes mentioning related problems in a thread leads to answers from people who may not have seen the other thread.) I haven’t had time to learn how to set up panels in Lovelace yet, so if it’s possible to set one up that talks to 2 HA instances, I’d even be okay with that and not directly connecting them.

[tmjpugh]

@ImaginaryTango

for when we could finally get an ISP

I was thinking fiber was back to main house but sound like it is separate to incoming ISP connection

I forget people have forests….I’m flat ground fairly desert area so never much of an issue. Bridges are definitely non-starter.

I think the statestream or event stream integrations will work well for you. Basically HA2 entitles will appear and will be controllable from HA1. It uses MQTT connection between the 2 instances. Add the HA2 entities to dashboard like any other device however you want.

Log info

Logger

[ImaginaryTango (Hal)]

@tmjpugh

I was thinking fiber was back to main house but sound like it is separate to incoming ISP connection

I’ve put in 1,500’ or more of fiber optic cabling. I was just including the reference to what it took to run 800’ out from the shed to where we could get a good spot for the Starlink dish. Then there’s another 200’ or so of fiber from the shed to inside the house. Add to that 500’ from the house to the barn. I was including the info about the internet connection as a way of saying we have a LOT of fiber out here and it’s been extremely handy and useful!

(I had some CAT6 in the barn, under the concrete floor that blew out components from lightning - that doesn’t happen with fiber in the ground instead of wires!)

@tmjpugh

I forget people have forests….I’m flat ground fairly desert area so never much of an issue. Bridges are definitely non-starter.

We’re even. I love visiting my sister and her ex’s hotel in New Mexico and love the desert area, but I keep forgetting people live places where they don’t have trees all around! Actually, I remember going through this with someone a year or so ago - and forgot until now. Weird the different things we can take for granted that change from place to place, isn’t it!

@tmjpugh

I think the statestream or event stream integrations will work well for you. Basically HA2 entitles will appear and will be controllable from HA1. It uses MQTT connection between the 2 instances. Add the HA2 entities to dashboard like any other device however you want.

That sounds good!

I have had a really hard time getting info from people on RHA. I thought I remembered discussion forums on GitHub, but I could be wrong - there isn’t one for RHA, so I reported one of my issues as a bug and FINALLY got in touch with someone. So it may turn out I have a choice. I’m exhausted after 2 nights of almost no sleep, but later tonight I’ll have time to look over your recommendation. If I can get it working, that’ll be great.

And thanks for the info on Logger. Even going through the Logbook (which doesn’t have but so much) and the logs under System, I was not finding info that was helping me. I couldn’t find what was happening with the failed connections on RHA or with the issue with the ZWave integration not installing. (And it’s a serious puzzle for me why I could set up the ZWave integration on this same Pi a week or less ago with a new HA image on an SD card and why it won’t set up easily now with the same situation.)

[ImaginaryTango (Hal)]

I’m continuing the ZWave issue (since we’re also discussing linking two HA instances as a separate issue) on this thread, since I’ve added logs and more information there. I think, after looking a the logs at debug level and finding a long stack trace, I think the reason I can’t add the Z-Wave integration is due to a bug. I tried it without any devices and still got the bug.

I may go for an older version of HA for the Pi (if I can find the image) and install that and see if it adds the integration, then upgrade it.

[vilord (Jesse Campbell)]

So you have a network link between the house and the barn / they are on the same private network? If so, you could just run one HA with two copies of the zwave integration on your primary HA, connecting to two copies of the zwavejs server (which is part of zwavejs2mqtt, and is also part of the zwavejs addon, you would use the zwavejs2mqtt on the pi as a docker container)

[ImaginaryTango (Hal)]

@vilord

If so, you could just run one HA with two copies of the zwave integration on your primary HA, connecting to two copies of the zwavejs server (which is part of zwavejs2mqtt, and is also part of the zwavejs addon, you would use the zwavejs2mqtt on the pi as a docker container)

Check my comment with the photo included. While both intsances of HA are on the same LAN, they can’t be linked by wifi. They’re in different buildings, connected by over 500’ of fiber optic cabling at least 2’ underground. I need a Z-Stick in the house and another in the barn and when I start putting Insteon devices in the barn, I’ll have the same situation.

[ImaginaryTango (Hal)]

I’ve got this working, but it turns out there’s a bug in USB discovery routines that I need to report. I had to use ZWaveJS2MQTT instead of ZWaveJS.