Reconfiguring ZWaveJS2MQTT to use MQTT after console only install

Hello,

I’m currently using Home Assistant Supervised, with ZWaveJS2MQTT installed in console only mode (no MQTT) as follows:
image

I have learned that the Scenes feature of ZWaveJS2MQTT will not work inside HA unless MQTT support is enabled as it does not do its magic over websockets, AND I have a bank of 6 blinds that stall and stutter when in motion unless I use a ZWave scene or multicast instruction to fire them all at once. So I need to enable MQTT support for this function.

I have the Mosquito MQTT broker installed and running - I added it after I implemented ZWaveJS2mqtt as it was needed for some of my other integrations.

Here’s the issue:
If I’m not mistaken, turning on that checkbox may cause HA to re-detect / duplicate all of these devices. There are nearly 100 devices on my ZWave network, so that would obviously be ‘difficult.’

Does anyone have a known good process for re-enabling MQTT In ZwaveJS2mqtt without causing a device duplication storm - else am I concerned for no reason?

You also have to set up MQTT discovery to point at ZwaveJS2MQTT’s discovery topics. So just don’t set that up and you can use all the service calls for ZwaveJS2MQTT. I had that on for a while to use services that didn’t exist in ZwaveJS.

Just keep in mind that it will create all the topics for all the devices. You just aren’t acting on them in Home Assistant.

1 Like

No, you probably don’t need MQTT.

The integration supports multicast via the zwave_js.multicast_set_value service call.

The scenes in zwavejs2mqtt will be the same as HA, they just set values in serial manner and don’t use multicast. It doesn’t use any kind of native Z-Wave Scene functionality.

However, if you find the MQTT api to be easier than using the integration service, then use it for that reason. For multicast the API is essentially the same.

If your blinds natively support Scenes (via the Scene Activation and Scene Actuator Configuration command classes), you could use that as well with the set value API. I assume zwave-js supports multicast for the Scene Activation command, but I wouldn’t know for sure.

1 Like

Heh unfortunately I know… I’ve tried firing this all kinds of ways, @freshcoast. It was actually the first problem I started working on when I migrated in last October - but I also was bitten by the ‘700 stick’ bug and just recently was able to circle back after the new firmware stabilized my ZWave network. (I didn’t know if it was related)

The only thing that’s been reliable so far has been the scene activation. Trigger that and all six go smoothly and reliability every single time.

Im working on seeing if a multicast service call works… but I’ve not managed to get that working 100%. Its generally better than just sending ‘on’ to all 6, but about 5% of the time one or more stutter or stall where the scene commands are always flawless…

I’d honestly rather direct associate the devices (Graber roller shades, fw 11.3) with a scene controller (zooz Zen32, 10.x firmware) but haven’t been able to get that to work at all (even after downgrading the security levelof the Zooz…)

How are you using the zwavejs2mqtt scenes, and how are you doing it from HA? It is literally invoking the “write value” API for all the values you specify, which is what HA does. If you’re using the same values as HA does for turning “on” a cover, then there shouldn’t be any difference. If you’re using something else, maybe that explains it.

How am I doing it now?

Originally, a cover group. THAT was trash, it’s just triggering 6 distinct write-values sequentially. Besides the popcorn effect on the blinds, debug log show multiple ‘set value’ retransmits being sent to each blind rapidly (and this is what seems to be causing the stutter) and I don’t know where they’re coming from.

Currently I’m just using a scene.turn_on call which if the network is quiet will trigger each blind sequentially and work about 80% of the time (seems more successful than the cover group) it also ‘popcorns’ - sequentially opens the blinds, or at least in appearance it does.

I have the Zwave scene setup in JS2mqtt. Manually triggering that one works well every single time. Simultaneous opening - really cool effect when all 6 behave.

I did the multicast experiments about a month ago and TBH, it wasn’t very successful. (I was in the same line of thought as you are.) but I don’t have the exact result in front of me anymore and they may have been impacted by my ‘fun with a 700 chip’ so I’m in the process of resetting that experiment.

This is literally what the z2m scene activation does (see link to the code). Are you using Multilevel Switch CC in your z2m scenes?

I beg to differ with you @freshcoast but in my experience it’s a totally different experience between the two.

Ill get side by side video of you would prefer. They DEFINITELY don’t do the same thing from a user experience point of view and it makes me believe there’s a fundamental difference we don’t know about somewhere.

Edit - just checked the scene - I’ve selected a value in the scene Which I BELIEVE translates to multilevel. I’m not issuing ON or Off.

Which in itself is interesting - I might try on/off instead and see if its the multilevel command.

I’m just telling you how the zwavejs2mqtt scenes are implemented, I’m not discounting your experience, and did not intend to. I said there “should not be” a difference between the two based on them being essentially the same implementation, I did not mean that you aren’t seeing a difference.

How about looking at the source code? Here is the code I linked to for scene activation:

	_activateScene(sceneId: number): boolean {
		const values = this._sceneGetValues(sceneId) || []


		for (let i = 0; i < values.length; i++) {
			setTimeout(
				() => {
					this.writeValue(values[i], values[i].value).catch(
						logger.error
					)
				},
				values[i].timeout ? values[i].timeout * 1000 : 0
			)
		}

z2m loops through all of the values in the scene and uses the driver setValue API to set it. This is the literally the exact same API HA uses. My point is that there isn’t a fundamental difference in implementation at the driver API layer. z2m is not using a multicast command internally, or using some other special API.

One difference between the two is that z2m writes all the values in a single MQTT command, whereas HA would send six individual websocket requests. Does that make a difference? Possibly. Maybe HA is waiting for some response (it’s not supposed to be) before the next one is sent. The way to determine that would be to capture detailed logs of the two interactions (across the entire stack) and compare them. As I said, I’m not discounting your experience, I believe you if you say there’s a difference.

I’m curious as to why the end result is different, when the implementations are fundamentally the same. I’m always interested in improving the integration when it’s possible, which is why I’m asking these questions.

Edit - just checked the scene - I’ve selected a value in the scene Which I BELIEVE translates to multilevel. I’m not issuing ON or Off.

I’d still be interested in what exactly it is. Multilevel Switch CC has different ways of opening and closing. HA open/close/position commands will set the target level to 99, 0 and X respectively.

1 Like

@freshcoast. Roger that well figure it out. I’ve obviously been in an all fired hurry (waited since October)

Indeed, Thats exactly what that code says.

Im increasingly convinced its multilevel now.

I think you’re familiar with the cover UI in HA - so is there a way to force On/off instead of multilevel for a scene using a cover?

The current implementation of my z2m scene is directly setting attribute 0 as 0 or 99 depending on open/close. Is there a way to force THAT in the HA scene UI

It sounds like you’re doing the same as the cover entities would, which use Multilevel Switch CC targetValue, with values 0 and 99:

    async def async_open_cover(self, **kwargs: Any) -> None:
        """Open the cover."""
        target_value = self.get_zwave_value(TARGET_VALUE_PROPERTY)
        await self.info.node.async_set_value(target_value, 99)


    async def async_close_cover(self, **kwargs: Any) -> None:
        """Close cover."""
        target_value = self.get_zwave_value(TARGET_VALUE_PROPERTY)
        await self.info.node.async_set_value(target_value, 0)

The async_set_value call is supposed to send a websocket command w/o waiting for any response. So in theory, that’s the same as the Z2M scenes, just with the per-entity websocket overhead, as well as the overhead of HA iterating over the entities.

I’m not familiar with scenes, but in the end they just use entity services. Your scene should be interacting with the 6 cover entities. A cover group does the same, setting a value on a cover group breaks down into setting the value on the group entities, so it’s surprising a scene and cover group would be any different. Also, a scene could end up doing much more than just setting a cover group, depending an how the scene was created.

I took another look at the python library code again, it actually defaults to waiting for a response from the driver before returning back to the application. I was certain it was doing the opposite. I guess this makes sense generally to be able to report failed commands. This could easily explain the behavior you’re seeing though (depending on how long it takes the driver to confirm a command). Presumably, for a Cover group, HA Core waits for the entity service calls to finish instead of doing them in parallel, but maybe scenes don’t have that restriction (I might be completely wrong on those assumptions).

Besides Multicast, there are two options you could try, but both would require dropping the use of a group. One would be to use the zwave_js.set_value service, setting wait_for_result to False. This means nothing will wait for the command to return a result, so they would be executed more quickly. The other choice would be to use the new parallel actions, setting the cover position for each entity in the group in a parallel action (could this be templated to expand the group?). Then they aren’t dependent on each other. Either of these are more equivalent to how z2m implements it.

1 Like

I just tried a light group vs a scene with the same light entities, and the results were the same. The service calls to turn on the lights are execute sequentially, and wait until the command is finished in the driver. I notice one difference with the scene is that it won’t make a service call if the state of the entity is already at the desired state. Also, with my three lights, setting the brightness and getting a response was basically instantaneous. It’s possible the covers might take longer, that all depends how the driver signals completion, I believe it’s usually after the command was accepted by the controller.

In summary, I could see a scene being better if some of the devices are already in their desired state; that would cut down on the number of service calls and Z-Wave commands. Otherwise, a group and scene seem to be the same to me. And finally, compared to zwavejs2mqtt, HA entity services will wait for the command to finish and they are executed serially. This time may be variable depending on the targeted devices. zwavejs2mqtt issues all set commands w/o waiting.

If you’re interested to see what’s going on for yourself, add these logging settings to your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.zwave_js: debug
    zwave_js_server: debug

You’ll see the websocket command being issued as they are sent, and the responses being received in the home-assistant.log file or stdout. Here’s a setValue command corresponding to turning on a light:

2022-05-09 13:59:12 DEBUG (MainThread) [zwave_js_server] Publishing message:
{'command': 'node.set_value',
 'messageId': 'fc34605f7be84c0c88db83407f4611fc',
 'nodeId': 43,
 'options': {'transitionDuration': 'default'},
 'value': 46,
 'valueId': {'ccVersion': 4,
             'commandClass': 38,
             'commandClassName': 'Multilevel Switch',
             'endpoint': 0,
             'metadata': {'label': 'Target value',
                          'max': 99,
                          'min': 0,
                          'readable': True,
                          'type': 'number',
                          'valueChangeOptions': ['transitionDuration'],
                          'writeable': True},
             'property': 'targetValue',
             'propertyName': 'targetValue',
             'value': 0}}

Here’s the response to that command:

2022-05-09 13:59:12 DEBUG (MainThread) [zwave_js_server] Received message:
WSMessage(type=<WSMsgType.TEXT: 1>, data='{"type":"result","success":true,"messageId":"fc34605f7be84c0c88db83407f4611fc","result":{"success":true}}', extra='')

In this case, I believe the response is sent after the driver verifies that the command was accepted by the controller, it doesn’t include any timing verifying that the value was correct.

The resolution of the logs is only 1 second, so you can’t get any perf measurements better than that, but it sounds like from your case that isn’t a problem, I imagine it’s in the seconds.

Are your cover devices one of these? https://products.z-wavealliance.org/Search/Index?regionId=-1&searchText=graber

Or, if you look at the Debug info tab of one of the nodes, does it say "isFrequentListening": true?

1 Like

Yes, It’s the Somfy Zwave motor. It’s the second one in that list. They retail under names Springs Window Fashions, Bali and Graber

Debug tab - actually has a timeout/frequency (in ms) instead of 0/1
“isFrequentListening”: “250ms”,

OK, maybe that’s part of the explanation. They are FLIRS devices, which means they’re battery powered but are listening for commands. The "250ms" value basically means that it can take 250ms for the device to acknowledge the command. My thermostat is "1000ms", so it responds after a second. 250ms doesn’t sound like a lot, but maybe with 6 devices it’s noticeable, or maybe it’s taking longer than that. My mains-powered devices are in the tens of milliseconds.

1 Like

Yep, add in a VERY busy Zwave network… (~100 devices, including a few active power monitoring outlets) Likely the same reason the 700 bug hit my install so hard. I’m on 17.2 and still seeing devices drop occasionally but at least it’s tolerable now.

I’m setting up experiments to test service calls to see if I can match the scene behavior with what you found The first one will be wait_for_result = false

Will let you know what I find.

This service call sent the commands really quickly in my testing:

service: zwave_js.set_value
data:
  command_class: '38'
  property: targetValue
  value: 99
  wait_for_result: false
target:
  entity_id: '{{ expand("light.test") | map(attribute="entity_id") | list }}'

Replace light.test with your cover.group. Regular group domain entities can be expanded by the integration automatically.

1 Like

There is a PR up that will resolve this problem for normal service calls (e.g. cover close): Set PARALLEL_UPDATES to 0 for all zwave_js platforms by raman325 · Pull Request #71626 · home-assistant/core · GitHub

There is also a PR up that will resolve the problem for zwave_js services without requiring you to set wait_for_result to False: Parallelize zwave_js service calls by raman325 · Pull Request #71662 · home-assistant/core · GitHub

1 Like