Hey @Tinkerer - sorry to revive an old thread but I have a question that may relate to above. I have a client who has over 180 CBUS lights and blinds which he wants to use with SIRI. As it’s CBUS, I use cgate<->cgateweb<->MQTT between HA and the CBUS to control everything. Everything seemed good at first but then the client started complaining that lights were sometimes not responding. After some analysis I realised that we’d hit the 100 devices per node limit you have with HomeKit. As a result I split his configuration into 2 x Home Assistant docker containers and have them both point to the same instance of MQTT. One HA (slave) did just lights (89 of then) and the other (master) did everything but excluded lights being pushed through to Homekit (the master does everything so that the owner only needs to go to one address). This means that one HA does lights for HomeKit and the other HA did blinds for HomeKit, keeping both nodes under 100.
So this seemed to stop the Homekit limit errors in the log and lights were more consistent but he is still having problems. When he asks Homekit to do a large number of devices at once (eg, turn on all living area lights) it sends the commands but not all make it through. I’m still investigating patterns (I think it’s something to do with the response from MQTT) but I am thinking that it might be something to do with my split HA both talking to MQTT at the same time. Perhaps one HA is getting some of the MQTT response and the other is getting the rest?
I don’t have MQTT Statestream set - do you think that is causing it? It’d make sense but I’d be really interested in your expert thoughts - or does the fact both HA are going to the single broker mean it should update both accurately. Sorry I’m still fairly new to MQTT.
I did initially have the QOS set to 0 and I though Eureka! I’ve found the problem. But setting it to 2 makes no difference…
I am going to do a test soon where I am going to shut down the master HA and just run the one controlling lights and see if the issue goes away.
I’ve split this into a fresh topic, there’s little value in resurrecting old threads that are only vaguely related
Now, I’m far from an expert on MQTT Statestream, but I’m not sure you need it here either. A few thoughts:
Do you have to individually control 89 lights? Can you not group those and expose the groups to Siri? Same goes for the blinds, you can group those too.
Is Siri required, what about Google or Alexa?
Doesn’t sound like you’re doing any automation, so there’s likely no need for MQTT Statestream since it’s all voice control.
Hey Tinkerer - thanks for putting into new topic - I probably should have done that from the start.
To answer your questions:
If I understand your question properly, Yes I need to individually control them as the client will ask Siri to just turn on one light. The advantage of using Siri is he can configure his house how he likes in Home App and then use Siri’s smarts. eg, turn on downstairs lights (in a zone in Home App). I could alter it to have the grouping/areas in Home Assistant but there is clearly a bug with my setup causing problems that I want to fix.
Yes they are an apple house and would prefer to use their phones/watches etc.
We will be doing lots of automation - that is for stage 2 & 3 of the project.
I understood Statestream to align two copies of HA but because all the lights/blinds in HA is driven by MQTT (via CBUS) the sync should happen automatically? ie, lights goes on in CBUS->MQTT->updates both HAs? Same if I turn light on in HA, goes to MQTT which in turns updates CBUS & other HA?
Just to add to this, I am not certain the 2 x HA is causing the problem. It could be something in the CBUS, Cgate, Cgateweb bits as well or just something wrong with the MQTT & HA on it’s own. I just have a loose theory it is something to do with the 2 x HA as I’m presuming its the only thing that is different from other peoples CBUS installations.
I definitely need to do some more testing to find out whether commands are getting through to HA properly and/or if its MQTT or further down the line? (limited time with the client unfortunately). I don’t want to waste peoples time diagnosing something where I can isolate it down a bit better.
I do have the following screen shot that I got from the cgateweb console where the request was for all kitchen lights to go on then request for all off (you’ll need to click to see it properly). As you can see there are varying results. 26, 28 and 30 don’t seem to respond with a confirmation of off after the command is sent…? And then they don’t go on as the system thinks it’s still on I presume?
I also have a large C-Bus install interfaced via MQTT. However not using Homekit and really don’t control large ‘groups’ of lights by sending all the individual light commands. My experience has been pretty reliable but I realise I’m not hitting the two aspects of your usage that cause trouble.
As a first point I would be fairly confident that MQTT is dependable especially with something like Mosquitto. TCP connections mean data isn’t dropped although badly written clients might lose messages. But I would suspect the problem lies elsewhere either in cgateweb (possibly cgate but unlikely), on C-Bus or at the other end (HA/HomeKit).
What I do here is my larger groupings are ‘scenes’ on C-Bus , programmed into some suitable device. If you can get your client to define some of his groupings as scenes you can then just trigger the scene from HomeKit and C-Bus will do the rest. I realise this takes away his flexibility to do things himself in HomeKit but he would in return get dependable operation. He could mix scenes with additional C-Bus groups (a group in C-Bus parlance being a single C-Bus endpoint address). This would cut down the number of devices in his groupings. Also you can address a whole DIN mount dimmer or relay unit as an ‘area’ e.g. kitchen if that helps.
I know this doesn’t fix the problem but it might buy a happier customer whilst you continue to try and find where the real problem is (which I’m interested in too).
Just wondered - are there any C-Bus bridges used on his network and is the C-Gate interface via Ethernet or PCI serial ?
Sorry for taking so long to respond. Yes, I will have a chat with the client and the CBUS guy to see what they say. I am pretty sure some groups have been setup but one of the attractions of going this route is he could do things himself as there is only 1 CBUS guy who knows what he is doing in Canberra and he is nearly impossible to nail down to a job. But I guess the groups would be a one off and could help a little until I fix it. He is a pretty chilled guy and not too stressed about the issue. Plus this work isn’t costing him anything… (getting all the CBUS lights working with HA was a free Proof of Concept as I hadn’t done it before - only once I prove this working successfully will I start charging for the rest of the work/automations/integrations etc).
I asked the client to do some tests for me but he still hasn’t done them. Just to isolate whether the issue is the multiple HA or not and to try fully determine where the messages are getting lost - HomeKit to HA or HA through to CBUS (via mqtt/cgateweb/cgate).
I am not sure about the Cbus bridges - I didn’t have anything to do with the initial install. I will try find out however. I do know he has an Ethernet adapter that my Cgate is using. There is also another copy of cgate running as part of the cbus toolkit on a PC at their place - mine cgate is a copy of that.
I will send him a reminder about trying the tests to see if we can isolate it down and go from there. Let you know how it goes.
It is something to look into as with bridges C-Bus groups can (and often need) to be linked across them. I have found in operation on my C-Bus network they occasionally don’t stay in synch.
What would be the ideal solution would be to allow your user to create groups in HA but then pass that ‘group’ information to C-Bus via C-Gate so that the ‘group’ can be triggered with one command. I am not sure if that is even possible (within C-Gate capabilities).
I use a different direct to C-Bus implementation (not via C-Gate) and so it is I think possible with that - although I’ve never tried to do it.
Another alternative (which you’re probably not going to like) is for the user to add a C-Bus Wiser unit to their system. This gives them a nice web based UI where they can (I believe) manage ‘scenes’.
Thanks @xAPPO. You mention you use a direct to CBUS implementation. Do you have any info on that? At the moment I am using CBus<->CGate<->CGateWeb<->MQTT<->HA. If there is an easier or alternative way, I’d be interested…
BTW - organised with the Client to go around this evening and do some proper testing. Hoping to isolate which integration is causing the problem and go from there.
hey @xAPPO - did some testing tonight and I think I have isolated it down to cgate/cgateweb, most likely cgateweb. I have raised as an issue with the author to see if he can help. More information here: https://github.com/the1laz/cgateweb/issues/20
Thought you might be interested and interested in any thoughts on what you think might be the problem.
The groups you need to create are on C-Bus aka ‘Scenes’ (groups as used on C-Bus refers to one endpoint like a light - confusingly) - this then allows you to send just one command to C-Gate to activate or trigger the scene (and to press just one key on a C-Bus switch to control the scene).
You didn’t mention if there are hardware C-Bus bridges installed on the network ?
The ‘direct to C-Bus’ was a solution I created that is my own own design. It is a $150/200 hardware board with a C-Bus interface, RS232/485 and Ethernet. It can connect directly onto C-Bus or, at lower cost, to a C-Bus CNI (Ethernet) or C-Bus PCI (RS232). It only supports C-Bus lighting and compatible applications and doesn’t use C-Gate at all instead directly decoding the C-Bus wire protocol. You can control and track all C-Bus groups realtime via Ethernet and it also supports a couple of HA controllers via serial links - like HomeVision and IHC. HV and xAP were my chosen HA protocols and controller at that time, hence the project. (I am a C-Bus Enabled partner).
I made up a couple of hundred units but have sold them all in dribs and drabs over the past 10 years I guess. I have no more left unfortunately and can’t economically do a re-run unless there’s 50+ (there’s about 20 takers atm). So order 30 and you’re all set to go !
Kevin
PS I think I’ve seen another C-Bus solution on here ??? If so, and if it was a cgateWeb issue, then that might work ? I may be wrong it might have been on the OH forum.
Ok cool thanks Kevin. I just called the CBUS guy and he confirmed there is some bridges but they only relate to the blinds which may or may not have issues but I am just seeing it in the lights at the moment.
He did however give me a few suggestions with how I call the commands. He suggested that I only use the levels not on/off commands. So set to 0 for off and 255 (or 100 with cgateweb) for on. So I will try that next. He also suggested that Cbus is not a high bandwidth network (which I already guessed) and that the network might just be getting confused being hit with so many commands at once. He is suggesting putting some millisecond delays between commands which I’m not sure I can do but perhaps the author of cgateweb can. Will see what he says.
I’m going to try that avenue first but if I have no success I will ask the CBus guy to setup a scene or two for me to test with as you suggest.
This shouldn’t be an issue. It is true C-Bus is low bandwidth and can be overwhelmed easily. But you are not sending commands to C-Bus you are sending them to C-Gate via MQTT. However MQTT can not pace changes at all.
C-Gate has an inbuilt queue and C-Bus has ‘pacing’ ‘ack’ signals that it sends back so all commands sent to C-Gate should make it to C-Bus intact and in order. Whether CGateWeb could somehow introduce a behaviour that overflows messages I am not sure. I don’t know if C-Gate has an ability to ask a client to ‘back off’ for a minute if it’s having trouble passing the commands onto C-Bus but as it’s a TCP connection it can create it’s own pacing for received data.
The issue is more likely that MQTT topics are changing but not all those changes are being passed to C-Gate. Whether this is a code issue or a buffer overflow within C-GateWeb I am not sure. MQTT will not pace reporting changes (except by inherent TCP flow control) , something must accept and queue them when presented.
PS Just to mention there are some C-Bus applications like MRA (audio) and Thermostats that generate a lot of continual C-Bus traffic which can quickly swamp a badly designed C-Bus network. Bridges are often used to segment this traffic from the lighting application , regaining bandwidth.
Hi @xAPPO - did some more testing last night and have had some progress. I changed the client’s MQTT config to optimistic mode: true and it seems to have fixed all lights going on and off with Homekit as it doesn’t rely on the dodgy response it is getting back from cgate as to whether it was on or off. However I am not certain it is a good fix and lights are still a bit off on one of the two instances of HA I am running (see my OP) so I need to do some more research on the MQTT confirm in HA to see if there are any other settings that can help fix/mask the problem.
I was wondering however, are you able to post how you have configured your MQTT/HA so that I can see if I have mine right/wrong? Would really be helpful even if it just supports what I have already in place.
