My First Add-On - CBUS to MQTT using cmqttd now available as a HA add-on

I wanted to migrate a friend of mine from his CBUS homebridge instance to Home Assistant and wanted all the CBUS stuff (CGATE etc or cmqttd ) to run as an add-on so he has nothing to manage.

I had a crack at taking the cmqttd docker available on github and making it run inside HA as an add-on.

If anyone wants to check it out my first add-on is here: https://github.com/MtSamsonite/ha-cbus2mqtt

2 Likes

Thanks for doing this ā€¦ it works well for me with a Wiser as the CNI that I use for testing (once I put port 10001 on the end of the IP address!)

However I was not able to get it working with my normal setup ā€¦ I have a CBus SIM module connected to the uart serial port on a RPi4 ā€¦ I would expect that to be available at /dev/ttyAMA0 (or maybe ttyS0, serial0 or serial1 epending on how the Pi is configured) however when I try that (those) I get a PermissionError like this:

Running cmqttd with flags: --broker-address core-mosquitto --timesync 300 --broker-port 1883 --serial /dev/ttyAMA0 --broker-disable-tls --broker-auth /etc/cmqttd/auth --verbosity DEBUG
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/serial/serialposix.py", line 322, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
PermissionError: [Errno 1] Operation not permitted: '/dev/ttyAMA0

I think the cure to this is to add a ā€œdevices /dev/ttyAMA0ā€ (or whatever serial port is configured in cbus_connection_string) to the docker config, or with a --devices parameter when the container is started ā€¦ I added that to a manual ā€œdocker runā€ from hassos and it worked ā€¦ ā€œuartā€ in the config might also work but Iā€™m not sure if that will work for USB-Serial convertors ā€¦

Also maybe a config option to enable ā€œā€“privilegedā€ (ā€œfull_accessā€ in the config) would help for debugging ā€¦

Summary: this command works from the OS

docker run --device /dev/ttyAMA0 -v /tmp/cmqttd:/etc/cmqttd -e "TZ=Pacific/Auckland" cmqttd cmqttd <add args here>

@djaggar

Thank you Dave for the feedback. It is appreciated.

I think its a rookie mistake, Iā€™m still fumbling my way around Docker and HA addons. Thank you for helping. I took a little look into it. I just added uart:true and usb:true to the config file and issued an update, but unfortunately am unable to test.

If you get motivated to try it again please let me know how it goes. I also understand if you donā€™t.

I had also thought of listing the devices to choose from rather than an input string but would still have to have an entry for IP as well. However I thought this way was less confusing than forcing a user to enter something in both fields for serial and IP (even though one would be ignored). Welcome feedback.

Mike

Good news, that worked.

I think the simple input string for the address works OK, and is fully flexible given even a Pi4 has 6 serial ports that someone could potentially use, plus the USB ports, all with diffrent names ā€¦ so youā€™d have quite a few choices if you listed them all ā€¦

I can also confirm the my tag file (project.cbz) was also used, nice work!

Using MQTT login details in /etc/cmqttd/auth
Using C-Bus Toolkit project backup file /share/project.cbz
 
Local time zone: Pacific/Auckland
Current time: Tue, 23 Aug 2022 20:56:06 +1200
Running cmqttd with flags: --broker-address core-mosquitto --timesync 300 --serial /dev/ttyAMA0 --broker-disable-tls --broker-auth /etc/cmqttd/auth --project-file /share/project.cbz --verbosity DEBUG
2022-08-23 20:56:07,832 Transport security disabled!
2022-08-23 20:56:07,844 send: ResetPacket()

The only other feedback I have is about cmqttd in general ā€¦ itā€™s weird to me that even if it is given a project file (which basically says which group addresses are in use) it still publishes every lighting group to MQTT ā€¦ I hacked that out fairly simply so that only named groups get published ā€¦ if you are keen to add that (maybe with a config switch to make it optional) itā€™s simply changing cbus/daemon/cmqttd.py from

        for ga in ga_range():
            name = labels.get(ga,  f'C-Bus Light {ga:03d}')
            self.publish(conf_topic(ga), {
            ...
            self.publish(bin_sensor_conf_topic(ga), {

to

            name = labels.get(ga)
            if (name):
                self.publish(conf_topic(ga), {
                ...
                self.publish(bin_sensor_conf_topic(ga), {

Also (sorry!) while I remember thereā€™s a one line bug fix here that I thinkā€™s worth incorporating (uncompressed xml project files donā€™t work without this)

https://github.com/micolous/cbus/commit/ef30f308ea722e0935d5edd10a25766e77e9090c

Thanks again for doing this,
Dave

Iā€™m so pleased there are other C-Bus users out there using this integration. Iā€™ve been running cmqttd as a separate container and it works well but I have remember to restart it when I restart HA. This add-on makes things a lot neater. Thank you @Samsonsite

While Iā€™m here thought @djaggar do you have issues with cmqttd missing messages if sent in rapid succession? Iā€™m trying to use either a group or NodeRed to switch off a number of lights at the same time and it seems to only handle about 3 of 6.

Love your patch to eliminate all 255 groups though!

One more Q - I find some of my groups come back after a reboot at 1% - ever seen that behaviour in your setup at all?

Pls ignore, I think this requires a change in libcbus.

@Samsonite thanks for the add-on.

How do people deal with ramped OFF commands from C-Bus that indicate the light is ON with brightness zero?

I am a newbie to HA, so have installed the cbus2mqtt add-on and run with default HA MQTT Discovery.
All C-Bus light GAā€™s appear in HA and all is good.

The question comes about when a GA is ramped to OFF in C-Bus, e.g. in a Goodbye Scene. The MQTT info received in HA shows the Binary Sensor as OFF, and the Light as ON with Brightness zero and ramp duration.
It is odd to watch lights in HA all turn on when C-Bus turns them all off.
It gets more mixed up when also using Homekit Bridge , where the lights in homekit now show 100% ON ā€¦

This has been around for a while so how do people deal with the anomaly?

Iā€™ve been working on moving to @Samsonsite 's HA Add-On. But first I wanted to rebuild the docker image with some of the updates that have been made to help cater for the CNI buffer filling up, limiting the groups provisioned, etc.

I notice that my container crashes on a regular basis - event trying the original from micolous it restarts after almost every message to the CNI is sent. Anyone else seeing this behaviour? Nothing in the container logs showing a stack trace or anything.

If you have your code on GitHub I am happy to test tonight. I have been working on the cGate implementations as these have been significantly stabler for the last 6 years;

Share a link to your repo/container and I can run in test environment

Sure.

The only changes (ā€œonlyā€ to be used lightly here) was to change the base image and add the ability set the log level via environment variable.

I am also ā€˜tryingā€™ to modify the Add-on, bumbling step by step through tutorials.

The mod in cmqttd.py for the ā€˜Ramp to OFFā€™ bug is simple, along with another mod to address the current warning ā€œDiscovered MQTT entities with a name that is equal to the device name, This stops working in version 2024.2.0ā€

Now if only the build & publish was as simple for a very occasional linux visitor :frowning:

If you have the changes just send me the files and tell me which ones change. or tell me what to modify. I can then build and update if you wish. If you prefer to handle yourself, I can try and assist with build and publish.

Do you know what changes were required to fix the buffer overloading on a CNI. This was also a big complaint.

I also need to advise users if things will break with the upgrade (more around the naming changes).

Much appreciated. This is irritating me, I am currently stuck trying to build. Not being familiar enough with docker I was trying to use the homeassistant/builder but keep running up against errors.
Do you have a cheat sheet for your method?

The changes (breaking) I wanted to make to cmqttd.py are compared to your cbusha here:
cbusha commit

Nothing has been tested.

And changes to your ha-cbus2mqtt as a consequence of changing cbusha:
ha-cbus2mqtt commits
The second commit to change url is a bit of tinkering that will probably be reversed.

I did not look at the buffer problem as I have programming access to C-Bus and keep the scenes on there, then HA only needs to set the single scene trigger.

I donā€™t really have a cheatsheet, only a memory which fades more each day.

I know I struggled for ages to build the images. the things that caught me out was first trying to do so on a machine that though it did list VTX support did not seem to be supported by the docker installation to do multi arch builds. It wont work if you are doing this in a VM or container as I understand it as it needs direct access to VTX, whereas with a VM the hypervisor is already utilizing this. (at least thats how I understand it though may not be technically correct)

You mention your changes have not been tested. The first step is to get it running as a local add-on, this way you can test everything before we build the image. Just make a directory config/addons/cbus2mqtt and place all your files in there, install your local addon and test.

To build the image, once I had tested the add-on as a local built add-on I simply copied the folder to another machine (my HA was running on a VM, I needed to use a Docker install on my PC as the VM would not support multi-arch builds.

When you have Docker installed on a OS installed on bare metal with a processor that supports VTX, then you simply put the add-on in a folder and run the below where the -t parameter points to the directory where the addon has been copied and the my_addon has been replaced with the name eg: cbus2mqtt

docker run \
  --rm \
  --privileged \
  -v ~/.docker:/root/.docker \
  -v /my_addon:/data \
  homeassistant/amd64-builder \
  --all \
  -t /data

I am unfortunately heading interstate until Tuesday next week and wont have time to look at this until next weekend if needed. As I mentioned get it running for yourself as a local built add-on (not using the image) and get it so its tested and your happy with it. That is the first step before building the image.

Thanks your help but something in my setup was not working, literally starting at bare metal each time and trying docker desktop, docker host, sudo ā€¦ I kept hitting different errors.
Anyway the CNI has now decided to it has had enough so I am snookered.
Given the price of these things I will take it as a sign it is time to move to a SHAC and give acMqtt a try.
Now to LUA 101 ā€¦

You have tried power cycling the CNI?

Yep, also tried replacing the power supply, itā€™s a goner.

Hi & thanks @Samsonsite for this, I have it installed & working.

I have a question, is there any chance you might extend this to allow other cbus applications to be visable to HA?

I have some speaker controlls via IR using application #55 and would like to control from HA.

Keep up the good work.

Steveā€¦

Hi all, totally new to Home Assistant but loving it. Have CBUS and great to get this all talking together based on the great advice provided by teh Community.

I have 1 issue I just canā€™t work out.
CBUS in HA works perfectly.
Got myself a Cloud account and all seems ok, although I still have trouble accessing from off network ( maybe the issue or part of )
Linked Google Assistant and all appears, but when I switch on a CBUS light it works, but the Google Assitant a few seconds later shows it as off, even though HA and CBUS have it on.

If I dim the light sometimes it sticks.

Any thoughts?

Appreciate your help in advance,
Marty

Without changing anything - this has suddenly started working - so after a month of head scratching and sending out a ā€˜helpā€™, today its behaving 100%.
Wonder if something external has changed such as as Nabu Casaā€¦