Home Assistant Add-On: X10 (CM11) to MQTT Gateway

madbrain · March 29, 2022, 4:50am

So, I finally got the add-on installed, and read the doc, but some questions remain.

mmotley999/hassos-addons/blob/a12b79a0204495c15a55833a89a21ef362d95fa3/x10mqtt/DOCS.md

# Home Assistant Community Add-On:  X10 to MQTT Bridge



This add-on provides MQTT control of X10 devices for the CM11 and CM17A "Firecracker" RS232 Serial interface to X10.

When using a CM11, the addon also monitors for X10 changes that occur outside of Home Assistant (e.g. the use of X10 remote controls) and updates the status in Home Assistant.

Only ON and OFF commands are supported.  Dimming is not currently supported.

## Configuration

Example add-on configuration:



```json
    "serial_port": "/dev/ttyUSB0",
    "cm17_in_use:" false,
    "mqtt_host": "core-mosquitto",

This file has been truncated. show original

Re: the top “Configuration” section, it was not obvious to me where those properties are supposed to be stored. Is it in configuration.yaml in the global section ? Or through GUI ?
I finally found it in the GUI in Configurations / Add-ons / X10 MQTT / Configuration tab.
The doc above mentions the Mosquitto add-on. It was not obvious at all how to set it up.
I tried to add it from the add-on store, but it did not get show up at all in the screen at Configurations / Add-ons .

However, when I go to Configurations / Add-ons / Add-on store / Mosquitto broker, the uninstall button appears, so it seems it got installed after all. I can see the start/stop/restart / uninstall buttons . And there is a configuration tab. I was able to start it manually. I think there is some major bug with the add-on installation in the GUI right now.

What’s very odd is that the Hostname - core-mosquitto appears to be hardcoded. It’s not part of the configuration. The hostname resolves to 172.30.33.3 in the terminal. Looks like it’s setup in the local DNS server. The address can be pinged, even though it’s clearly not part of my LAN. This must be a docker thing.

I created a new user/password per the doc at addons/DOCS.md at 4cd2e8846a9e3373a6de87f6f194a6c5492fece5 · home-assistant/addons · GitHub , and set those up in the X10 to MQTT add-on configuration tab. I was able to start the X10 to MQTT add-on . It seems to be fine.


[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[21:16:40] INFO: Configuring Heyu...
[21:16:43] INFO: CM11 is enabled
starting heyu_relay
Establishing MQTT to core-mosquitto port 1883...
(Using MQTT username mqtt-user)
Waiting for MQTT messages and monitoring for remote changes
Connected to MQTT broker, result code 0

I gathered that I also needed the MQTT integration to be installed, in order for HAOS to find X10 devices. The integration asks for a “broker” rather than “host” or “hostname” as named in the X10 to MQTT add-on and Mosquitto broker respectively. And there is no default value. I setup “core-mosquitto” as in the broker field, and used the previously added mqtt-user and configured password.
I still didn’t see any X10 devices being discovered, unfortunately. I have the following configured in configuration.yaml :



switch:
  - platform: mqtt
    name: "Office ceiling light"
    state_topic: "x10/stat/b1"
    command_topic: "x10/cmd/b1"
    payload_on: "ON"
    payload_off: "OFF"
    retain: false

Does this look correct ? I wonder about the - sign in the platform line. I tried with and without it, but there was no change.

Perhaps the add-on hasn’t taken into account the changes from configuration.yaml yet. If so, what do I need to do to make it do that ? Is “restart server” the only option ? Or should Server controls / YAML configuration load / Manually configured MQTT entities also work ?

I restart HAOS with a power-down / power-up . The log from the Mosquitto add-on shows this :

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] mosquitto.sh: executing... 
[21:54:56] INFO: SSL is not enabled
[cont-init.d] mosquitto.sh: exited 0.
[cont-init.d] nginx.sh: executing... 
[cont-init.d] nginx.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[21:54:58] INFO: Starting NGINX for authentication handling...
[21:54:58] INFO: Starting mosquitto MQTT broker...
1648529699: mosquitto version 1.6.12 starting
1648529699: |-- *** auth-plug: startup
[21:55:02] INFO: Successfully send discovery information to Home Assistant.
[21:55:03] INFO: Successfully send service information to the Supervisor.
1648529699: Config loaded from /etc/mosquitto/mosquitto.conf.
1648529699: Loading plugin: /usr/share/mosquitto/auth-plug.so
1648529699:  ├── Username/password checking enabled.
1648529699:  ├── TLS-PSK checking enabled.
1648529699:  └── Extended authentication not enabled.
1648529699: Opening ipv4 listen socket on port 1883.
1648529699: Opening ipv6 listen socket on port 1883.
1648529699: Opening websockets listen socket on port 1884.
1648529700: Warning: Mosquitto should not be run as root/administrator.
1648529700: mosquitto version 1.6.12 running
1648529700: New connection from 127.0.0.1 on port 1883.
1648529700: Socket error on client <unknown>, disconnecting.
1648529740: New connection from 172.30.32.1 on port 1883.
{"result": "ok", "data": {}}1648529740: New client connected from 172.30.32.1 as 1NAgxre6li50NIxp50EX4T (p2, c1, k60, u'mqtt-user').
1648529781: New connection from 172.30.33.2 on port 1883.

The log from the X10 to MQTT add-on is the same as already posted above.

It seems like I shouldn’t be far from getting it to work, but unfortunately, no dice yet after a couple hours of playing with it. Any help from those who succeeded would be greatly appreciated !

madbrain · March 29, 2022, 5:19am

I’m running HAOS 2022.3.7 on a Pi3B+.

I think Mark assumed that MQTT integration was present, yes. I believe mine was installed, but not configured. I had to configure it.

In order for the MQTT integration configuration to work, I had to install the Mosquitto broker add-on. I did so. But still no luck. See my longer post above.

Thanks for the tip about checking the configuration. I have not run into that yet. I normally use raspiBackup to my NAS for my other PIs. Looks like this option is not available in HAOS, and I have to investigate different backup solutions. I’m still very new to HAOS.

markm · March 29, 2022, 3:32pm

@madbrain, if you go into the Configuration tab of the add-on, you can also enter an IP address or hostname for the MQTT broker in the ‘mqtt-host’ setting.

I set it to default to ‘core-mosquitto’, so it works straight away if you have the Mosquitto add-on installed. But, an IP address of another broker should work as well.

See below:

yeeCdn · March 29, 2022, 8:46pm

I am no expert, but in your long post, #3 last line of x10 log showed it had connected to your mqtt broker.

You can test by publishing an mqtt command via developer tools.

If working, the X10 addon log will show something like this

madbrain · March 29, 2022, 9:29pm

Mark,
Thanks. I use HAOS, which has this crazy Docker layer on top of almost everything, and it seems that the real IP addresses and hostnames are opaque. That makes it very confusing to fill out these fields.

But the biggest confusion comes from the fact that the field in the UI is called 3 different things in the Mosquitto add-on, MQTT integration, and X10 to MQTT add-on .

In Mosquitto add-on, it is called hostname, and hardcoded. There is no way to change it :

In your add-on, it is called mqtt_host, and configurable :

In the MQTT integration, it is called broker :

I realize you only have control over your own add-on, so this is not your fault, but some consistency would go a long way to figuring things out.

I believe I matched all these to the same name, but things are not working yet, unfortunately.

Thanks for creating the add-on, and any help you can provide getting it to work.

madbrain · March 29, 2022, 9:41pm

Thanks. I did the publish exactly as you instructed.

Here is what the add-on log looks like afterwards :

Waiting for MQTT messages and monitoring for remote changes
Connected to MQTT broker, result code 0
Remote status change, publishing stat update: x10/stat/b1 is now OFF
Remote status change, publishing stat update: x10/stat/b2 is now OFF
Remote status change, publishing stat update: x10/stat/b1 is now ON
Remote status change, publishing stat update: x10/stat/b2 is now ON
Remote status change, publishing stat update: x10/stat/a1 is now ON
Remote status change, publishing stat update: x10/stat/b1 is now OFF
Remote status change, publishing stat update: x10/stat/b2 is now OFF
Remote status change, publishing stat update: x10/stat/a4 is now ON
Remote status change, publishing stat update: x10/stat/a4 is now OFF
Remote status change, publishing stat update: x10/stat/a4 is now ON
Remote status change, publishing stat update: x10/stat/a1 is now OFF

Not clear why it’s showing those status changes.
The b1 device didn’t change status while I was doing this, or any of the other X10 devices.
The a4 X10 device does not exist in my house, at least not as far as I know
a2, a3, c1, c2, c3, c4, d1 and e1 exist, but are not shown in the list.

b1 is an XPS3 wall switch that controls my office light. If I press the switch manually, and turn it on or off, there is no update in the log.

If I use an RF remote and RF plug-in transceiver to turn that switch on or off, the add-on log does update.

Remote status change, publishing stat update: x10/stat/b1 is now ON
Remote status change, publishing stat update: x10/stat/b1 is now OFF
Remote status change, publishing stat update: x10/stat/b1 is now ON
Remote status change, publishing stat update: x10/stat/b1 is now OFF
Remote status change, publishing stat update: x10/stat/b1 is now ON

I still don’t see any X10 devices in the GUI under Configuration / Devices and services, unfortunately. Is there something else I’m missing ?

madbrain · March 29, 2022, 9:48pm

Actually, I see the “Office light switch” under “Overview” in the “Switch” section at the bottom. And I can operate it ! That works. It’s major progress. I also see another switch, “Office audio amplifiers”, which is X10 b2, that I defined in configuration.yaml, even though I didn’t perform another publish step for it.

It looks like I need to assign it a unique ID in order to manage it through the GUI, for example, assign it to a particular room, or use it as part of automations. I’m not sure how/where to assign that unique ID. The X10 to MQTT add-on doc didn’t shine light on this. Maybe it’s a normal HAOS or MQTT thing to do, but I just don’t know how.

yeeCdn · March 29, 2022, 10:43pm

Pressing any wall switch will NOT have any effect on Mark’s addon, the status of the light in HA UI won’t change either. The wall switch doesn’t need to send any powerline data when pressed, as it can control the light itself. No powerline data = no effect seen by addon.

As for unique ID, or other HA stuff, ask Mr goggle. Your addon is working, now comes the hard stuff of learning HA . Just remember HA is constantly being updated, so some answers from goggle may be out of date. To your previous query of backup, the goggle drive backup addon is supposed to be the best n easiest to setup. It is the one above the X10 addon in my previous screensnot.
Have fun (n some frustration, until you get the hang of it)

madbrain · March 29, 2022, 11:24pm

Yes, it makes sense there would be no update in the plug-in in those cases. Unfortunately, seems like an X10 hardware limitation. I almost never press the XPS3 switch in my home theater due to its inconvenient location. That’s why it’s an X10 switch in the first place, to allow it to be turned on remotely.

The three XPS3 switches in my home theater on the other hand are controlled either from my IR543, RF remote, or by hand. And I guess the latest case means the switch state will not be correctly reflected in any automation program that uses the CM11. Too bad those switches aren’t 2-way devices, and there is no way to manually query the status.

Re: Google, believe me, I have been using Google a lot. It’s how I got to the state last night, where I was only short of the publish step that you provided. And I still haven’t seen that step mentioned in any other document I read, so I’m curious how you became aware of it.

As to the unique ID, the doc contradicts itself.

In one place, it says it can’t be changed. And in another, it says a few selected integrations allow the user to define a unique ID. Both on the same page, too.

Clicking the link to mqtt brings this page

No mention of unique_id even under advanced features .

Googling on change unique id home assistant mqtt brings this up :

Seems like some people got past it, but no working example of the switch configuration is shown.

However, there is a link to the MQTT switch configuration .

That one mentions that unique_id is optional .

I added it to my two switches in configuration.yaml, and restarted the server.

Still no X10 switches under Configuration / Devices and services.
The MQTT integration is there, but has no devices listed underneath it .

Finally, I went to the overview screen, and clicked on one of the switches, and the gear icon (settings). And now, the error message about missing unique_id is gone. Hallelujah ! I am able to change the area, and assign it to office. It doesn’t show in that room under overview until I refresh the browser with F5, though.

So yes, it looks like I finally figured it out. The number of steps required to do so was insane, though.

From a user experience point of view, it could/should have been so much simpler :

a simple way to locate/install the X10 to MQTT integration, that works the first time around
settings for the serial port / cm17_in use presented at the time the X10 MQTT add-on is installed
if not already installed and configured locally, MQTT broker and integration automatically installed and auto-configured, maybe with autogenerated user and random password .
some sort of GUI to create new X10 devices and input the X10 code, and name. Unique ID either auto-generated from switch name, or another separate field in the GUI.
X10 devices should show somewhere under configuration/devices, like all others devices do

madbrain · March 31, 2022, 9:08pm

I am running into a problem with when editing existing device configuration in configuration.yaml .

I made some typos originally when I first set things up.

For instance, for the following device, I originally set it up as code c3, which is my terrace light, not backyard light. I fixed the definition in configuration.yaml to be code c4, which is the correct x10 code.

  - platform: mqtt
    name: "Backyard light"
    state_topic: "x10/stat/c4"
    command_topic: "x10/cmd/c4"
    payload_on: "ON"
    payload_off: "OFF"
    retain: false
    unique_id: backyard_light

Unfortunately, the new definition does not take. The backyard light switch in the UI continues to use code c3, and controls my terrace light, instead of my backyard light.

I have used the “check configuration” feature in HAOS, and the configuration is all OK.

I have tried to restart the X10 to MQTT gateway add-on.

I have tried to restart the Mosquitto MQTT add-on.

I have tried “restart server” in HAOS.

I have shutdown HAOS and rebooted the Pi with power off / on toggle.

None of those things have fixed the problem. The initial incorrect device configuration sticks.

I’m not sure where the problem lies, but if anyone knows of a solution, I would really appreciate it.

I’m in the process of switch house codes on several X10 devices to consolidate them all to 2 house codes instead of 5 house codes, so I’m going to run into this a lot.

madbrain · March 31, 2022, 10:47pm

I found that removing the Mosquitto broker add-on, and reinstalling it, solved the problem.

I’m wondering is there is a better solution as that seems a bit heavy-handed thing to do every time I need to do an X10 code swap. The issue affects the entity IDs and even the friendly name as well.

UniKyrn · April 2, 2022, 3:48am

I was going to suggest deleting both devices from mqtt and re-adding them. The integration leaves persistent data in mqtt so that the last known state of the device can be the current state after a reboot. If you look for a program called MQTT Explorer you can actually watch the traffic and each entity as it changes, and delete misconfigured ones directly.

freewilly3d · April 13, 2022, 3:20pm

I’m intested testing mochad if you would package it or point me to a good source on how to package it. I’d be willing to test it. I have a lot of x10 stuff which has been super reliable. I’m slowing swtiching to zwave but the stuff has got very expensive I’m assuming because of the supply chain issues.

UniKyrn · April 16, 2022, 8:39pm

@markm I’ve been looking through the code for the Integration, finally figured out how the docker container worked and I’m thinking of trying to add basic dimming support since heyu already supports it. I want to make very sure though I don’t step on any toes here so I’d like to know you’re OK with me playing around in the code. Never used GitHub, never setup a repo, not sure how I’d even publish anything that I got working,

tarzan_nojane · June 22, 2022, 3:05am

Go back to post 33 of this thread and you will find links to some X10 documentation which may be helpful in implementing dimming capability in the existing scripts.

I would be glad to test/evaluate your work if I can be of any assistance.

PAOK1926 · July 23, 2022, 4:24pm

its going to drive me mad…
this is the mqtt broker log:

[19:15:53] INFO: Starting mosquitto MQTT broker...
1658592953: Warning: Mosquitto should not be run as root/administrator.
time="2022-07-23T19:15:53+03:00" level=debug msg="got 3 users from passwords file"
time="2022-07-23T19:15:53+03:00" level=debug msg="got 0 lines from acl file"
time="2022-07-23T19:15:53+03:00" level=info msg="Backend registered: Files"
time="2022-07-23T19:15:53+03:00" level=debug msg="new hasher: pbkdf2"
time="2022-07-23T19:15:53+03:00" level=info msg="Backend registered: HTTP"
time="2022-07-23T19:15:53+03:00" level=info msg="registered acl checker: files"
time="2022-07-23T19:15:53+03:00" level=info msg="registered user checker: files"
time="2022-07-23T19:15:53+03:00" level=info msg="registered superuser checker: files"
time="2022-07-23T19:15:53+03:00" level=info msg="registered acl checker: http"
time="2022-07-23T19:15:53+03:00" level=info msg="registered user checker: http"
time="2022-07-23T19:15:53+03:00" level=info msg="registered superuser checker: http"
time="2022-07-23T19:15:53+03:00" level=info msg="redisCache activated"
time="2022-07-23T19:15:53+03:00" level=info msg="started go-cache"
[19:15:53] INFO: Successfully send discovery information to Home Assistant.
[19:15:53] INFO: Successfully send service information to the Supervisor.
time="2022-07-23T19:16:29+03:00" level=debug msg="checking auth cache for x10"
time="2022-07-23T19:16:29+03:00" level=debug msg="to auth record: [97 117 116 104 45 120 49 48 45 120 49 48 218 57 163 238 94 107 75 13 50 85 191 239 149 96 24 144 175 216 7 9]\n"
time="2022-07-23T19:16:29+03:00" level=debug msg="checking user x10 with backend Files"
time="2022-07-23T19:16:29+03:00" level=debug msg="user x10 authenticated with backend Files"
time="2022-07-23T19:16:29+03:00" level=debug msg="setting auth cache for x10"
time="2022-07-23T19:16:29+03:00" level=debug msg="to auth record: [97 117 116 104 45 120 49 48 45 120 49 48 218 57 163 238 94 107 75 13 50 85 191 239 149 96 24 144 175 216 7 9]\n"
time="2022-07-23T19:16:29+03:00" level=debug msg="checking acl cache for x10"
time="2022-07-23T19:16:29+03:00" level=debug msg="to auth record: [97 99 108 45 120 49 48 45 120 49 48 47 99 109 100 47 43 45 97 117 116 111 45 57 67 69 49 69 69 54 54 45 53 68 56 68 45 51 52 70 69 45 56 52 57 67 45 68 56 56 67 55 65 53 53 55 50 57 69 45 52 218 57 163 238 94 107 75 13 50 85 191 239 149 96 24 144 175 216 7 9]\n"
time="2022-07-23T19:16:29+03:00" level=debug msg="Superuser check with backend Files"
time="2022-07-23T19:16:29+03:00" level=debug msg="Superuser check with backend HTTP"
time="2022-07-23T19:16:29+03:00" level=debug msg="http request approved for x10"
time="2022-07-23T19:16:29+03:00" level=debug msg="superuser x10 acl authenticated with backend HTTP"
time="2022-07-23T19:16:29+03:00" level=debug msg="setting acl cache (granted = true) for x10"
time="2022-07-23T19:16:29+03:00" level=debug msg="to auth record: [97 99 108 45 120 49 48 45 120 49 48 47 99 109 100 47 43 45 97 117 116 111 45 57 67 69 49 69 69 54 54 45 53 68 56 68 45 51 52 70 69 45 56 52 57 67 45 68 56 56 67 55 65 53 53 55 50 57 69 45 52 218 57 163 238 94 107 75 13 50 85 191 239 149 96 24 144 175 216 7 9]\n"
time="2022-07-23T19:16:29+03:00" level=debug msg="Acl is true for user x10"

this is the X10 to MQTT gateway log:

[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[19:16:25] INFO: Configuring Heyu...
[19:16:25] INFO: CM11 is enabled
starting heyu_relay
Establishing MQTT to core-mosquitto port 1883...
(Using MQTT username x10)
Waiting for MQTT messages and monitoring for remote changes
Connected to MQTT broker, result code 0

and finally this is my entry in configuration.yaml

switch:
  - platform: mqtt
    name: "X10 G3"
    state_topic: "x10/stat/g3"
    command_topic: "x10/cmd/g3"
    payload_on: "ON"
    payload_off: "OFF"
    retain: false
    unique_id: X10_G3

i still cannot see this entity in Settings/Devices & Services/Entities

what am i doing wrong ?

Erytheis · August 28, 2022, 10:18pm

This may seem like a stupid question, but can you use the X10 switches to control other HA devises? Like a zigbee bulb through automation, or is this strictly to control X10 while having it in Ha?

yeeCdn · August 29, 2022, 1:49am

The X10 switches switch 120/240Vac. You can just plug a regular bulb into them. I do use the X10 rf remote HR12A to control other HA devices via automation, scripts. HR12A control up to 16 HA entities.

Basically, any X10 remote can send 315Mhz X10 signal to this addon, which sends mqtt to HA. Non X10 devices can send mqtt to HA to control X10 switches.

Erytheis · August 29, 2022, 4:07am

Awesome, that’s just what I was looking for. Thanks!

Erytheis · August 31, 2022, 2:54am

Have any luck? I seem to be having the same issue.