Grouped Lights

So I finished this today, well when I say “finished” I prototyped it to a functional state.

I think I will make this my first project for HomeAssistant. At the moment…

The lights are controlled by SOnOff S20s with custom firmware. They have REST APIs with:

Short press on their buttons switches that individual light on/off. Long press on their buttons switches the group on/off.

It is currently using a hack and the group is controlled by publishing a demand into my heating system (Yep! I did say hack) and a separate lighting controller polls that demand and send the HTTP “ON” or “OFF” to the plugs. The heating controller ignores this demand as it’s not in the “heating” category of demands.

{ “groupLight1”: {“category”: “lighting”, “expiry”: “60000”, “timestamp”: 1568473201.3778918, “key”: “groupLight1”, “demandState”: “ON”}

The RF switch is attached to a a SOnOff RF Bridge running ESPurna with MQTT enabled. Mosquitto is the broker. The MQTT topic is received by a nasty little python script which just publishes the demand for ON or OFF into the existing mechanism.

The downside of hacking it into the heating controller is there is up to a 1 second delay in it responding to the demand.

So I think this is a fairly reasonable introduction project to try in Home Assistant. I’ll read some tutorials on connecting MQTT to a switch and the switch to a group of switches and see if I can get it to work that way.

All of this can be done without hacking your heating system. Group the lights via your setup yaml. Turn the group off/on in an automation.

In short, don’t know why you are making it ao complicated.

That’s a little amusing. It’s like someone explaining how they built their own washing machine being told “Why don’t you just stick it in the Hotpoint and select program 5?” without realising just how complicated their Hotpoint or program 5 is.

But, yes, the plan is to see how much easier HA makes this.

ESPurna was used as an experiment, I would prefer I could do without it. It already makes inconvenient stipulations and gets in the way slightly. ESPurna models “swtiches”, switches have an ON/OFF status. However the actual “switch” is just a momentary push button and has no state and it has no way to know what state the actual relays are in. Toggle the relay by other means and ESPurna is out of sync.

It’s even worse if you consider there are three relays. What if they are in different states but ESPurna only has a single ON/OFF state modelling them?

These are 1st world problems, I’ll have to press the switch twice instead of once, but they urk me a little. I could try and keep ESPurna in sync somehow, but it still puts the modelling of relay state in the wrong place. The only thing that knows the true status of the relay is the relay, so that is where the state should be modelled. That’s why I gave them a “status” end point.

I have a fairly strict design criteria. Automation should be completely transparent to manual intervention and automated devices should continue to function as a user would expect them to, as intended, regardless of automation. This results in there being more than one way to switch something on or off, not all of those under automated control, and if you want to know if it’s ON or OFF the only way is to ask the device itself.

I could just accept whatever ESPurna (and possibly HA) (Hotpoint program 5) provides me and be done with it, but … that’s just not me. I like choice. If I know something is possible and some automation software says, “I’m sorry, I can’t let you do that Dave” then that automation software is not fit for my purposes.

By the way, I wrote the heating system, by “hack” I just mean I borrowed some of it’s functionality to “spot weld” things together quickly. I’m not finished yet. As to complicated, including soldering the pin headers on the RFBridge, flashing software, installing an MQTT broker for the first time and writing the python MQTT receiver took me all of about 2 hours.

So working with HA this morning I think the RESTful Switch will allow me to manage state in the relay itself and I can otherwise (I hope) ignore what ESPurna thinks.

In this instance HA is right and I am the hack as it basically stipulates the switch is a proper REST resource, where as I only made a HTTP GET interface which is kinda cheating.

RESTful switch wants to send a PUT or POST to toggle the switch, which is fine, but my switch uses a different URL for ON, OFF and status, not the same endpoint with different HTTP methods.

I’m wondering if I feel like fixing that to be properly RESTful or if I try something different.

Can I do a combination of RESTful sensor to get the status and RESTful command to send on/off based on that status returned by a RESTful sensor?

I don’t have much experience with espurna, but if you put tasmota on these switches you would be able to:

  1. Autodetect them in HA

  2. detect long or short presses in HA

  3. have no delays

  4. group your lights so long press will trigger switching the group

  5. ignore the restrictions of the REST switch that you have identified (indeed you won’t need a REST solution at all.

1 Like

Hmm. I already have this available, except discovery and canned integrations. I wonder what it is about Tasmota that allows HA to “discover” it.

(So reading up on it, it seems to use an MQTT topic that I presume Tasmota publishes to and HA subscribes and then knows about the devices). I still have questions though. How often does it publish it’s presence? How long do discovered devices persist? What happens when a discovered device moves, changes it’s name, goes offline etc.

ESPurna on the RFBridge provides the integration with the RF switch. I’m basically using it to convert the RF event into an MQTT message that the switch was pressed.

I originally excluded Tasmota because I didn’t really need any of it’s functionality. HA integration “might” provide enough leverage for me to reconsider, but I need to do a bit more reading.

There is Tasmota for the RF Bridge as well which I can consider instead of ESPurna.

Yes tasmota uses MQTT discovery. It is documented. Have you read the HA docs?

I’m reading yes, but not all of it, yet obviously.

So far I have got it to “work” with HA. Using auto discovery and ESPurna all 8 channels/switches show up. Toggling the switch I have hooked up with MQTT does generate MQTT messages and that triggers the switch.

The problem is the status message is not received by HA so the switch on the HA UI defaults back to off again. I looked into fixes for this and discovered the current config migration is a bit in the way. The autodiscovered stuff in in the config entries in .storage and most of the solutions involve manually configuring through configuration.yaml, I’m not sure how to convert one to the other or if I should.

The state message looks something like this:

Client mosqsub|15943-everest received PUBLISH (d0, q0, r0, m0, 'RFBridge/data', ... (147 bytes)) {"rfin":"29E001720442F8A074",**"relay/0":1**,"time":"2019-09-15 19:29:44","mac":"DC:4F:22:DB:BB:95","host":"ESPURNA-DBBB95","ip":"","id":856}

But I don’t think that is where HA is looking right now.

If you’re interested in even MORE POWER, perhaps take a look at ESPHome as an alternative?

I strongly recommend using the API, so that your devices will be “integrations”, and you can remove the delay imposed by communication through the mqtt server, however you can always go the “old” way by using mqtt and having the devices announced for discovery, instead.

You have the additional advantage of being able to create all sorts of customisations that actually run on the device itself, should you wish.

I was thinking of referring to esphome, but @paulcam is confused enough already :slight_smile:

1 Like

At the moment I hold the references to the hardware, the digitalWrite() etc. What makes me slightly resistive to is that you more-or-less relinquish some or all of that low level control into higher level APIs. While that can have advantages it does also have disadvantages. While sometimes APIs and abstractions make things easier, they can also be a pain in the neck to get around when they don’t work or don’t want to do the thing you want to do. “I’m sorry, I can’t let you do that Dave”, are all too common. When the API really means, “I don’t know how to do that.”

I know the home automation space is still in flux, but just because a convention, api or system is open source does not mean you aren’t “boxing yourself in” to a system if you adopt it. To that end I don’t really want to create “Home Assistant” (or any other) specific end points/devices. I would rather use a standard protocol that is supported or can easily be supported by any and all such systems.

As an example. MQTT discovery. I have seen this banded around as a “standard”, but there is no mention of it in the MQTT protocol documentation. Now while that’s not surprising as MQTT does not involve itself in how you use the application protocol, the only references to MQTT discovery are within Home Assistant.

There is a good potential that the MQTT Discovery used by HA will propagate out to other systems and more devices, but it could also just get surpassed by something else. I suppose that’s the space we are in though, different systems bumping elbows for dominance. I would prefer that, where possible, standard protocols that exist outside of anyone system are used to communicate freely between them. They did this once before, it was called the Internet, apparently it was quite successful.

One related thing I want to avoid is having devices which are “Owned” by one hub or other. “Owned” meaning that you must communicate with them via that hub or risk corruption of state etc. I want my devices to be standalone and smart enough to be tinkered with by any API/Device/mechanism. If HA turns it on, can OH turn it off and they still play nice for example?

On discovery. It works absolutely fine as “Discovery”, but unfortunately that is, just like most other machine discovery mechanism ever made, even DHCP, full of draw backs and annoyances. Double discovery, missed discovery, cryptic names, discovered devices not responding, incorrect auto-generated config, the config buried off in machine generated, overly verbose, uninitiative places like the .storeage hierarchy.

And when you come to try and fix it… hash keys everywhere.

Hash keys are brilliant for machines to be able to generate and have a very low probability of collision, but they are NOT human consumable. Peppering config files with hash keys is irritating even for experienced developers. Especially when they create associations between things using them. I get why they use them, but the cost is too great IMHO. My attempts to clean out of the rubbish generated by discovery in the .storeage hierarchy was painful and tedious.

I’d go as far as to say the .storeage hierarchy is no better than the Windows Registry and I have yet to meet one person who like dealing with that.

I think I will take the risk that the configuration.yaml style config will be around for a while and stick to manually configuring things.

I know I may sound negative. It’s a bad habit of mine to immediately look for problems in things. However if you don’t look for problems, you won’t look for solutions and what is the point of being an engineer if you don’t have problems to solve. Every problem has a solution, but never forget that ever solution has problems.

Do you mean confused or resistant and awkward? :smiley:

Just not getting home assistant is my view.

I think I’m coming at it from a different perspective. I would gather a lot of home automation people just want their lights to work, but have no idea how to make that happen. So when they can follow a YouTube video or read a blog post that gives them step by step instructions to flash some black magic wizardry onto the device and install HA and things work that’s fine with them.

So they have no real perspective on how these things work or what is possible. They get used to HA, the learn what it can do and in their minds that is all that can be done. They are fine with that, I’m sure.

I am coming to it from soldering devices together, toiling over datasheets and hardware apis, serializing JSON blobs into sockets etc. So I’m, possibly understandably, suspicious of “wizardy” firmware and canned solutions/protocols.

There’s no intention to get rid of that.

Personally, I have not seen any disadvantages of MQTT discovery. Yes it stores stuff in plain text (JSON) in .storage folder but it needs to store stuff somewhere. Even storing in configuration.yaml is still storing a config…

So write your own firmware and flash that and integrate the device using MQTT. Job done.

Yes. But what it stores is wrong, non-functional in my case. I can’t edit .storeage while HA is running and when I fix things up and switch it back on again it generates more junk so I end up with duplicated and still not working stuff. In the specific case I don’t think HA is looking for the right status message. It’s expecting a /state message, but my device sends a /data message containing a key for which relay and state it’s referring to. I can probably do this with templates to extract the value, but I’m still just picking this stuff up and with it being auto-magically generated I feel like I am having to fight with discovery to achieve things.

So don’t use discovery.
It’s the same if I use Tasmota… if I use Discovery, I don’t do manual config. If I do manual config I don’t use discovery. Use whatever works for you. There’s no hard and fast way you have to do it.

Currently this is my plan. MQTT is at least an external standard. Though it will probably require more work, I like that kind of work. I don’t mind something being broken because I screwed up, I can fix what I screwed up. I do mind when someone else who wrote an API screwed up but it’s either inaccessible or requires lots of hours trying to work out their code and how to fix it for them. (Not that I’m excluding providing pull requests to fix bugs I can find and fix)

1 Like

Or learn to use discovery properly. If your firmware is causing junk discovery, fix your firmware.