Detect when Zigbee bulb goes unavailable

Thanks for your answer, but this seems not be what I look for. I know the bulbs are powered off, as I switched them them off myself. I need HA to know faster (and more reliable) to also know they are off (switching on problem is solved).

I guess / assume uptime kuma will not know this before z2m or HA would also know this. So, I don’t understand how uptime kuma would speed up the process… Please enlighten me, if I wrong here.

Sorry I had to read it again because I obviously didn’t understood your problem. What are you taking about is that you turn off light blub but it is showed in ha with a state on.
Light bulbs in theory are routers in zigbee network but in my experience they are the worst one you could have.
This is sometimes happening because probably, for some reason, they don’t upodate state or are doing that late. This might be also problem with bulb manufacturer like tuya.
The only way to be sure where the problem is would be, in my opinion, to buy one smart bulb flashed with esp home and try that.
If this is working then I don’t think that you could do much about it except to add more routers to your zigbee network, like switches, plugs or outlets. As you said that you live in renatl you probably can’t modify it. I had that problem but it went away. But on the other hand a have a lot of zigbee devices and routers.

Taking away power from a mains powered Zigbee device that also acts as a router is a very bad idea.

Every time you power off the bulb you turn off a router and any device that was connected to your bulb will need to find a new router and the whole network goes into a small panic. You never get a stable zigbee network when you do this

Personally I have put cover plates on all my wall switches (that can be removed) and I have put a battery powered switch next them which is used instead. This also give the advantage that you can dim up and down or long press to control additional light modes from the same 4 button control. Or just use a two button IKEA Zigbee switch. They are dirt cheap - light weight - and can even be fastened by a double sticky tape. Remember to use one of the types that can removed with a little heat from a hair dryer without leaving a trace.

If you have to use the wall switch you may have other options.

If you have both switched live and permanent live in the light fitting in the roof our other lightening outlet, then you can move the lamp to permanent mains. And then hidden/covered at the outlet, you can put a small Zigbee or WIFI switch of the type that has a control input.

For example this. Vesternet VES-ZB-DIM-004 control via MQTT | Zigbee2MQTT
There are many similar.

You can either use it as intended and use dumb light bulbs.

But if you want a smart bulb that can change color or whiteness then you can simply connect this device to live and neutral. And then connect the switched live to the input

You now get a device on your network that turns on and off with the physical switch in the wall which is always powered, always on the network, always routing, and instantly reporting on and off state. And you then use that state for automations in home assistant. In Danish wall switches it is easy to put a spring in the switch so it becomes a toggle. And it can be removed without leaving a trace.

Otherwise you may get the wall switch and software power on/off out of sync so your wall switch is not always up or down when the lamp is on. There are many possibilities. But the important is to avoid mesh network devices like Zigbee and Zwave having routers that get their power cut regularly. It really gives a bad and unstable overall experience that ripples through to other devices on the network. Especially battery powered devices will disconnect or switches not reacting until button pressed multiple times.

For a few routers it might, but if you have enough of them then it isn’t. Because zigbee network is a mesh network and all clients aka end devices will pair them self to a new router. Some will do that silently some will fall off the network and reconnect after some time. Very rarely you will need to manually pair them back.

I have enough (?) / many zigbee devices that are never switched off. Can I somehow manually indicate to the network which these are, so they are used as routers?

And what leaves me puzzled is that operating this network with hue delivered a much more stable experience… Why can’t we achieve something similar using free technology?

Correct and incorrect

Yes, they will pair again - if they are not Aqara window/door sensors and other sensors - after some time

But that is not my point. My point is exactly what you say. They will silently fail and then reconnect after some time.

In real life this means

  • You randomly click on a remote and nothing happens. You click again. Maybe even a 3rd time. And then it works. If you have 20 remotes like I have - that means that someone in the family will experience this daily.
  • You randomly turn on a room and one of the lights remain off. Maybe it works if you toggle light on and off a few times.

Yes it works. But it is not a nice experience and you will have negative reactions from your partner if this is the quality level your smart home has. If you live alone and you are happy with it - so be it. I think most of us want a more stable network. We can live with a zigbee network to need a day to get stable after a completely new installation or major upgrade. But in day to day life I get annoyed when switches do not work the first time, when lights do not turn on immediately when I enter a room, when some lights do not turn on in a room. And that is the experience you get if you keep on cutting power to your zigbee routers.

1 Like

I dont think so. They are recognized as routers or end devices during paring process. In my case bulb started reporting state normally when i added more routers to the network.

When I moved from Hue Bridge to Deconz I experienced an improvement after the network had become stable.

And zigbee2mqtt that I moved completely to not so long ago is also very stable.

But it is not stable the first day or two. The network has to heal and fail and find the most stable routes.

When people move from one zigbee coordinating hub to another they always experience a drop in reliability. Be patient. And stop cutting power from your routing devices.

Well im monitoring my devices avalability with uptime kuma so i know if any of them fall of the network in the real time or became unavailable. I have one smart bulb in bedroom lamp that my brother then to turn off. Nothing special happend. One or two switches became unavailable and sortly after reconnects and sometimes even that dont happen.

Thanks a lot for the answers. Though, I guess it is not possible for me to not turn off bulbs.

Your workarounds are interesting, but seem to not fit our situation here. Of course I can cover all switches and stick smart switches on top of the cover (I guess you are referring to IKEA Rodret?), but this tends to go beyond what I want to do as physical changes, seeing this is a rented flat.

The hue solution was working great in this scenario and I don’t really understand, why we cannot achieve a similar experience with free software. Also noticing a device is turned on works great even now with the constantly broken network…

Also I have to reboot HA from time to time, I cannot wait each time for days for the network to become stable again.

So, the main questions for me stay:
It there any way to make detection of switched off bulbs faster then the 1 minute I can set in Z2M for timeout checks?
And can I make the network more reliable by declaring reliable / unreliable routers to the network? Or by setting up the routing map by hand?

But, from the current answers, it looks like back to Hue for me :frowning: Lots of work, seeing that I have 60+ smart bulbs around the flat…

Long after posting my initial question I decided to follow the advice of not turning off my bulbs. It has worked quite well for me. I started really simple, by just fixing the physical switch in the “on” position with a piece of tape. Later I gradually installed connected devices like the one @KennethLavrsen linked behind my switches. When I moved houses, I had to open up all the switches and remove the smart devices again. As long as you don’t move houses 3 times a year, that is totally doable though, and it doesn’t leave a trace.

Summary of the thread for future readers:

  • Mains-connected Zigbee devices should not be turned off. They act as routers in the Zigbee mesh network. Turning them off will cause the network to be reconfigured, which can lead to not responding switches and sensors. You don’t want that.
  • Even if you still see a use case for detecting when a Zigbee device goes offline, do realize that this can only be based on a timeout mechanism. Like the coordinator deciding to mark the device as offline if it hasn’t sent an update for X amount of time. This is generally unreliable and always lags behind.

So the advice is to make sure your bulbs are always connected and online and use a wireless switch to turn then on and off.

Another option is to install dumb bulbs and smart in-wall dimmers / switches that you connect to the original switch. That way you can continue to use the old light switch and still get control of the lighting through HA.

Example
image

2 Likes

Thanks a lot for all your feedback.

What I still don’t get is: Why does Hue work so well with bulbs getting removed from the network and we cannot handle this? It is a real pita, because I try to get all non free software and devices out of the house and fully rely on self controlled / FOSS stuff. This was why I moved away from Hue.

So, if I get it right and to stay constructive, if I want to keep my home Hue free, I basically need to

  1. disable all wall switches to provide steady power, as I have smart bulbs everywhere. For this I guess I have to remove the interior of the wall switch and directly connect the power so lamps are always on, then cover the former switches. Not being an electrician, this is a bit scary I have to admit. Or are there any easier solutions? Any suggestions or web resources how to do this best? (And no, ducktape to fix the switch in a certain position will likely not get family approval :slight_smile:

  2. Get more smart switches that integrate with HA (e.g. the IKEA ones) and stick them on top of the cover, which of course is easy - I already have some integrated.

I don’t know. I’m no Zigbee expert. But I can imagine that Hue uses some proprietary / undocumented techniques. ZHA and Zigbe2mqtt have to rely on public documentation about the standard. Plus, the open implementations try to support as many different hardware types as possible, while Hue might just drop support for bulbs that don’t support their feature that allows them to detect bulbs going offline sooner.

Regarding your questions:

  • The official advice is always to hire a skilled electrician to do any modifications on your electrical installation.
  • That said, there are many, many videos on YouTube on how to do this on your own. And if you understand what you’re doing, it’s actually not that hard.
  • If you decide to go the DIY route, please make sure to ALWAYS switch off the breaker before opening up the wall box. And use a tester to verify that there’s no voltage on any of the wires.
  • In my opinion, the most elegant solution is to keep the switches, and install one of those smart devices behind them. You will still be able to use the existing switches to turn the lights on and off, but this will done smartly. You can use a zigbee-device like the one @fleskefjes linked to before. Personally, I have good experiences with Shelly devices. Those use WiFi instead. But with HA you can combine different technologies easily!

I guess I don’t understand how the zigbee-devices @fleskefjes suggests work. They are installed on top (or better below) the current switch? I did not find any videos how this works, perhaps don’t know what key words to search for. How you describe it, it sounds interesting. Do you have any resources / videos I can get more infos?

Edit: I now found some video tutorials - these switches seem to only be usable with dumb bulbs, right? So no solution for me having smart bulbs everywhere, because they would be turned off and the same problem occurs - bulb-routers going offline.

Yes, as I stated you use them with dumb bulbs. I didn’t have you specifically in mind when I mentioned it, I replied to the “summary for future readers”.

Check your light fitting in the ceiling or wall. You have a multimeter? Get one if you don’t. Even a cheap one is good enough for this.

How many wires?

Depending on house age you may or may not have protective earth which should be yellow/green or green. Whatever you do, never screw up protective earth and use it as neutral!!!

Then there should be a neutral and a live wire that turns on when the wall switch is on. But often you also have a permanent live wire. Reason is that a common way installations are made is that PE, live and neutral is routed to all wall plugs and all light fittings. And then two wires from the light fitting are routed to the wall switch. That is why so many wall switches have no neutral unless the wallswitch is combined by a power outlet.

If your installation is like this you can move your lamp from the switched live to the permanent live just by moving the live wire. Then the bulb is permanently on.

You still have your wall switch that switches power on and off in the now vacant terminal.

The trick is now to plug a small zigbee or wifi smart switch to the outlet so that

Live and neutral are connected parallel with the lamp so this is also permanently powered. And then connect the switch terminals to neutral and the swiched live wire

The switch does nothing but reporting the state of the wall switch but this is what you need to make your automation.

If you can change the wall switch to momentary by adding a spring then do that. If this is not a standard accessory you can get do not try to add something home made and risk accidents. Then you will have to live with the wall switch being sometimes up and sometimes down when the light is on as it changes its sync each time you toggle the light from another source.

The small smart switch needs to be hidden away where you can fit it. You can often buy a plastic cover for light fittings used to hide a hook the lamp hangs in. I assume the fitting is so high that no kids or animals can fiddle with it.

Except when you measure for voltage always work with circuit breaker off. Not just the wall switch. The circuit breaker in your main installation box (consumer unit in British english). Always measure voltage present first. Then cut the breaker. Then measure again that power is off

The big nonos are

Protective earth must never be used as a neutral. It connects to other PE wires or to metal parts. Only that

Never borrow a neutral wire from another circuit. That also defeats the circuit breaker protection and trips RCDs/GFCIs

Do not be afraid of mains power. Have respect for it. Take pictures and ask someone if you have doubts. Or call electrician.

1 Like

Thanks a lot for the detailled description. I guess I am getting the picture now. Sorry for being so slow…

So, quick update on my attempts - Got myself some SONOFF ZBMINIR2 - but unfortunately there is not enough space in the box below the wall switch to integrate it :frowning:

Would have been a really smart solution though.

Is anyone aware of wall switches that integrate this technology directly into the switch, so I can replace the current wall switch with a smart one?

The only other alternative I see, is connecting the bulbs to always power on, remove the wall switch and get some blind covers instead and pin cheap IKEA switches on top of the blind covers. Not really a good looking solution though.

Any other options?