Nuki Card with Callback support (supports both Lock & Opener, it replaces the official integration)

alexdelprete · July 19, 2021, 10:50pm

Spokin,

thanks a lot for this, but I’m not convinced it’s the best way to solve the language problem, too many IFs to do a translation. The code will become bloated and probably slower.

I will think about it and let you know…I think we can come out with a more efficient and flexible solution, that abstracts the code of the automation from the strings of the labels/messages to be translated…we need a file, or something that users can themselves edit and the code references labels, whose contents can be translated.

Will let you know…thanks a lot for trying to improve this integration with new ideas, I greatly appreciate it.

spokin · July 19, 2021, 11:06pm

Hi Allesandro,

I agree with you. Unfortunately yaml is just a scripting language with it’s limitations. The only way to make the code more efficient is transform it into an integration (integration documentation) but then we need to know python, that you and I unfortunately don’t know. An integration can have language files (internationalization). So maybe in the future I can see if I manage to learn python, but that will definitely then be after the summer holiday season.

Maybe, after your effort, the people at Nuki, now know how to integrate it in HA and will do it…

alexdelprete · July 19, 2021, 11:10pm

True, we are limited in what we can do, but many times while developing this package/automation with plain yaml, I learned that it’s possible to overcome many problems that initially I thought required a custom integration. That’s why I told you to give me some time to think about it…I love solving problems due to specific constraints, it spikes my creativity.

If I don’t come up with an elegant solution, I will tell you. I’m already thinking about something: a compact JSON structure, to be saved somewhere, that can be referenced in the code…

kalinadi · July 21, 2021, 5:53am

Good idea
the automation only controls the callback registry?

unfortunately I’ve experienced 2 bugs in the meantime…

the lock became unavailable for 3 hours during the daytime so it’s not related to the battery saving
the automation registered 2 new callbacks even though an earlier one was register correctly. So when I manually checked the bridge I got 3 callbacks registered.

I think the the two might be related…
any ideas?

mamoel · July 21, 2021, 6:10am

I have no answers to your questions, but I experience the same issues.

alexdelprete · July 21, 2021, 6:14am

I don’t think I fully understood the question: there’s no registry. The callback is done by the bridge, that calls an HTTP url pointing to HA, basically it refreshes the status of the lock/doorsensor immediately.

Yes it happened to me too yesterday, and I found the problem in my case: HA is not resolving DNS names, it happens every 1 hour:

➜  ~ curl -X GET 'http://nuki-bridge.axel.dom:8080/callback/list?token=tyzyzyz'
{"callbacks": [{"id": 0, "url": "http://hass.axel.dom:8123/api/webhook/xTE4NzYzzzzzzzzzzsImlhdCI6MTYyMTcyODg0MSwiyyyyyyyxxxxxxRJwRTfJsoW-Rrrrrrrrrrrrr"}]}

➜  ~ date
Wed Jul 21 05:41:03 CEST 2021

➜  ~ curl -X GET 'http://nuki-bridge.axel.dom:8080/callback/list?token=tyzyzyz'
curl: (6) Could not resolve host: nuki-bridge.axel.dom
➜  ~ curl -X GET 'http://nuki-bridge.axel.dom:8080/callback/list?token=tyzyzyz'
curl: (6) Could not resolve host: nuki-bridge.axel.dom

➜  ~ ha dns restart
Processing... Done.
Command completed successfully.

➜  ~ curl -X GET 'http://nuki-bridge.axel.dom:8080/callback/list?token=tyzyzyz'
{"callbacks": [{"id": 0, "url": "http://hass.axel.dom:8123/api/webhook/xTE4NzYzzzzzzzzzzsImlhdCI6MTYyMTcyODg0MSwiyyyyyyyxxxxxxRJwRTfJsoW-Rrrrrrrrrrrrr"}]}

It never happened to me before, and it seems that many users are having this issue and the solution is not available yet. (in case someone’s interested, I’m discussing the issue in this old thread).

So back to the Nuki Card bug: the code regarding the reconfiguration of the bridge didn’t take into account several things, I’m modifying it so it can be more resilient to these issues.

In your case, it would be interesting to see what happened: for sure HA was not able to communicate with the bridge, that’s what triggers the bug. While I complete the new version, you can delete the 2 duplicated callback entries from the bridge manually (use curl).

Can you show me your nuki_bridge_url in the secrets?

One workaround, while I complete the updated version, if the problem is the DNS, is using the ip address:

so, from this:

nuki_bridge_url: "http://nuki-bridge.axel.dom:8080"

to this:

nuki_bridge_url: "http://10.1.10.126:8080"

Sorry and thanks for the feedback…

Alessandro

alexdelprete · July 21, 2021, 6:15am

Sorry for this, I’m working on the fix.

Can you do a little bit of analysis of the logs and tell me if there’s any indication of no communication with the bridge?

Also, can you show me your nuki_bridge_url in the secrets?

One workaround, while I complete the updated version, if the problem is the DNS, is using the ip address:

so, from this:

nuki_bridge_url: "http://nuki-bridge.axel.dom:8080"

to this:

nuki_bridge_url: "http://10.1.10.126:8080"

Thanks.

mamoel · July 21, 2021, 6:45am

My nuki_bridge_url was already set to an IP.

Here’s my log:

2021-07-20 13:37:41 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with Server disconnected without sending a response.
2021-07-20 13:37:41 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest_command] Client error http://192.168.178.46:8080/callback/add?url=http%3A//192.168.178.46%3A8080/api/webhook/*****************&token=*****************
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:37:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-20 17:13:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-20 17:13:53 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-20 17:13:53 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 17:13:53 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-20 17:15:41 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-20 17:15:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_info is taking over 10 seconds
2021-07-20 17:15:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with 
2021-07-20 17:15:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 17:15:52 WARNING (MainThread) [homeassistant.components.rest_command] Timeout call http://192.168.178.46:8080/callback/add?url=http%3A//192.168.178.46%3A8080/api/webhook/*****************&token=*****************
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_callback_list is taking over 10 seconds
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_info is taking over 10 seconds
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 05:59:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:52 ERROR (MainThread) [homeassistant.components.rest_command] Client error http://192.168.178.46:8080/callback/add?url=http%3A//192.168.178.46%3A8080/api/webhook/*****************&token=*****************
2021-07-21 05:59:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 06:01:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:09:52 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_info is taking over 10 seconds
2021-07-21 08:09:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with 
2021-07-21 08:09:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 08:11:52 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-21 08:11:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-21 08:11:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 08:11:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:13:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:15:52 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-21 08:15:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-21 08:15:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 08:15:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:17:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:27:52 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-21 08:27:52 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-21 08:27:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 08:27:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:29:42 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running

kalinadi · July 21, 2021, 6:53am

Hi @alexdelprete and thanks for your quick reply!
In my case the bridge address was an IP from the beginning, so for me its probably not a dns issue.
I think the problem resolves when I delete the extra callbacks from the bridge.
for now, I disabled the Nuki Card automation.
sending you some pictures, hope it will help
URL in secrets

errors:

alexdelprete · July 21, 2021, 8:00am

You shouldn’t have all those errors. It’s not DNS in your case.

Looking at the timings, I see something happened at 13.37 and 16.35 yesterday, then at 5.59 and 8.11 this morning.

2021-07-20 13:37:41 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with Server disconnected without sending a response.
2021-07-20 13:37:41 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-20 16:35:42 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with [Errno 111] Connect call failed ('192.168.178.46', 8080)
2021-07-20 16:35:42 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
url=http%3A//192.168.178.46%3A8080/api/webhook/*****************&token=*****************
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_callback_list is taking over 10 seconds
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_list is taking over 10 seconds
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.nuki_bridge_endpoint_info is taking over 10 seconds
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/callback/list?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/info?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:51 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.178.46:8080/list?&token=***************** failed with 
2021-07-21 05:59:51 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 05:59:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running
2021-07-21 08:11:52 WARNING (MainThread) [homeassistant.components.rest.sensor] Empty reply found when expecting JSON data
2021-07-21 08:11:52 WARNING (MainThread) [homeassistant.components.automation.nuki_card_callback] Nuki Card Callback: Already running

Can you send this command to the bridge and post the log please? Thanks.

curl -X GET 'http://nuki-ip:8080/log?token=xxxyyy'

alexdelprete · July 21, 2021, 8:02am

Please show me the HA logs with errors related to the bridge communication, and also the bridge log, sending this command to the bridge from a terminal. It will help me debugging the issue, thanks.

curl -X GET 'http://nuki-ip:8080/log?token=xxxyyy'

Sym0nd0 · July 21, 2021, 8:18am

Just to add, I’ve always used the internal IP of my bridge and it’s absolutely rock solid.

alexdelprete · July 21, 2021, 8:32am

Yes, I think, judging from the logs I saw, that there are communication issues for some users. But the code should prevent adding the callback to the bridge 3 times. So the bug is present, in my case the DNS triggered the issue, in other cases I think it’s the wifi.

Thanks a lot for the feedback. Did you check HA logs for errors in your case? If you can post them you’ll help me collect more info, I’d appreciate it.

kalinadi · July 21, 2021, 8:36am

hi,
it keeps only few hours back, and the automation was disabled since yesterday, so I’ll send it next time it will go offline (BTW, I noticed that the lock state didn’t update when the automation was off)

if the problem comes up because of the automatic callback add to the bridge, I’ll be happy to let it go and do it manually…
if its an option I’ll appreciate it a lot if you could post the code for it.
Thanks again for all your effort!

alexdelprete · July 21, 2021, 8:42am

The automation updates everything.

It’s not an option, I can’t maintain 2 versions of the code, I’ll fix it.

mamoel · July 21, 2021, 8:48am

The log only shows events from the last hour. So I can’t see, what really happened, when the errors occured.

alexdelprete · July 21, 2021, 9:02am

Thanks, if it happens check it please, it would help. Also HA logs.

spokin · July 21, 2021, 2:31pm

Hi Allessandro,

I think the problem with the multiple registered callbacks is that if for what ever reason the “Nuki Bridge Callback List” rest call fails, the binary sensor nuki_bridge_callback is switched to off and that will trigger the “Nuki Card Callback” automation which will trigger the script nuki_bridge_check_callback. And there if the binary sensor nuki_bridge_callback is off a new callback url will be added. This keeps going on until a “Nuki Bridge Callback List” rest call succeeds…

So you’ll possibly have an endless loop…

It may also flood the bridge because every 5 seconds a call to the bridge is done in that case which might get the bridge in a kind of unresponsive state.

spokin · July 21, 2021, 3:10pm

Hi Alessandro,

I might have an idea on how to achieve a better way for the translations. It might be possible to create a template file with tags that can be replaced by a bash script. The idea is then, to create an automation that is triggered by a change of the input_select and that automation will then:

Invoke the script to replace the tags in the template file and write the result to the nuki.yaml file in the packages directory.
Invoke the service automation.test_template_reload

It is just an idea, I’ll try to work on it tonight.

alexdelprete · July 21, 2021, 4:01pm

No endless loop, because I have a max loop number (5) exactly for that reason. The big issue is when the rest_command fails, for whatever reason, I need to reiterate the command, and the problem is that it doesn’t provide a status of the call.

Anyway, I rewrote many of those parts, now it’s much more robust, and when I add the callback, right after I use a wat_template with the state of the sensor and I added a timeout to the wait, as a safe exit. I also added the delete of the duplicate callbacks, but I need to test it well. I’m working on it since yesterday night.

We’re very limited to what we can do with pure yaml and templates. But it’s a nice challenge.

If you could help me test it once I think it’s ready I’d really appreciate it. So after your feedback I can release it.

Thanks a lot for being here.