Synology DSM7 new feature: webhook

raspbender · July 10, 2021, 2:07pm

Hi all,

I’ve updated my Synology NAS to the new DSM 7 and one of the new features are webhooks, which I’m aiming to use for some of the DSM’ push-notifications.

Did any of you manage to get it working?

The issue seems to be squarely on DSM side, with CURL from the NAS CLI I’m able to trigger the webhook on HA side and see it in the automation trace, but from the DSM interface (using the Send Test Message) no luck.

alias: Synology Event Webhook
description: ''
trigger:
  - platform: webhook
    webhook_id: synology
condition: []
action:
  - service: notify.your_platform_of_choice
    data:
      message: >
        'Your Synology says: ' {{ trigger.platform }}, {{ trigger.webhook_id }},
        {{ trigger.json }}, {{ trigger.data }}, {{ trigger.query }}
mode: single

Note: this post is not related to the DSM HA Integration.

Edit: automation code, for who’s interested

BeardedConti · July 11, 2021, 12:02pm

Just a question are you using HTTP for both Synology and Home Assistant or HTTPS for both?
EDIT: How did you setup notification in Synology for Webhooks to work in HA?
You are using PUSH instead of GET for this I presume?

What is the error you get in log file.

rccoleman · July 11, 2021, 4:14pm

I’m getting this error in the HA log with a simple test:

2021-07-11 09:13:17 DEBUG (MainThread) [homeassistant.components.webhook] Handling webhook POST payload for test_webhook
2021-07-11 09:13:17 ERROR (MainThread) [homeassistant.components.webhook] Error processing webhook test_webhook
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/webhook/__init__.py", line 96, in async_handle_webhook
    response = await webhook["handler"](hass, webhook_id, request)
  File "/usr/src/homeassistant/homeassistant/components/webhook/trigger.py", line 28, in _handle_webhook
    result["json"] = await request.json()
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_request.py", line 614, in json
    return loads(body)
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid control character at: line 1 column 80 (char 79)

I get that no matter what Content-Type I provide, or if I provide none at all. I’m not sure what it’s sending, but HA doesn’t like it. A quick Google makes me think that Synology is adding a newline character at the end, which isn’t valid JSON.

BeardedConti · July 11, 2021, 6:24pm

Did you manage to capture JSON from Synology - I still haven’t tried and the error is the same as you get. Will play with it in next few days to see what I can ger.

rccoleman · July 11, 2021, 8:05pm

This is what I get from a mock Postman server:

For this:

I don’t see any of the “Request Body” stuff in the Postman dump, DSM7 requires the “Message Body” to be “hello world” (?!), and I can’t figure out how to get a raw dump from Postman.

raspbender · July 12, 2021, 6:55am

Hi @BeardedConti ,

I’m using plain HTTP, and the request is a POST.

I just checked the logfile and it’s the same error as @rccoleman, seem the request is malformed.

Do you guys know of any way to check raw data on HA side, before parsing?

Another thought I have could be to point the Synology to a machine with Wireshark and inspect the inbound packet there.

raspbender · July 12, 2021, 7:47am

Same here, also setting the Content-Type to anything (quotes or no quotes) does not seem to pass it to the call, in postman I see always application/json as content-type in the Request Headers.

Edit: if I set in DSM an HTTP Header like foo : hello world and put that as Message Content, it does not even get to Postman (either Postman cannot parse, or DSM just fails to send)

BeardedConti · July 12, 2021, 7:59am

I’m at work ATM, will try to use Wireshark later to look at the package sent by Synology.

raspbender · July 12, 2021, 8:10am

Just found the logfile on the NAS, if it helps:

root@Synology:/var/log# ls -lart
root@Synology:/var/log# tail synoscgi.log
2021-07-12T09:48:54+02:00 Synology synoscgi_SYNO.Core.Notification.Push.Webhook.Provider_1_send_test[26448]: curl.cpp:116 Curl Execute Failed
2021-07-12T09:48:54+02:00 Synology synoscgi_SYNO.Core.Notification.Push.Webhook.Provider_1_send_test[26448]: webhook_send_message.cpp:135 Failed to exec CURL on Postman

Seems it’s a CURL wrapper

raspbender · July 12, 2021, 9:49am

Well well, found the culprit, someone must have left dummy code in the DSM production release.

Digging around the filesystem I found the directory /usr/syno/etc which contains the configuration file of the webhook UI form: synowebhook.conf.

This is how mine looks like for the Webhook named “Postman”:

{"Postman":{"needssl":true,"port":443,"prefix":"A new system event occurred on your %HOSTNAME% on %DATE% at %TIME%.","req_header":"Content-Type:text/plain\rtestheader:testheadervalue\r","req_method":"post","req_param":"{\"MessCont\":\"@@FULLTEXT@@\",\"testbody\":\"testbodyvalue\"}","sepchar":" ","template":"postmanmockserverurlhere","type":"custom","url":"postmanmockserverurlhere"}}

or, formatted:

{
  "Postman": {
    "needssl": true,
    "port": 443,
    "prefix": "A new system event occurred on your %HOSTNAME% on %DATE% at %TIME%.",
    "req_header": "Content-Type:text/plain\rtestheader:testheadervalue\r",
    "req_method": "post",
    "req_param": "{\"MessCont\":\"@@FULLTEXT@@\",\"testbody\":\"testbodyvalue\"}",
    "sepchar": " ",
    "template": "postmanmockserverurlhere",
    "type": "custom",
    "url": "postmanmockserverurlhere"
  }
}

First thing we notice is the @@FULLTEXT@@ which is in place of the hello world “Message Content” required by the UI (wtf…), and also it’s unclear to me why the req_header escaping is done differently than the req_param but I’m not a developer so there may be a reason there.

Anyway, lo and behold, removing one @ in the .conf and then sending a test message via UI passes the headers and body correctly, with the exception of content-type which is overwritten/received as application\json not matter what I try.
Works as well in HA and the trace shows the testbody and MessCont both.

The issue is of course that FULLTEXT is probably supposed to be a placeholder for both the prefix field and the notification message.

Since I’m out of my depth here, do any of you know if it could be an escaping issue with the @@s, i.e. the webapp writes poorly its config file, or that part may be completely missing altogether?

Im1Random · December 1, 2021, 12:18am

After a lot of trial and error, I finally found a fix!

You have to manually edit the /usr/syno/etc/synowebhook.conf and replace @@FULLTEXT@@ with @@PREFIX@@\\n@@TEXT@@ to format the newline correctly. Of course you can also add whatever you want around the placeholders.

Just keep in mind that you won’t be able to edit the webhook in the gui anymore cause it doesn’t let you save if the FULLTEXT attribute isn’t there.

scyto · February 21, 2022, 5:16am

how did you figure out the @@PREFIX@@ and @@TEXT@@ - i am interested to know if there are other variables that can be passed in (i am using apprise-api to send notifications to multiple servicesm and would like to pass in body, title, type etc caronc/apprise-api: A lightweight REST framework that wraps the Apprise Notification Library (github.com)

Im1Random · March 12, 2022, 4:19pm

Well I basically guessed this two. Maybe just try out if BODY, TITLE etc. exists and share your result if you find anything.

raspbender · May 8, 2022, 5:59pm

After a while I got finally around to test it, this workaround provided by @Im1Random solves it for me. Thanks!

Also, it makes one wonder how such a glaring UI bug has still not been fixed by Synology … my DSM version is DSM 7.0.1-42218 Update 3, so already a few updates from the baseline 7.0.

I’ve experimented with other @@s but It seems only PREFIX and TEXT are valid at this point, still it works so no complaints.

Another thing, it may be useful for others, found a similar solution for Discord notifications here: Balthazar - Blog – Sending a webhook from Synology DSM to Discord

Here my /usr/syno/etc/synowebhook.conf:

{
  "Home Assistant":{
    "needssl":false,
        "port":8123,
        "prefix":"A new system event occurred on your %HOSTNAME% on %DATE% at %TIME%.",
        "req_header":"Content-Type:application/json\r",
        "req_method":"post",
        "req_param":"{\"description\": \"@@TEXT@@\", \"title\": \"@@PREFIX@@\"}",
        "sepchar":" ",
        "template":"http://<HASSOS_IP>:8123/api/webhook/<WEBHOOK_NAME>",
        "type":"custom",
        "url":"http://<HASSOS_IP>:8123/api/webhook/<WEBHOOK_NAME>"
  }
}

Worth noting that I tried various combinations for template and url, with fqdn, without port number, etc. - and it seems only an IP address works (fqdn being resolvable from the synology host, but nevertheless not working), and the port number needs to be specified explicitly even though it’s already in the port parameter.

Here the HA automation to forward this webhook to a Telegram bot:

alias: Synology Event Webhook
description: ''
trigger:
  - platform: webhook
    webhook_id: <WEBHOOK_NAME>
condition: []
action:
  - service: notify.telegram
    data:
      message: '{{trigger.json.description}}'
      title: '{{trigger.json.title}}'
mode: single

Hope this helps !

poudenes · May 20, 2023, 9:23am

finally its working here as well. Thanks

Kyar · August 13, 2023, 10:37am

Hello, it don’t work with my DSM 7.2-64570 Update 2 :

Test messgae it’s OK but I don’t have notification with all notifications

BeardedConti · August 18, 2023, 8:54pm

How does your automation look?

Kyar · August 19, 2023, 7:52am

My automation :

alias: Synology WebHook
description: ""
trigger:
  - platform: webhook
    webhook_id: "xxxxxx"
    allowed_methods:
      - POST
      - PUT
    local_only: false
condition: []
action:
  - service: notify.telegram
    data:
      message: "{{trigger.json.description}}"
      title: "{{trigger.json.title}}"
mode: single

BeardedConti · August 19, 2023, 1:59pm

Looks ok. Does automation get triggered on Synology event? What does trace say for it.