Installation of DSM Integration fails

armin-gh · October 14, 2021, 5:00pm

Hi
you might be able to switch on the access log for nginx, but I think it won’t tell us much more as the Synology seems to refuse the connection on TCP-Level, the nginx-Server never processes the request.
Any luck to run the “wget” command I wrote about earlier? wget might throw a better error message than the Python-Exception in the integration
Might also be worth to run “wget” from any other computer in your network, most Linux distributions have it installed by default, you can also run from powershell on any windows computer.

wget  "http://192.168.x.x:5000/webapi/query.cgi?api=SYNO.API.Info&version=1&method=query" -O syno.out

produces a file here in JSON format ( have a DS416play) and this is the request failing in the integration.

Armin

labertouche · October 15, 2021, 9:39am

I managed to execute the command. The output is a 69122 character long file.

What am I looking for?

Alex

armin-gh · October 15, 2021, 11:27am

Hi
Means that from where you started the command it can reach the Synology, using the IP/Hostname, port and Protocol as used in the wget command. Additionally ignoring a possible mismatch on the certificate
I’m at work now, will send later how my file retrieved from wget looks like, but the content does not really matter, retrieving ~70k means that the Synology responds
Using the same parameters in the integration it should work too… that much about the theory…

Depending on your type of HA installation the docker environment could be a problem too, that the integration can’t reach to the Synology from inside the docker environment running the HA server process

Armin

labertouche · October 15, 2021, 12:24pm

Hi Armin,

I used the wget command from my Mac in the same network and the syno.out gives me a lot of lines.
You would think the connection to the Synology works just fine right?
File is to large to copy/paste. I agree with your theory. But as mentioned in the initial post: I tried all possible combinations you van possibly think of.

Alex

armin-gh · October 15, 2021, 3:22pm

Hi
looks like this, a file in JSON-Format where Synology provides information about the functions available via http-Requests

Size likely depends on the packages installed in Synology, mine is around 90k

This is the documentation:

Synology accepts the connection from the Mac, leaves two options…

Synology blocks the requests coming from HA
HA does not get to the Synology through the network

We can try the following to make HA to send the same http-Request
Add the following to configuration.yaml:

notify:
  - name: SynologyInfo
    platform: rest
    resource: https://192.168.0.99:5001/webapi/query.cgi?api=SYNO.API.Info&version=1&method=query
    verify_ssl: false

This adds the same request to HA as a notification, might require to restart HA
Then in HA go to “Developer Tools”/Services

in the “Services” field you should see this notification service

Select it and enter some random text in the message (it insists on having something there…)

the press the “Call service”-Button
if it goes through and retrieves a response from Synology, the button just turns green and then blue again, I have not yet managed to show the response.
when I use an incorrect IP, the button turns red and an Error is logged

My hope is that this has more details than the error message from the Synology integration

we might also see just the same:

but at least we would know if this is a problem with the Synology integration or if there is a problem in reaching the Synology-Server from HA in general.

hope I did not add more confusion with my lengthy posts…
Armin

labertouche · October 18, 2021, 7:32am

Hi Armin,

Not lengthy at all but very helpful!
When calling the service, the button does turn green for a few seconds.
The logs show NO Synology errors.
So you would think the connection is successful right?

When I use the wrong IP address I get two error messages in the logfile:

Logger: homeassistant.helpers.script.websocket_api_script
Source: components/rest/notify.py:196
First occurred: 9:45:42 AM (1 occurrences)
Last logged: 9:45:42 AM

websocket_api script: Error executing script. Unexpected error for call_service at pos 1: HTTPSConnectionPool(host='192.168.1.215', port=5001): Max retries exceeded with url: /webapi/query.cgi?api=SYNO.API.Info&version=1&method=query&message=hello (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
OSError: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.1.215', port=5001): Max retries exceeded with url: /webapi/query.cgi?api=SYNO.API.Info&version=1&method=query&message=hello (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 371, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 571, in _async_call_service_step
    await service_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1491, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1526, in _execute_service
    await handler.job.target(service_call)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 174, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 149, in async_send_message
    await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/rest/notify.py", line 196, in send_message
    response = requests.get(
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='192.168.1.215', port=5001): Max retries exceeded with url: /webapi/query.cgi?api=SYNO.API.Info&version=1&method=query&message=hello (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable'))

and

Logger: homeassistant.components.websocket_api.http.connection
Source: components/rest/notify.py:196
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 9:45:42 AM (1 occurrences)
Last logged: 9:45:42 AM

[547166043728] Error handling message: Unknown error
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
OSError: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.1.215', port=5001): Max retries exceeded with url: /webapi/query.cgi?api=SYNO.API.Info&version=1&method=query&message=hello (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
    await func(hass, connection, msg)
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 525, in handle_execute_script
    await script_obj.async_run(msg.get("variables"), context=context)
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1219, in async_run
    await asyncio.shield(run.async_run())
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 353, in async_run
    await self._async_step(log_exceptions=False)
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 371, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 571, in _async_call_service_step
    await service_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1491, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1526, in _execute_service
    await handler.job.target(service_call)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 174, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 149, in async_send_message
    await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/rest/notify.py", line 196, in send_message
    response = requests.get(
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='192.168.1.215', port=5001): Max retries exceeded with url: /webapi/query.cgi?api=SYNO.API.Info&version=1&method=query&message=hello (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f75d09910>: Failed to establish a new connection: [Errno 113] Host is unreachable'))

armin-gh · October 18, 2021, 4:32pm

Hi
ok, that means that HA does receive a response when sending the HTTP-Request to the Synology
And that the problem ist somewhere with the Synology integration.

Wondering if some garbage data was left in the internal configuration files of HA when you discarded the integration during initial install or when you tried with the various combinations of parameters.

Can you locate a file “core.config_entries” in a folder “.storage” on HA? The .storage folder is where HA keeps the configurations for devices, entities, Lovelace, …
there sould be exactly one entry with domain “synology_dsm” and the connection data should be what you entered in the GUI ( I masked some of the data below)

            {
                "entry_id": "a66dc3f084e0b093f061399c4236ad89",
                "version": 1,
                "domain": "synology_dsm",
                "title": "DS416",
                "data": {
                    "host": "192.168.0.x",
                    "port": "5001",
                    "ssl": true,
                    "verify_ssl": false,
                    "username": "userid",
                    "password": "secret",
                    "mac": [
                        "00-11-32-aa-bb-cc",
                        "00-11-32-aa-bb-dd"
                    ]
                },
                "options": {},
                "pref_disable_new_entities": false,
                "pref_disable_polling": false,
                "source": "ssdp",
                "unique_id": "uuuuuuuuuuuuu",
                "disabled_by": null
            },

Armin

labertouche · October 19, 2021, 9:38am

Hey!

I checked the “core.config_entries” and see a DSM entry here:

 {
                "entry_id": "cd3dac2599fdad2cb8494284d214efa1",
                "version": 1,
                "domain": "synology_dsm",
                "title": "BLACKBOX (192.x.x.x)",
                "data": {},
                "options": {},
                "pref_disable_new_entities": false,
                "pref_disable_polling": false,
                "source": "ignore",
                "unique_id": "00113271322D",
                "disabled_by": null
            },

It does not have the “data” entries shown in your example:

"host": "192.168.0.x",
                    "port": "5001",
                    "ssl": true,
                    "verify_ssl": false,
                    "username": "userid",
                    "password": "secret",
                    "mac": [
                        "00-11-32-aa-bb-cc",
                        "00-11-32-aa-bb-dd"
                    ]

I removed the complete Synology and rebooted HA.
Then: Tada! New device discovered.

I enter my credentials and went to the next screen: 2FA
Haven’t setup 2FA, but I quickly logged in to DSM using a webbrowser and clicked “I don’t have my phone with me”. Then I used that code to get in.

IT WORKS!

Many many thanks Armin! Very much appreciated!

Now to find out why I get to enter a 2FA code even while that’s disabled in DSM.

armin-gh · October 19, 2021, 3:57pm

Hi,
very good, perhaps that helps the other user reporting a similar issue.
That’s what I wanted to suggest, to remove the whole section, of course after making a copy of the file. I guess the incomplete entry stopped it from discovering the device again.

I was never asked for 2FA and also have it not yet enabled on my Synology.
I believe you wrote earlier you have Quick Connect enabled and also a Synology Account ( for DDNS?), perhaps that triggers something. Did it put the internal name/IP or the external name into this config file?

Armin

labertouche · October 20, 2021, 8:54am

Thanks again for your help Armin, much appreciated!

I never activated 2FA, and don’t use quick connect. I only used the Synology DDNS service, but for internal traffic the internal ip address is used of course. Based on that HA found the integration.

Hope other people with similar problems will find this post!

Kudos Armin!

Alex.

Johnnyyr · November 11, 2021, 7:58pm

I checkt the core.config_entries nothing from Syno.
And i can´t connect. It works fine 6 month and now no connection .
Logs:
Logger: homeassistant.components.synology_dsm.config_flow
Source: components/synology_dsm/config_flow.py:190
Integration: synology_dsm (documentation, issues)
First occurred: 20:25:17 (12 occurrences)
Last logged: 20:53:03

{‘api’: None, ‘code’: -1, ‘reason’: ‘Unknown’, ‘details’: “ReadTimeout = HTTPConnectionPool(host=‘192.168.2.240’, port=5000): Read timed out. (read timeout=30)”}

Any ideas?

smellisssss · March 12, 2022, 12:28am

This is what solved it for me, took me a while to find this block list

fizz · April 9, 2022, 4:50pm

I was having a lot of trouble getting this integration to work and stumbled across this threat. For me, the issue was a firewall that I had between my Home Assistant and my Synology box. They were on separate VLANs with a firewall between them, and I forgot to allow 5001 from Home Assistant to the Synology. Just thought I’d mention it here in case someone else may have overlooked this too…

hombrevg · July 9, 2022, 7:03pm

OK posting here as well. I will look at the auto block list HOWEVER. I have one docker container with 6.2022 in it and it works fine. Exact same setup for the docker container for 7.2022 and it cannot discover/failed to connect DSM. Running 6.2.3 DSM…

GenGT · July 30, 2022, 4:18pm

Hello,

Is this integration compatible with versión 7.1? With versión 7.0 is working without any problem, but updated my dsm and isn’t working anymore.
Deleted core.config_entries, DSM is autodetected, core.config_entries is generated with the correct info, service notification button test become green, but the integration show this error “{‘message’: ‘Failed to check new dsm from server.’”

Thanks

MoebuisRex · August 4, 2022, 3:05pm

Just a follow up on this issue. Maybe this will help.
I was having these same issues, specifically when trying to use the hostname of the NAS.
A ping from my computer revealed that it is fully qualified as “HostName.LocalDomain”
When I entered that into the integration, it was able to connect.
I know there are more issues listed in the thread, but this might help some people.

jorgeelima · August 30, 2022, 8:44am

Hello all

Any update on this issue?
I was able to get the integration running but after a few hours it fails and can’t get it running again

Any help, please?

Thanks

labertouche · October 8, 2022, 6:31pm

Please check above: Armin’s solution worked for me!

hroman · April 16, 2023, 9:55am

I had the same problem:

{‘api’: ‘SYNO.Core.System.Utilization’, ‘code’: 105, ‘reason’: ‘Insufficient user privilege’, ‘details’: None}

DSM 7.1.1.

In my case the solution was to add user that was created to administrators group in DSM:

funstuff · June 24, 2024, 9:51am

Another weird thing that happened to work for me was entering my username in capital letters (XXX) instead of uncapitalized (xxx) letters.
Really no idea why this worked….