Graph your Z-Wave mesh (Python, auto-update)

jhhbe · March 26, 2018, 7:30pm

It took me some time to figure it out, so I’ll summarize what I did. Log on to your ssh using putty or whatever on port 22222.

Find your container (your container id will be different):

root@hassio:~# docker container ls
CONTAINER ID        IMAGE                                     
c4d935187d4f        homeassistant/raspberrypi3-homeassistant

install the dependency networkx:

root@hassio:~# docker container exec c4d935187d4f pip3 install networkx
Collecting networkx
  Downloading networkx-2.1.zip (1.6MB)
Requirement already satisfied: decorator>=4.1.0 in /usr/lib/python3.6/site-packages (from 
networkx)
Installing collected packages: networkx
  Running setup.py install for networkx: started
    Running setup.py install for networkx: finished with status 'done'
Successfully installed networkx-2.1
You are using pip version 9.0.1, however version 9.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@hassio:~#

you can easily test the script (assuming it is in the share folder):

root@hassio:~# docker container exec c4d935187d4f /usr/bin/python3 /share/z-wave-graph.py

I suppose you need to reinstall the dependency when the container is rebuilt.

Indeed an addon would be really nice - I tried a bit yesterday with bestlibre’s python executor but got stuck and ended up following the instructions.

Have fun!

Timple · March 29, 2018, 7:29pm

Having an general home-assistant component or add-on would make it easier on all installations
Including other docker setups

bogdan · April 7, 2018, 8:25pm

Yes, an add-on would be great

In the mean time, can someone please write a step by step on how to install it? The README is not enough from my pov.
I did install the networkx, what’s next?
I tried to run the bin/py script but got different errors, the last one is ‘no homeassistant module’. And definitely there is hass.
Thank you

Kronyx · April 18, 2018, 11:57am

Same here, i have installed networkx but don’t understand what to do next, the readme is not enough from my pov

wachux · May 21, 2018, 9:00pm

Hi,

Great job OmenWild ! Can you please tell how can I change the default value of nodeDistane ? Every time I refresh the page it is 120. I tried to change the value in html file with no result. Can you give me a hint ?

ebnerjoh · May 24, 2018, 5:11am

Hi,

I am also struggeling with the docker section.

What I understood is that I have to copy the py to the config directory, and also the automation files and so on.

Then I have to install networkx.

But then I am out of idea.

Br,
Johannes

wachux · May 24, 2018, 8:43am

HI,

ebnerjoh - was struggling with it and finally got it to work. This is what i’ve done:

1- install networkx inside venv (Virtual Envirnoment). To do this you need to log into venv first (docker exec -it *************** bash) - where “******” is your Container ID. To see the container ID type “docker ps” in host CLI. Then inside venv type “pip3 install networkx”.

2 - copy z-wave-graph.py to the main config directory (where all the *.yaml files are).

3 - paste content of /automations/z-wave-graph.yaml to your automations.yaml

4 - paste content od /panel_iframe/z-wave-graph.yaml to your configuration.yaml

5 - in your secret.yaml file (if you dont have then create it in main config directory) add:
z_wave_graph_url: http://YOUR_LOCAL_URL:8123/local/z-wave-graph.html

6- paste content of /shell_commands/z-wave-graph.yaml to your shell_commands.yaml (if you dont have then create it in main config directory). Check inside venv where your python3 is located, it should be either in /usr/bin/python3 or /usr/local/bin/python3. Change the shell_commands.yaml to “z_wave_graph: /usr/bin/python3 /config/z-wave-graph.py” or “/usr/local/bin/python3 /config/z-wave-graph.py” dependig on where the python is.

7 - paste content of /www/z-wave-graph.html to your /www folder in config (if you dont have www folder then create it). The best way to do this. Log into the venv the same way as in point 1. cd into /config/www and get the file from Github: wget https://raw.githubusercontent.com/OmenWild/home-assistant-z-wave-graph/master/config/www/z-wave-graph.html

8- Restart HA

Please let me know if it woks.

ebnerjoh · May 24, 2018, 10:14am

Mhh,

getting:

2018-05-24 12:13:58 ERROR (MainThread) [homeassistant.loader] Unable to find component z_wave_graph
2018-05-24 12:13:58 ERROR (MainThread) [homeassistant.setup] Setup failed for z_wave_graph: Component not found.

wachux · May 24, 2018, 10:20am

did you downolad the z-wave-graph.py to the main folder of HA ?

ebnerjoh · May 24, 2018, 10:59am

Yes, I have, into the directory, where I have all config files. I also gave executable rights to everyone (a+x).

Stupid Question: When loading a component, shouldnt the component not be placed in components/ or custom_components/ ?

ebnerjoh · May 24, 2018, 11:20am

I guess it is only a small issue.

If I run

/usr/local/bin/python3 /config/z-wave-graph.py

Within the docker I am not getting any error message. After that I can also access

http://IP:8123/local/z-wave-graph.html

and I am able to see the graph.

wachux · May 24, 2018, 11:23am

As i see it its not a component built into HA. Its a py script loaded each time with an Automation. Forgot about one thing. Do you have at the end of your configuration.yaml:
script: !include scripts.yaml
shell_command: !include shell_commands.yaml ?

wachux · May 24, 2018, 11:32am

and if you try to enter http://IP:8123/z_wave_graph do you see the graph ?

ebnerjoh · May 24, 2018, 3:33pm

@wachux

Yes, I have the includes in:

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
shell_command: !include shell_commands.yaml
z_wave_graph:
  title: 'Z-Wave Graph'
  icon: mdi:vector-square
  url: !secret z_wave_graph_url

I created a file in the main config directory named shell_commands.yaml

z_wave_graph: /usr/local/bin/python3 /config/z-wave-graph.py

I have the following entry in my secrets file:

z_wave_graph_url: http://192.168.1.253:8123/local/z-wave-graph.html

and of course added the automation part into the automation-file.
I also added the py-file into the main config directory.

But I still get the error:

2018-05-24 17:26:31 ERROR (MainThread) [homeassistant.loader] Unable to find component z_wave_graph
2018-05-24 17:26:31 ERROR (MainThread) [homeassistant.setup] Setup failed for z_wave_graph: Component not found.

As mentioned above: If I connect to the docker and execute:
/usr/local/bin/python3 /config/z-wave-graph.py

I get a updated picture on:
http://192.168.1.253:8123/local/z-wave-graph.html

ebnerjoh · May 24, 2018, 5:53pm

I got it now temporary working with a workaround:

I am now executing this docker command as a scheduled script on my UnRaid-Server:

docker exec home-assistant /bin/bash -c "/usr/local/bin/python3 /config/z-wave-graph.py"

It is working, I only have no integration into HA Frontend. But I can live with that for the moment.

firstof9 · May 25, 2018, 6:41pm

Putting this line in secrets.yaml is the best option:

z_wave_graph_url: /local/z-wave-graph.html

This allows usage of reverse proxy and local network access without screwing up and giving you a 404.

Shell command for venv install (your path might be a bit different):

z_wave_graph: /srv/homeassistant/homeassistant_venv/bin/python <path to where you placed the python script>/z-wave-graph.py

finity · May 27, 2018, 7:29am

I think i’ve got everything installed correctly but I’m confused about setting the URL’s.

I’m getting the following errors in HA when it tries to run the script with the automation:

2018-05-27 03:15:08 ERROR (MainThread) [homeassistant.components.shell_command] Error running command: `/srv/homeassistant/bin/python3 /home/homeassistant/.homeassistant/bin/z-wave-graph.py`, return code: 1
NoneType: None

When I run it from the command line in the venv using the following command:

/srv/homeassistant/bin/python3 /home/homeassistant/.homeassistant/bin/z-wave-graph.py

I get:

Error connecting to server
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x754335b0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='XXXX.duckdns.org', port=8123): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x754335b0>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/remote.py", line 89, in __call__
    url, params=data, timeout=timeout, headers=self._headers)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='XXXX.duckdns.org', port=8123): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x754335b0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Error connecting to server
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x75433d70>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='bussnet.duckdns.org', port=8123): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x75433d70>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/remote.py", line 89, in __call__
    url, params=data, timeout=timeout, headers=self._headers)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='bussnet.duckdns.org', port=8123): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x75433d70>: Failed to establish a new connection: [Errno 111] Connection refused',))
Error, unable to connect to the API: cannot_connect

I can only assume that it’s because of the URL in the config yaml but i’m not sure I know how it needs to be setup. Right now it’s:

http:
  api_password: !secret http_password
  base_url: xxxx.duckdns.org:8123
  ssl_certificate: /etc/letsencrypt/live/xxxx.duckdns.org/fullchain.pem
  ssl_key: /etc/letsencrypt/live/xxxx.duckdns.org/privkey.pem
  ip_ban_enabled: True
  login_attempts_threshold: 5

do I need to set the base_url above to xxxx.duckdns.org:443?

how will that affect the certs and access from the outside of my lan? I thought we need to have the base_url set that way to enable proper port access.

firstof9 · May 27, 2018, 2:12pm

Remove your base_url it’s not needed and causes your errors.

In your venv execute: which python then use that path in the shell_command.yaml

finity · May 28, 2018, 1:53am

I have removed the “base_url” entry from configuration.yaml and restarted HA.

I activated the venv and ran “which python” and it returned “/srv/homeassistant/bin/python”.

i then ran “/srv/homeassistant/bin/python /home/homeassistant/.homeassistant/bin/z-wave-graph.py” from the command line over SSH and I still received the following errors:

Error connecting to server
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/remote.py", line 89, in __call__
    url, params=data, timeout=timeout, headers=self._headers)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
Error connecting to server
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/srv/homeassistant/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/remote.py", line 89, in __call__
    url, params=data, timeout=timeout, headers=self._headers)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
Error, unable to connect to the API: cannot_connect

I haven’t tried to run it from the automation. I don’t see the usefulness in that until I get it to run from the command line.

finity · May 28, 2018, 3:19am

If anyone didn’t see I updated the error I was getting. I had a little snafu when I posted the error log part.