Appdaemon with https

mbuffat · November 9, 2017, 3:41pm

I am running home-assistant on a rpi3 using https without problem.
I acces HA version 0.57.2 on hasbian with an URL like:
https://my_externalIP:8123
i.e. using the port 8123 with SSL (using SSL certification), because the classical port 443 is used by another server at home. It works without problem and I can access HA from outside my home.

I have tried to install appdaemon (on the same virtual env as HA) , but without success using
/srv/homeassistant/bin/appdaemon -c /home/homeassistant/.homeassistant
2017-11-09 16:24:14.799289 INFO AppDaemon Version 2.1.12 starting
2017-11-09 16:24:14.799889 INFO Configuration read from: /home/homeassistant/.homeassistant/appdaemon.yaml
2017-11-09 16:24:14.912715 WARNING Unable to connect to Home Assistant, retrying in 5 seconds
2017-11-09 16:24:20.044586 WARNING Unable to connect to Home Assistant, retrying in 5 seconds

with an error
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://myextIP:8123/api/config

my appdaemon.yaml looks like:

AppDaemon:
  logfile: STDOUT
  errorfile: STDERR
  threads: 10
#  cert_path: /home/homeassistant/ssl/
  cert_verify: False
#  api_port: 5000
#  api_key: !secret api_key
 api_ssl_certificate: /home/homeassistant/ssl/fullchain.pem
 api_ssl_key: /home/homeassistant/ssl/privkey.pem
HASS:
  ha_url: https://myextIP:8123

Any help will be appreciated

Best regards
Marc

ReneTode · November 9, 2017, 4:33pm

do you have a password set for HA?
then you need to provide the ha_key or appdaemon cannot connect to HA.

if you have access to your HA without using the external address (like http://ip:8123) you probably want to use that to connect appdaemon to your HA.

aimc · November 9, 2017, 4:33pm

If you have a key for HASS, you need to add it to the HASS section.

mbuffat · November 9, 2017, 9:50pm

Thank you
i have add the ha_key: with the HA passwd in the file appdaemon.yaml
and now it works.
I need however to use the external address IP and not the local IP address of HA (i can connect HA using this local address with https)
If I use the local address with https I got a lot of error using appdaemon

/srv/homeassistant/bin/appdaemon -c /home/homeassistant/.homeassistant
2017-11-09 22:31:14.411019 INFO AppDaemon Version 2.1.12 starting
2017-11-09 22:31:14.411574 INFO Configuration read from: /home/homeassistant/.homeassistant/appdaemon.yaml
2017-11-09 22:31:14.586684 INFO Starting Apps
2017-11-09 22:31:14.715025 INFO Got initial state
2017-11-09 22:31:14.715952 INFO App initialization complete
2017-11-09 22:31:14.716647 INFO Dashboards are disabled
2017-11-09 22:31:14.716976 INFO API is disabled
2017-11-09 22:31:14.805051 WARNING Disconnected from Home Assistant, retrying in 5 seconds
2017-11-09 22:31:19.927178 INFO Got initial state
2017-11-09 22:31:19.927763 INFO App initialization complete
2017-11-09 22:31:20.014757 WARNING Disconnected from Home Assistant, retrying in 5 seconds
2017-11-09 22:31:25.132346 INFO Got initial state
2017-11-09 22:31:25.132924 INFO App initialization complete
2017-11-09 22:31:25.220007 WARNING Disconnected from Home Assistant, retrying in 5 seconds
2017-11-09 22:31:30.354950 INFO Got initial state
2017-11-09 22:31:30.355893 INFO App initialization complete
2017-11-09 22:31:30.460134 WARNING Disconnected from Home Assistant, retrying in 5 seconds
2017-11-09 22:31:35.581613 INFO Got initial state
2017-11-09 22:31:35.582192 INFO App initialization complete
2017-11-09 22:31:35.669020 WARNING Disconnected from Home Assistant, retrying in 5 seconds

aimc · November 9, 2017, 9:51pm

That’s the way https works - you are REQUIRED to use the external address as used to generate the certificate or https will refuse to connect.

ReneTode · November 9, 2017, 10:34pm

i connect locally with http and not with https.
i dont know if you can do that how you have setup now, but if you can then that would be the smartest way.

if setup like that AD doesnt need to go on the internet to come back to your local network to connect to HA.
so you will gain in speed and AD will still work if the internet is down.

mbuffat · November 27, 2017, 1:49pm

I have used HA with appdaemon using https but with the external address without problem until I lost my internet connection at home during several days. The appdaemon goes down and my home automation with HA becomes unusable.
Thus using an external address with https is not reliable for my home automation.
I have use both the

cert_path: <path/to/root/CA/cert>
cert_verify: False
in the configuration file appdaemon.yaml, but it does not work.
Debugging appdaemon, the problem appears in line 1207 of appdaemon.py

            sslopt = {}
            if conf.certpath:
                sslopt['ca_certs'] = conf.certpath
            ws = create_connection(
                "{}/api/websocket".format(url), sslopt=sslopt
            )

during the call create_connection.
From the documentation of websocket, to disable the ssl cert verification, you have to supply the following option sslopt:

sslopt={“cert_reqs”: ssl.CERT_NONE})

By adding this line before the call to create_connection (and importing ssl) ,

                import ssl
                 ..........
                sslopt['cert_reqs'] = ssl.CERT_NONE

I was able to run appdaemon using a local internet address with https.
May be this should be included in the next release with a test of the parameter cert_verify.

aimc · November 27, 2017, 3:20pm

OK thanks for your research -, this will be in the next version (Probably the v3 Beta, releasing early in the new year).

A.

ReneTode · November 27, 2017, 5:24pm

but why?
why would anyone want to use certification inside his private network?
certification is to securre a secure connection between client and server.
and i may hope that ANY connection inside your own network is secured.
as i see it:
certification inside a secure network:
advantages:

none

disadvantages:

slowing down the speed
more difficult to setup
it creates difficulties with combining in and outside because in most situations only 1 port can be secure.

mbuffat · November 27, 2017, 8:07pm

The problem is if you use https to access HA from outside , you have to use https even from your local network. You cannot use http from your local network.
As appdaemon needs to access HA, the only way is to use https but without certification for the connection between appdaemon and HA

ReneTode · November 27, 2017, 8:45pm

i access HA from outside with https and from inside with http.
the way to set that up: (and avoid the risk of having your automations stop when the internet is down)

aimc · November 27, 2017, 11:44pm

I agree with Rene and have a similar setup, it makes everything simpler AND more resilient.

mu3 · December 25, 2017, 8:54pm

Can you please share your nginx conf?

ReneTode · December 25, 2017, 11:49pm

depends on what you want.
i have HA and AD enabled by NGINX.
but i have no need for dashboard outside my network, so i didnt enable that.

this is my NGINX setup for the moment:

server {
        listen 80;
        server_name *.duckdns.org;
        return 301 https://$server_name$request_uri;
}
server {
        listen 443 ssl;
        server_name YOURNAME.duckdns.org;
        ssl on;  
        ssl_certificate /etc/letsencrypt/live/YOURNAME.duckdns.org/fullchain.pem; # /etc/nginx/cert.crt;
        ssl_certificate_key /etc/letsencrypt/live/YOURNAME.duckdns.org/privkey.pem; # /etc/nginx/cert.key; 
        ssl_session_cache shared:SSL:10m;
        ssl_protocols TLSv1.1 TLSv1.2;
        ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
        ssl_prefer_server_ciphers on;
        proxy_buffering off;

        location / {
            proxy_pass http://HA_IP:8123;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }

        location /api/appdaemon/ {
            allow all;
            proxy_pass http://AD_IP:5051;
            proxy_set_header Host $host;
            proxy_redirect http:// http://;
        }

}

shlomki · February 12, 2018, 7:50am

Hey @aimc, is this in the beta? I’m experiencing the same issue.
I have cert_verify = False and still getting “certificate verify failed” in the error log.
Running latest beta of appdaemon.

aimc · February 12, 2018, 1:51pm

cert_verify is deprecated, but the current beta does this by default. What error are you getting?

shlomki · February 13, 2018, 8:40pm

I just followed the docs and cert_verify was there…
This is the full error I’m getting:

2018-02-11 21:06:13.057974 WARNING AppDaemon: Traceback (most recent call last):
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\site-packages\urllib3\connectionpool.py”, line 601, in urlopen
chunked=chunked)
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\site-packages\urllib3\connectionpool.py”, line 346, in _make_request
self._validate_conn(conn)
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\site-packages\urllib3\connectionpool.py”, line 850, in validate_conn
conn.connect()
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\site-packages\urllib3\connection.py”, line 326, in connect
ssl_context=context)
File "c:\users\shlomi\appdata\local\programs\python\python35\lib\site-packages\urllib3\util\ssl.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\ssl.py”, line 385, in wrap_socket
_context=self)
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\ssl.py”, line 760, in init
self.do_handshake()
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\ssl.py”, line 996, in do_handshake
self._sslobj.do_handshake()
File “c:\users\shlomi\appdata\local\programs\python\python35\lib\ssl.py”, line 641, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)

aimc · February 13, 2018, 8:58pm

Damn, not one of those lines is in my code so I’m not sure where to start. One question, are you using SSE or Websockets?

Can you post the entire log from startup so I can figure out how far it got, ideally with “-D DEBUG” set on the command line…

EDIT: I was wrong about the cert verify param, it is still in the code.

szentendre · October 16, 2019, 3:51pm

I found myself facing the same problem as mentioned in this thread that Appdaemon stops working when internet connection is lost. I installed Nginx and got it to work with Hass, though Appdaemon won’t connect to Hass locally (they run on the same server).

Will you also share the relevant part of your appdaemon.yaml and configuration.yaml? I need help to see if I missed something.

ReneTode · October 16, 2019, 8:21pm

my appdaemon.yaml wont really help you.
but when you share yours i can tell what is wrong (if any)

do you have SSL activated in Hass?, in that case you need to deactivate that.
everything about https and ssl should be removed from appdaemon.yaml and configuration.yaml

its also helpfull if you tell in which way you did install hass and AD. (hassio, venv, docker)