Hadashboard - entities connection issues

SoL · November 21, 2018, 2:17pm

Hi eveyone,
first thank you for sharing your developments.
I just installed appdaemon in my hassbian RPI3b to enable dashboard for HA (so, not hass.io addons).

Main issue: I can reach the dashboard at IP.5050, but all my HA entities are marked as “not found”, e.g. clock and reload work.

My HA ( https://my.duckdns.org:8123 with Let’sEncrypt) is running on virtualenv as well as appdaemon. Cause I’m not so confident, I followed BurnHA instructions (https://www.youtube.com/watch?v=yu7Rd-Rf2BU).

I made an initial mistake installing virtualenv on pi user and directory (can I simply remove ‘appdaemon’ dir under pi?), then I restarted the installation using ‘homeassistant’ user with ‘sudo su -s /bin/bash homeassistant’ and ‘source bin/activate’. I had to switch to ‘pi’ user for creating ‘/etc/systemd/system/appdaemon.service’ as homeassistant user is not sudoer.

This is my appdaemon.yaml in /home/homeassistant/appdaemon/conf:

log:
  logfile: /home/homeassistant/appdaemon/conf/appdaemon.log
  errorfile: /home/homeassistant/appdaemon/conf/aderror.log
appdaemon:
  threads: 10
  app_dir: /home/homeassistant/appdaemon/conf/apps/
  plugins:
    HASS:
      type: hass
      ha_url: http://192.168.0.38:8123
hadashboard:
  dash_url: http://192.168.0.38:5050
  dash_dir: /home/homeassistant/appdaemon/conf/dashboards

This is my /etc/systemd/system/appdaemon.service:

[Unit]
Description=Appdaemon for HASS
After=network-online.target

[Service]
Type=simple
User=homeassistant
ExecStart=/home/homeassistant/appdaemon/bin/appdaemon -D DEBUG -c "/home/homeassistant/appdaemon/conf"

[Install]
WantedBy=multi-user.target

This is appdaemon.log in /home/homeassistant/appdaemon/conf:

2018-11-21 13:47:15.344351 WARNING AppDaemon: HASS: ------------------------------------------------------------
2018-11-21 13:47:20.354093 DEBUG AppDaemon: Event type:ha_disconnected:
2018-11-21 13:47:20.355306 DEBUG AppDaemon: {}
2018-11-21 13:47:20.356436 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2018-11-21 13:47:20.357224 WARNING AppDaemon: HASS: ------------------------------------------------------------
2018-11-21 13:47:20.358044 WARNING AppDaemon: HASS: Unexpected error:
2018-11-21 13:47:20.358988 WARNING AppDaemon: HASS: ------------------------------------------------------------
2018-11-21 13:47:20.361333 WARNING AppDaemon: HASS: Traceback (most recent call last):
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/appdaemon/plugins/hass/hassplugin.py", line 152, i$
    "{}/api/websocket".format(url), sslopt=sslopt
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_core.py", line 511, in create_connection
    websock.connect(url, **options)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_core.py", line 223, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_handshake.py", line 79, in handshake
    status, resp = _get_resp_headers(sock)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_handshake.py", line 150, in _get_resp_h$
    status, resp_headers, status_message = read_headers(sock)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_http.py", line 298, in read_headers
    line = recv_line(sock)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_socket.py", line 102, in recv_line
    c = recv(sock, 1)
  File "/home/homeassistant/appdaemon/lib/python3.5/site-packages/websocket/_socket.py", line 94, in recv
    "Connection is already closed.")
websocket._exceptions.WebSocketConnectionClosedException: Connection is already closed.

This is my aderror.log in /home/homeassistant/appdaemon/conf :

2018-11-21 07:36:24.509215 ERROR AppDaemon: Required attribute not set or obtainable from any plugin: latitude
2018-11-21 07:36:24.510418 ERROR AppDaemon: AppDaemon is terminating
2018-11-21 07:36:24.511436 ERROR AppDaemon: Required attribute not set or obtainable from any plugin: longitude
2018-11-21 07:36:24.512179 ERROR AppDaemon: AppDaemon is terminating
2018-11-21 07:36:24.512950 ERROR AppDaemon: Required attribute not set or obtainable from any plugin: elevation
2018-11-21 07:36:24.513658 ERROR AppDaemon: AppDaemon is terminating
2018-11-21 07:36:24.514414 ERROR AppDaemon: Required attribute not set or obtainable from any plugin: time_zone
2018-11-21 07:36:24.515121 ERROR AppDaemon: AppDaemon is terminating

This is my starting dashboard ‘main.dash’ in /home/homeassistant/appdaemon/conf/dashboards:

title: main
widget_dimensions: [120,120]
widget_margins: [5,5]
columns: 8

clock:
    widget_type: clock

temp_esterna:
    widget_type: temperature
    title: Temp esterna
    entity: sensor.netatmo_zatmoe_temperature

temp_interna:
    widget_type: temperature
    title: Temp interna
    entity: sensor.netatmo_zatmoi0_temperature

luce_cucina:
    widget_type: light
    title: Luce cucina
    entity: light.hue_lightstrip_plus_1
    on_attributes:
      brightness: 100
      color_temp: 250

luce_pranzo:
    widget_type: light
    title: Luce cucina
    entity: light.hue_pranzo
    on_attributes:
      brightness: 100
      color_temp: 250

cancello:
    widget_type: switch
    title: Cancello
    entity: switch.cancello
    state_text: 1

reload:
    widget_type: reload

I hope I provided all the information required.
Thank you for your help
SoL

ReneTode · November 21, 2018, 4:57pm

you provided a lot of info, but i still got some questions.

it isnt clear to me if you installed appdaemon and home assistant in 2 different venvs.
in your appdaemon.yaml i see no authentication. so you dont use any kind of authentication in HA?
if you do use authentication then you need to add the token to your yaml, if you dont use it, you are asking for trouble with your HA opened to the world.

did you install duckdns and letsencrypt in HA?
if so then you probably cant reach HA using this url: http://192.168.0.38:8123
the ha_url needs to be a working url (test in browser)
if you did install letsencrypt inside HA, then my advice is to remove that (and duckdns) from HA and install a proxyserver (NGINX, caddy, etc) because with those stuff inside HA you cant get to HA locally anymore and you will always need internet to reach your HA.
also when you got SSL (letsencrypt) inside HA and you want to reach your dashboards without ssl you will get trouble because browsers dont allow that.

i see from the service that you use homeassistant user and appdaemon inside the homeassistant dir. so i guess you have only 1 venv.
thats not real good practice and can give trouble. you can use the same user but better have 2 venvs. or 1 of the both programs could break the other when you update.

that you can reach the dashboard says that appdaemon is running, but the errors show that you dont have a connection to HA, which is obvious from what i mentioned before.

so my advice:

keep home/homeassistant for homeassistant
create home/appdaemon and move your appdaemon configuration there
create a new venv and install appdaemon in it.
install a proxy server
remove letsencrypt and duckdns from HA
connect appdaemon with HA locally with token.

about venv (i didnt get it at first also)

first go to the place where you want the venv created with cd
mkdir the_name_from_your_venv (this just makes a new dir)
python3 -m venv the_name_from_you_venv (this copies all needed python files to the dir you just made)
source the_dir_to_your_venv/the_name_from_your_venv/bin/activate (this starts the activate program)
pip3 install wheel (in my case wheel doesnt install automaticly, might be unneeded)
pip3 install appdaemon

always start appdaemon when the venv is activated.
and in your service better not use debug for appdaemon. it gives you a shitload of info in your logs that you normally dont need.

source the_dir_to_your_venv/the_name_from_your_venv/bin/deactivate
this gets you out of the venv. you can create as much venvs as you like.
i got 6 (4x appdaemon, 2x HA)

i hope this helps you.

SoL · November 21, 2018, 6:26pm

Thank you Rene for your timely and detailed reply, I’ll try to answer you step-by-step hoping I dont vomit in the bus…
AD was installed in the same way as HA, so as novice I assume it’s the same venv.
I added no ha_key cause it was not indicated in BurnsHA video, I’m gonna add it as soon as I arrive at home.
Yes my HA is password protected ofc, I installed Duckdns, Let’sEncrypt!, PiHole, network restrictions and fail2ban. I reach my HA using duckdns while outside, VPN with OpenVPN for managing local configurations (ipcameras, dashboard etc).
I follow your advices and will install AD into a second venv, just one noob question: do I have to create it using pi or homeassistant user?
About the proxy server I’m not that confident about it, can you please suggest one link for instructions just to avoid big fails (I have tons of devices in HA it would be a disaster if I fail) ? Does it provide the same services as installing duckdns + letsencrypt (domain + ssl)? Indeed you are right, I’m having some issues for http and https calls.
Many thanks for your time and patience
SoL

ReneTode · November 21, 2018, 8:41pm

thats actually up to you.
the reason that HA is adviced to install under the user homeassistant is because the pi user is more vulnarable
so in general they advice to disable the pi user at all.
i dont think its that big of a risk if you keep your RPI as closed as possible and when you give the pi user a save password.
because you use the user homeassistant for home assistant i would keep it simple and do everything as the user homeassistant und forget about the pi user. (so dont log on as pi at all)

there are quite some links about installing nginx. but you shouldnt worry about losing HA.
if you install VLC on a pc you dont even think about it doing anything to open office dont you?
NGINX is just another program.
how to install nginx according to thepi.io is found here.
How to set up a web server on the Raspberry Pi - The Pi
and ha describes its part here
Reverse proxy using NGINX - Community Guides - Home Assistant Community

actually after sudo apt-get install nginx
all you need to do is edit your nginx config file and the file how to is at the bottom from the HA doc page.
you already got letsencrypt, so those files are there. and you dont need mysql or php.

you would still use duckdns and letsencrypt.
duckdns is just a nameserver and letsencrypt encrypts your data.
the only difference is that you have 1 extra stop in between.
here is a visual to make it more clear:

SoL · November 21, 2018, 9:32pm

Thanks again for you time and explanation.
I’m gonna restyle my configuration according to your advices, I built up mine by own without experience. It will require some time (babies own me) but I think I will do it on a new RPI. I read wrong your previous lines, I dont have to unistall Duck and Lets, just remove them from HA conf. The NGINX schema is ok, I just need to deeply understand the mechanisms ( I’m just a biologist ;-] ).
Will let you know the progress, tnx again and sorry for possible further requests.
SoL

ReneTode · November 21, 2018, 10:00pm

dont worry. i am glad to help.
i started out the same way, but in that time there was no option to use SSL and duckdns directly inside HA.
i think with that they send people the wrong direction.

NGINX is nothing more then another webserver, but then one that functions as gatekeeper.
everyone trying to come in must go through nginx and then nginx decides where something is rerouted to.

i had my reserves for a long time about installin nginx. and then at the point it hit me that homeassistant and dashboard are also webservers. and then i realised that installing nginx wasnt any different from installing home assistant.

if you run into trouble just ask, and ill response as soon as possible.

SoL · November 23, 2018, 4:54pm

Hi Rene,
just a (noob) clarification:

create home/appdaemon and move your appdaemon configuration there

/home/ dir includes just users, isnt it? If so, should I create an ‘appdaemon’ user to install and manage it?

and

because you use the user homeassistant for home assistant i would keep it simple and do everything as the user homeassistant und forget about the pi user. (so dont log on as pi at all)

If I should not use ‘pi’, then ‘homeassistant’ user should be a sudoer, isnt it?

Thanks again (mostly for your patience)
SoL

ReneTode · November 23, 2018, 5:04pm

yeah /home is for users. but if you are a root user you can create anything there.
i got my config all on an mounted usbdrive.

you can place your config anywhere you like, but my advice is to keep them side by side and not 1 inside the other. (allthough when you use the same user that problem is less severe.)

the user homeassistant needs to be sudoer to be able to create a venv i think (not even sure about that)
but for installing HA and AD you dont need sudo. (unless you install somewhere where you need sudo rights )

your welcome again