Continuous Set-up Woes with what seems to be every aspect/component of my installation. Please help

Where do I even start. First of all, I apologize, but this will be a LONG post. It’s part venting & then asking for help in resolving issues.

Begin Venting
After about a day of frustration and surfing resolutions on forums I managed to get Hass.io installed on an old laptop running Linux Mint 18.3 (Yes, yes, I know!). It was the only available device I could run the software on for testing until my rPi3 arrives, plus I was thinking to myself

“If I install it on this I’ll at least have some time to mess around with the environment and get used to things. Ohh, maybe I can even set up the devices I already have before I put my big order in on amazon.”

Needless to say I was pretty excited because like many of you I love programming and getting my hands dirty. Only problem is I hate linux and don’t really know my way around it well enough. But, despite that I still decided to give it a go. Again, I thought to myself

“If I can manage to get HA installed on this machine I should have no problem with the Pi since this is considered an advanced installation method”

I can’t stress how advanced the method would ultimately be. The issues that follow are in the order that they presented themselves.

The Goal: Install Hassio inside docker on the laptop using these commands as a general guide.

  1. Linux Mint 18.3 OS natively supports Python2 & Python3, but the Python3 version it “officially” supports and was built around is 3.5.3 and guess what the version minimum for HA is? 3.5.4
    • Luckily, a user on the discord server pointed me to this page on the linux mint site telling me how to install Python3.6 alongside the native Python3. So, first obstacle conquered.
  2. This next issue was more so a nuisance, rather than an actual problem.
    • Turns out Linux Mint has it’s own version of docker it uses & the one in the guide wasn’t compatible. But, that was resolved by simply installing mint’s native docker using pip3 install docker.io. Boom, obstacle 2 conquered.
  3. I finally had access to the interface & opened it up to access it remotely from work to fiddle during the day. I got my wifi switches integrated with not too much trouble.
    • Two are Tuya, so I ended up having a little trouble with them, but once I ditched the “official” component and used a custom one that allows completely local control over the switches they were good to go.

Here’s where I start running into the problems I have yet to solve
This is where I need your help!

  1. I tried adding a weather-related component and have yet to be successful. I’ve tried

    Made sure to specify them correctly in the configuration.yaml file with their respective api_key each time. Mind you, I was just looking to install a weather service. However, none of them worked. No weather info showed on my dashboard/main ui screen. Example text in the configuration.yaml file:

    configuration.yaml example
    weather:
      - platform: darksky
        api_key: REDACTED
    

    The above would lead to errors like the following:

    error log
     2018-10-04 15:59:02 ERROR (SyncWorker_2) [homeassistant.components.weather.darksky] Unable to connect to Dark Sky. HTTPSConnectionPool(host='api.darksky.net', port=443): Max retries exceeded with url: /forecast/REDACTED/40.12424,-74.97188?units=us&lang=en (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0xb45baaec>: Failed to establish a new connection: [Errno -3] Try again',))
     2018-10-04 15:59:02 ERROR (MainThread) [homeassistant.components.weather] darksky: Error on device update!
     Traceback (most recent call last):
        File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 251, in _async_add_entity
          await entity.async_device_update(warning=False)
        File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 352, in async_device_update
          yield from self.hass.async_add_job(self.update)
        File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
          result = self.fn(*self.args, **self.kwargs)
        File "/usr/local/lib/python3.6/site-packages/homeassistant/components/weather/darksky.py", line 147, in update
          self._ds_currently = self._dark_sky.currently.d
      AttributeError: 'NoneType' object has no attribute 'd'
      2018-10-04 15:59:12 WARNING (SyncWorker_2) [netdisco.ssdp] Found malformed XML at http://192.168.0.10:9080: status=ok
    
  2. I tried adding my netgear router (model C3000-100NAS) using the device_tracker.netgear component using the following variations in the configuration.yaml:

    configuration.yaml variations
    device_tracker:
       - platform: netgear
         host: 192.168.0.1
         port: 80
         username: admin
         password: REDACTED
    
    device_tracker:
       - platform: netgear
         host: 192.168.0.1
         username: admin
         password: REDACTED
    
    device_tracker:
       - platform: netgear
         host: 192.168.0.1
         password: REDACTED
    
    device_tracker:
       - platform: netgear
         password: REDACTED
    

    None of which worked. I kept receiving the following errors in the log:

    error log
     2018-10-04 08:04:46 ERROR (MainThread) [homeassistant.components.device_tracker] Unable to load /config/known_devices.yaml: Config file not found: /config/known_devices.yaml
     2018-10-04 08:04:47 ERROR (SyncWorker_3) [pynetgear] Error talking to API
     Traceback (most recent call last):
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
         chunked=chunked)
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
         six.raise_from(e, None)
       File "<string>", line 2, in raise_from
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
         httplib_response = conn.getresponse()
       File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
         response.begin()
       File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
         version, status, reason = self._read_status()
       File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
         line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
       File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
         return self._sock.recv_into(b)
     ConnectionResetError: [Errno 104] Connection reset by peer
    
     During handling of the above exception, another exception occurred:
    
     Traceback (most recent call last):
       File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
         timeout=timeout
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
         _stacktrace=sys.exc_info()[2])
       File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
         raise six.reraise(type(error), error, _stacktrace)
       File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
         raise value.with_traceback(tb)
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
         chunked=chunked)
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
         six.raise_from(e, None)
       File "<string>", line 2, in raise_from
       File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
         httplib_response = conn.getresponse()
       File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
         response.begin()
       File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
         version, status, reason = self._read_status()
       File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
         line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
       File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
         return self._sock.recv_into(b)
     urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
    
     During handling of the above exception, another exception occurred:
    
     Traceback (most recent call last):
       File "/usr/local/lib/python3.6/site-packages/pynetgear/__init__.py", line 247, in _make_request
         data=message, timeout=30, verify=False)
       File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 112, in post
         return request('post', url, data=data, json=json, **kwargs)
       File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
         return session.request(method=method, url=url, **kwargs)
       File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
         resp = self.send(prep, **send_kwargs)
       File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
         r = adapter.send(request, **kwargs)
       File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 495, in send
         raise ConnectionError(err, request=request)
     requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
     2018-10-04 08:04:47 ERROR (SyncWorker_3) [homeassistant.components.device_tracker.netgear] Failed to Login
     2018-10-04 08:04:47 ERROR (MainThread) [homeassistant.components.device_tracker] Error setting up platform netgear
     018-10-04 12:15:55 WARNING (SyncWorker_0) [netdisco.ssdp] Found malformed XML at http://192.168.0.10:9080: status=ok
    
  3. I even tried to add something as easy as a traffic cam using the camera.generic component to no avail. Example text in the configuration.yaml file:

    configuration.yaml example
    camera:
      - platform: generic
        still_image_url: http://www.dot35.state.pa.us/public/Districts/District6/WebCams/CAM06260CM124.jpg
        name: US 1 NB At Hulmeville Rd
    

    Which didn’t work, throwing tons of the same error in the log:

    error log
    2018-10-04 14:17:41 ERROR (MainThread) [homeassistant.components.camera.generic] Error getting new camera image: Cannot connect to host www.dot35.state.pa.us:80 ssl:None [Try again]
    

My thoughts
I’m thinking there’s some issue with my setup that’s preventing any HTTPS requests, but that wouldn’t really explain the Netgear issues since I’ve been able to log into my router insecurely (HTTP). I tried configuring the duckdns add-on to add ssl certs, etc. Afterwards, I was able to login to HA using the HTTPS endpoint, But then I just got bombarded with Handshake errors in the log and the previous issues were still present. So, I ditched the idea of DuckDNS for now since that doesn’t seem to be the issue. I also created an issue in the pynetgear repo with as much info as I could provide that I thought would help.

So, here I am. Exhausted, discouraged & about to give up the whole idea of using Home Assistant. Can some please help me?

A docker based install won’t care what version the host machine has for python, that is kind of the point of docker, each container takes care of itself.

Does you host machine have internet access? The error log says that it was unable to contact darksky.

and

Both are pointing to connection issues.

Check that you have a connection internally ping 192.168.0.1 and externally ping 8.8.8.8 both should return information.

If you can ping internally but not externally you may need to look into DNS settings.

The laptop is directly plugged into the router/modem (it’s a combo) via ethernet and the fact that I can access it from work tells me that it has internet access (otherwise I wouldn’t be able to access it). Also the fact that I was successly able to do sudo apt-get upgrade/updates directly on the laptop prior to installing HA.

See? That’s what has me baffled! I’m see all sorts of connection errors, but the machine obviously has internet access. Any thoughts?

Edit
Oh, and for the netgear component, the fact that laptop’s DIRECTLY WIRED TO THE THE FREAKING ROUTER/MODEM has me very annoyed and lost, because it has direct access to login to the router.

Do you have the SSH add-on installed? Possibly try ssh to the Hassio cli interface and check connectivity from there. That will rule out if the host machine has access but for some reason Hassio doesn’t.

I’m guessing it does as it was able to pull the latest version for you on install but worth a check.

I’m headed home now, so I’ll direct access to the laptop. Would you mind giving me a step-by-step guide on how to check if the hassio docker instance has connectivity?

Tried changing the dns addresses in daemon.json for docker and also enabled ipforwarding, but all issues still prevail :frowning_face:

From the hassio ssh login (not the host OS) can you ping?

Results from internal & external pings in the ssh session can be found below.
No connectivity issues are showing.

SSH Results for internal & external router pings
login as: root
[email protected]'s password:

  _    _                 _
 | |  | |               (_)
 | |__| | __ _ ___ ___   _  ___
 |  __  |/ _` / __/ __| | |/ _ \
 | |  | | (_| \__ \__ \_| | (_) |
 |_|  |_|\__,_|___/___(_)_|\___/




Our Cli:
$ hassio help

debug1: permanently_set_uid: 0/0
core-ssh:~# ping -c 10 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=63 time=0.581 ms
64 bytes from 192.168.0.1: seq=1 ttl=63 time=0.619 ms
64 bytes from 192.168.0.1: seq=2 ttl=63 time=0.662 ms
64 bytes from 192.168.0.1: seq=3 ttl=63 time=0.620 ms
64 bytes from 192.168.0.1: seq=4 ttl=63 time=0.602 ms
64 bytes from 192.168.0.1: seq=5 ttl=63 time=0.625 ms
64 bytes from 192.168.0.1: seq=6 ttl=63 time=0.613 ms
64 bytes from 192.168.0.1: seq=7 ttl=63 time=0.591 ms
64 bytes from 192.168.0.1: seq=8 ttl=63 time=0.655 ms
64 bytes from 192.168.0.1: seq=9 ttl=63 time=0.623 ms

--- 192.168.0.1 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 0.581/0.619/0.662 ms
core-ssh:~# ping -c 10 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=122 time=11.237 ms
64 bytes from 8.8.8.8: seq=1 ttl=122 time=10.317 ms
64 bytes from 8.8.8.8: seq=2 ttl=122 time=10.945 ms
64 bytes from 8.8.8.8: seq=3 ttl=122 time=10.614 ms
64 bytes from 8.8.8.8: seq=4 ttl=122 time=10.866 ms
64 bytes from 8.8.8.8: seq=5 ttl=122 time=11.933 ms
64 bytes from 8.8.8.8: seq=6 ttl=122 time=10.631 ms
64 bytes from 8.8.8.8: seq=7 ttl=122 time=10.795 ms
64 bytes from 8.8.8.8: seq=8 ttl=122 time=10.516 ms
64 bytes from 8.8.8.8: seq=9 ttl=122 time=10.538 ms

--- 8.8.8.8 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 10.317/10.839/11.933 ms
core-ssh:~#

Can you ping google.com or yahoo.com or any other website that would require some dns work?

No issue reaching either of them via ping in ssh

SSH Results for google and yahoo pings
core-ssh:~# ping -c 10 google.com
PING google.com (172.217.12.142): 56 data bytes
64 bytes from 172.217.12.142: seq=0 ttl=52 time=12.089 ms
64 bytes from 172.217.12.142: seq=1 ttl=52 time=12.271 ms
64 bytes from 172.217.12.142: seq=2 ttl=52 time=12.223 ms
64 bytes from 172.217.12.142: seq=3 ttl=52 time=11.055 ms
64 bytes from 172.217.12.142: seq=4 ttl=52 time=10.584 ms
64 bytes from 172.217.12.142: seq=5 ttl=52 time=11.349 ms
64 bytes from 172.217.12.142: seq=6 ttl=52 time=10.850 ms
64 bytes from 172.217.12.142: seq=7 ttl=52 time=11.144 ms
64 bytes from 172.217.12.142: seq=8 ttl=52 time=11.388 ms
64 bytes from 172.217.12.142: seq=9 ttl=52 time=12.156 ms

--- google.com ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 10.584/11.510/12.271 ms
core-ssh:~# ping -c 10 yahoo.com
PING yahoo.com (98.137.246.8): 56 data bytes
64 bytes from 98.137.246.8: seq=0 ttl=51 time=92.237 ms
64 bytes from 98.137.246.8: seq=1 ttl=51 time=87.047 ms
64 bytes from 98.137.246.8: seq=2 ttl=51 time=87.003 ms
64 bytes from 98.137.246.8: seq=3 ttl=51 time=86.477 ms
64 bytes from 98.137.246.8: seq=4 ttl=51 time=87.774 ms
64 bytes from 98.137.246.8: seq=5 ttl=51 time=89.400 ms
64 bytes from 98.137.246.8: seq=6 ttl=51 time=86.281 ms
64 bytes from 98.137.246.8: seq=7 ttl=51 time=86.544 ms
64 bytes from 98.137.246.8: seq=8 ttl=51 time=85.976 ms
64 bytes from 98.137.246.8: seq=9 ttl=51 time=88.538 ms

--- yahoo.com ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 85.976/87.727/92.237 ms
core-ssh:~#

Hopefully this’ll help diagnosing the issue. Results for another ping. This time for darksky.net.

SSH Results for darksky ping
core-ssh:~# ping -c 10 darksky.net
PING darksky.net (52.7.92.68): 56 data bytes

--- darksky.net ping statistics ---
10 packets transmitted, 0 packets received, 100% packet loss
core-ssh:~#

CC: @nickrout

Edit:
Question, what ports do I need to forward on my router other than the typical (25, 22, 8143, 30000-30010)? Do I need to forward 443 as well? I noticed a pattern with the weather components at least, where they show port 443 in the logs.

As it stands right now, I have my router forwarding the following ports.
25 - FTP (Active)
22 - SSH
8143 - HA Instance
30000-30010 - FTP (Passive)

What other ports should I forward?

And for completion I’ll post my system log for the laptop that’s running the HA instance. Does anything look off or raise suspicion?

(Log was too long to post directly)
https://pastebin.com/1Z3xzq3y

You shouldn’t need to forward any ports for HA to work. The preference among most users is to have all ports closed.

This really has me stumped. Can you try pinging darksky from the Mint host? This would tell if its an issue on the mint host or an issue within the docker/hassio setup.

I don’t have physical access to the laptop at the moment (and don’t believe I can ssh into it since I never set up direct ssh to it), but will try pinging darksky directly from it when I get home from work.

Even so, why won’t the netgear component work? Obviously, the internal & external ping to the router are successful. So, how come I continually fail to configure that component?

And to add to the ever-confusing situation, pings to openweathermap.org all succeed. Yet, component still throws Exception when calling OWM web API to update data

openweathermap.org ping results
core-ssh:~# ping -c 10 openweathermap.org
PING openweathermap.org (138.201.197.100): 56 data bytes
64 bytes from 138.201.197.100: seq=0 ttl=46 time=107.746 ms
64 bytes from 138.201.197.100: seq=1 ttl=46 time=107.587 ms
64 bytes from 138.201.197.100: seq=2 ttl=46 time=107.285 ms
64 bytes from 138.201.197.100: seq=3 ttl=46 time=107.230 ms
64 bytes from 138.201.197.100: seq=4 ttl=46 time=107.291 ms
64 bytes from 138.201.197.100: seq=5 ttl=46 time=107.493 ms
64 bytes from 138.201.197.100: seq=6 ttl=46 time=107.888 ms
64 bytes from 138.201.197.100: seq=7 ttl=46 time=107.772 ms
64 bytes from 138.201.197.100: seq=8 ttl=46 time=108.673 ms
64 bytes from 138.201.197.100: seq=9 ttl=46 time=107.101 ms

--- openweathermap.org ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 107.101/107.606/108.673 ms

throws hands up

I think I’ve just about given up on the HA environment :disappointed:

This is really odd. My only guess is that there is some odd interaction between the networking on the 18+ series of Debian/Ubuntu/Mint and Hass.io. They made a bunch of changes between 16.x and 18.x and I wonder if there are compatibility issues. I have fought with the 18.x series networking a few times recently for different reasons.

When does your Pi come in? I’m interested to see if you have issues there also as it would point to an issue in your network vs a hass.io issue. If you don’t have issues there then one could assume there are issues with Hass.io and Mint 18.x working together.

Edit, the install method you used does work, I did it recently and it was smooth.

It’s so disheartening tbh. I was hoping to be able to explore things, but just ended having to deal with all these issues that diluted my overall experience.

My Pi should be here within the next week. I ordered via Purse.io, so I don’t have a tracking number, but it was ordered Oct 3rd.

I really hope it’s just an issue with Mint 18.x and not my network. But, in the case that it is my network would you mind me messaging you to work directly to resolve it? You seem like one of the few people that’s very responsive.

Im not sure Ill be able to help with the networking but you are welcome to contact me on discord. Same name over there.

I appreciate it, will add you now.