I made an improved NOAA Tides sensor for my family's summer house

Share your Projects!
, , ,
41

Spent the evening more or less rewriting the whole thing.

  • Sensors are now using async framework, so they won’t slow down startup
  • Connection failures are more graceful
  • Larger batches of tide data are requested at once
  • New batches of data aren’t requested while existing data is “fresh” (ie at least one tide prediction at least 3 hours in the future exists)
  • Using inheritance to keep things DRY
  • Added lots of debug logging

Going to leave this running on my personal instance overnight and if the logs look clean I’ll tag a new version (0.1.0 maybe!) tomorrow.

1 Like
42

Does changing the scan help?

scan_interval: 360    # Polling interval for the NOAA API

I changed from the 360 to 60 rebooted and both worked. Maybe coincidence?

43

Definitely just a coincidence!

44

Tried another reboot and back to errors

Logger: homeassistant
Source: custom_components/noaa_tides/sensor.py:83
First occurred: 6:44:05 PM (1 occurrences)
Last logged: 6:44:05 PM

Error doing job: Future exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.8/http/client.py", line 1240, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1286, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1235, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1006, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.8/http/client.py", line 946, in send
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xa6c88610>: Failed to establish a new connection: [Errno 110] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='tidesandcurrents.noaa.gov', port=80): Max retries exceeded with url: /mdapi/v1.0/webapi/stations/8531991.json?expand=details,sensors,products,disclaimers,notices,datums,harcon,tidepredoffets,benchmarks,nearby,bins,deployments,currentpredictionoffsets,floodlevels?units=metric (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xa6c88610>: Failed to establish a new connection: [Errno 110] Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/noaa_tides/sensor.py", line 66, in setup_platform
    noaa_sensor = NOAATidesAndCurrentsSensor(name, station_id, timezone, unit_system)
  File "/config/custom_components/noaa_tides/sensor.py", line 83, in __init__
    self._station = nc.Station(station_id)
  File "/usr/local/lib/python3.8/site-packages/noaa_coops/noaa_coops.py", line 25, in __init__
    self.get_metadata(self.stationid)
  File "/usr/local/lib/python3.8/site-packages/noaa_coops/noaa_coops.py", line 71, in get_metadata
    response = requests.get(metadata_url)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='tidesandcurrents.noaa.gov', port=80): Max retries exceeded with url: /mdapi/v1.0/webapi/stations/8531991.json?expand=details,sensors,products,disclaimers,notices,datums,harcon,tidepredoffets,benchmarks,nearby,bins,deployments,currentpredictionoffsets,floodlevels?units=metric (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xa6c88610>: Failed to establish a new connection: [Errno 110] Operation timed out'))
Home Assistant has started!
45

Yep- these are the exceptions the new branch catches. In a few hours I’ll have my final data on whether it’s working well enough to release.

46

Is there anyway to force a refresh of the tide information?

47

As far as I know, no, but there may be something built into sensors that allows it. In this case it won’t work. The fix for you will be to upgrade to 0.1.0

48

0.1.0 is tagged and has several data stability fixes and other good things!

1 Like
49

Installed and so far it is working. Does the scan_interval: tell it how often to poll the NOAA API? Is it in minutes?

50

installed latest. getting

2020-08-04 13:29:08 ERROR (SyncWorker_5) [custom_components.noaa_tides.sensor] Couldn't create a NOAA station object. Will retry next update. Error: HTTPConnectionPool(host='tidesandcurrents.noaa.gov', port=80): Max retries exceeded with url: /mdapi/v1.0/webapi/stations/8531680.json?expand=details,sensors,products,disclaimers,notices,datums,harcon,tidepredoffets,benchmarks,nearby,bins,deployments,currentpredictionoffsets,floodlevels?units=metric (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xad2d29b8>: Failed to establish a new connection: [Errno 110] Operation timed out'))
51

@Alex_Pupkin - that should be an intermittent error. It will keep retrying until the NOAA responds, and cache the results. You should only see that on startup.

52

@bschatzow yes, but in seconds

53

My sensor has been out for 6 or so hours. At first I thought it was related to the hurricane but it seems like the noaa_coops library is querying the NOAA “metadata” api, and that API is 404ing.

No easy workaround for now. I may fork and rewrite noaa_coops to use the data getter api instead, or I may just write my own query building logic and response parsing in the sensor itself. Either one will probably take a few days.

Edit:
Emailed the NOAA and found the issue with noaa_coops. I opened a PR which will hopefully get merged and released and the spice will flow again.

54

Didn’t post earlier as I assumed it was an issue with NOAA. Still not sure how tides near me and live weather is able to query the same API and get the data successfully

55

The answer is rather banal- noaa_coops hasn’t updated in over a year and the API moved locations.

56

Good luck.

57

Do you have new location - can i just change it, stick custom noaa_coops into custom_components/noaa_tides?

58

Yes, the fork on my github has a fix. I’m not sure if just putting it in custom_components/noaa_tides will work… I’m not familiar with python dependency management, and much less familiar with it when it comes to home-assistant, but it’s worth a shot.

If you do get it working please share the process here :slight_smile:

60

i just put noaa_coops.py into noaa_tides folder
changed import to from .noaa_coops import Station as nc and then changed nc references from nc.Station to nc - works fine now!

61

Not seeing any of my sensors this morning. Wind, temp or tides.