Reviewers needed: Environment Canada component with weather, sensor, and camera (radar) platforms

michaeldavie · March 10, 2019, 2:08pm

I updated my PR with all three pieces. If you’re on Github, feel free to comment with your support!

davidbb · March 11, 2019, 7:27pm

Thanks again!

This morning my logs showed 4 instances of an error like this:

Error handling message: {‘id’: 18, ‘type’: ‘camera_thumbnail’, ‘entity_id’: ‘camera.mcgill_radar’}
Traceback (most recent call last):
File “/usr/local/lib/python3.5/site-packages/homeassistant/components/websocket_api/decorators.py”, line 17, in _handle_async_response
await func(hass, connection, msg)
File “/usr/local/lib/python3.5/site-packages/homeassistant/components/camera/init.py”, line 464, in websocket_camera_thumbnail
image = await async_get_image(hass, msg[‘entity_id’])
File “/usr/local/lib/python3.5/site-packages/homeassistant/components/camera/init.py”, line 94, in async_get_image
image = await camera.async_camera_image()
File “/usr/local/lib/python3.5/asyncio/futures.py”, line 381, in iter
yield self # This tells Task to wait for completion.
File “/usr/local/lib/python3.5/asyncio/tasks.py”, line 310, in _wakeup
future.result()
File “/usr/local/lib/python3.5/asyncio/futures.py”, line 294, in result
raise self._exception
File “/usr/local/lib/python3.5/concurrent/futures/thread.py”, line 55, in run
result = self.fn(*self.args, **self.kwargs)
File “/root/.homeassistant/custom_components/environment_canada/camera.py”, line 76, in camera_image
self.update()
File “/usr/local/lib/python3.5/site-packages/homeassistant/util/init.py”, line 224, in wrapper
result = method(*args, **kwargs)
File “/root/.homeassistant/custom_components/environment_canada/camera.py”, line 101, in update
self.image = self.radar_object.get_loop()
File “/root/.homeassistant/deps/lib/python3.5/site-packages/env_canada/env_canada.py”, line 257, in get_loop
frames = self.get_frames(self.LOOP_FRAMES)
File “/root/.homeassistant/deps/lib/python3.5/site-packages/env_canada/env_canada.py”, line 246, in get_frames
frames.append(frames[count - 1])
IndexError: list index out of range

Perhaps some corner case?

michaeldavie · March 11, 2019, 11:16pm

Yes, it looks like it; thanks for letting me know. That could happen if there were no images available at all, which is something I’m not handling at the moment. Are you able to tell we which station you’re using, and when the errors occurred? With that I can check the source at http://dd.weatheroffice.ec.gc.ca/radar/PRECIPET/GIF/ to see what might have been going on.

davidbb · March 12, 2019, 12:20am

It’s the CASBV station, but my logs rotated and I have lost the exact time/date of the errors. I will post again if I notice them.

michaeldavie · March 12, 2019, 3:29am

Wow, this is a real can of worms. I wasn’t handling selecting the 5-letter stations properly, which I’ve fixed now. I also needed to update the Wikipedia article that I use to get their coordinates, as the Timmins site recently changed. The new sites also output images at a different frequency, every 6 minutes instead of every 10. To handle both cases I’ve changed the way I’m scraping the GIF files from the website. I still need to fix how I detect failovers to the composite imagery.

Thanks for bringing this up!

pg.sylvestre · March 16, 2019, 3:32pm

Hi! Hope you find yourself well Michael. I’ve been testing and see things progressing well. One item that seems to not be yet addressed is the correct selection of the weather station. In my case, [ West End Ottawa ]. I’m roughly 16km from Ottawa airport and 35km from Gatineau Airport.

Here is the problem…Sensors are values from Chelsea and the Gatineau Airport. Somehow Geo Coordinates are not making the right station value. It may also be of use for some folks to pick their own station choice. Not sure if you have the code for that or not…
Let me know your thoughts and any help would be useful to help make the testing more accurate for me.

Tnx.

michaeldavie · March 16, 2019, 5:31pm

It looks like this could be because Environment Canada only reports the station locations to 2 decimal places, so they’re a bit off. If more than one station is relatively close, you might get the wrong one.

No worries though, you can definitely override it. Just add the following to your weather and sensor configurations to get the Ottawa airport instead:

station: 'ON/s0000430'

For anyone in a similar situation, the station codes are available at http://dd.weatheroffice.ec.gc.ca/citypage_weather/docs/site_list_en.csv.

michaeldavie · March 16, 2019, 10:10pm

Okay, time for another update. Here are the changes, in descending order of importance:

env_canada has been bumped to 0.0.8
I’ve removed the ec_ prefix from the sensors, so you’ll need to update your groups.yaml and customize.yaml if you were referring to those.
I’ve improved support for the 5-letter radar stations, and changed how radar fails over to composite images.
I’ve reordered the overlays on the radar images, so the roads are underneath the city labels.

To update:

Install the updated env_canada library:
- For Docker: docker exec [container name] pip install env_canada==0.0.8 --target /config/deps/lib/python3.6/site-packages --upgrade
- Otherwise: pip install env_canada==0.0.8 --target [config folder]/deps/lib/python3.6/site-packages --upgrade
Place the files from Dropbox in a folder at [config folder]/custom_components/environment_canada/
Restart your Home Assistant installation

As always, thanks for your feedback!

davidbb · March 21, 2019, 2:30pm

Is anyone else having issues with this component after updating to 0.90? For me it looks like weather and sensors are working, but the camera failed to start.

2019-03-21 06:07:36 ERROR (MainThread) [homeassistant.components.camera] Setup of platform environment_canada is taking longer than 60 seconds. Startup will proceed without waiting any longer.

pg.sylvestre · March 21, 2019, 6:33pm

Setup of platform environment_canada is taking longer than 60 seconds. Startup will proceed without waiting any longer.

using 0.0.8.

Entity not found: camera.franktown_radar…

Camera fails to initialize.

cariboo · March 21, 2019, 7:19pm

I updated yesterday, and am running into the same problem. The camera worked fine running 0.0.7

davidbb · March 21, 2019, 11:53pm

@pg.sylvestre @cariboo can you please post your version of Home Assistant? The camera platform worked fine for me on HA 0.88 and 0.89 with env_canada 0.0.8. I believe the issue is with HA 0.90.

cariboo · March 21, 2019, 11:55pm

I’m just updated to 0.91 to see if it made a difference.

michaeldavie · March 22, 2019, 12:04am

Yes, it seems like something changed in the master camera component in 0.90 that broke the radar platform, and I haven’t figured out what yet. It looks like 90.1 only changed the new stream component, so I don’t know if that will help.

I’m still looking into it; thanks for your patience.

Edit: I didn’t change anything, but when I restarted my 0.90 container to capture the logs it seems to have come up properly. Weird. If you’ve tried restarting and you’re still stuck, please let me know.

northcountry · March 22, 2019, 2:15am

@michaeldavie I’ve restarted several times and having the same issue with the radar image not working after upgrading to 0.90.

cariboo · March 22, 2019, 4:32am

I just restarted my development version, and the camera started working again:

northcountry · March 22, 2019, 12:47pm

Now my radar image is working this morning

pg.sylvestre · March 22, 2019, 12:48pm

well. I changes my real cameras to use stream_source: instead of input: and not the problem with the EC radar has gone away in HA version 0.90. using env_canada 0.0.8 . Don’t understand the relationship but seems to be a conflict with 0.90. stream to chrome cast function added in HA 0.90.

Just got HA 0.90.1 and everything looks good.

PS. Used Stream Component: Confirmed Cameras and Configurations to find correct streaming settings for my cameras.

davidbb · March 22, 2019, 1:17pm

Update to say that after restarting 0.90 with env_canada 0.0.8 and no changes in configuration, the radar is working again.

jazzyisj · March 26, 2019, 3:57pm

@michaeldavie

Was hoping to take your component here for a preview spin before it gets integrated.

When I run this command:

pip install env_canada==0.0.8 --target /config/deps/lib/python3.6/site-packages --upgrade

I get this error

C compiler: gcc -fno-strict-aliasing -Os -fomit-frame-pointer -g -DNDEBUG -Os -fomit-frame-pointer -g -DTHREAD_STACK_SIZE=0x100000 -fPIC

compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -c'
gcc: _configtest.c
failure.
removing: _configtest.c _configtest.o
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-CDQIcq/numpy/setup.py", line 415, in <module>
    setup_package()
  File "/tmp/pip-install-CDQIcq/numpy/setup.py", line 407, in setup_package
    setup(**metadata)
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/core.py", line 171, in setup
    return old_setup(**new_attr)
  File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/install.py", line 62, in run
    r = self.setuptools_run()
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
    return distutils_install.run(self)
  File "/usr/lib/python2.7/distutils/command/install.py", line 563, in run
    self.run_command('build')
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/build.py", line 47, in run
    old_build.run(self)
  File "/usr/lib/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/build_src.py", line 148, in run
    self.build_sources()
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
    self.build_library_sources(*libname_info)
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/build_src.py", line 292, in build_library_sources
    sources = self.generate_sources(sources, (lib_name, build_info))
  File "/tmp/pip-install-CDQIcq/numpy/numpy/distutils/command/build_src.py", line 375, in generate_sources
    source = func(extension, build_dir)
  File "numpy/core/setup.py", line 667, in get_mathlib_info
    raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program

----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-CDQIcq/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-4i8egh/install-record.txt --single-version-externally-managed --compile --home=/tmp/pip-target-uLuuDx" failed with error code 1 in /tmp/pip-install-CDQIcq/numpy/

Any idea where I might have went wrong? Running Hassio 0.90.1 on rPi3.