Reviewers needed: Environment Canada component with weather, sensor, and camera (radar) platforms

Fascinating… according to the metadata the images from NOAA and the images from the Govt. of Canada are both from GOES-16 (aka GOES-EAST), but the images have different projections.
As well, the Govt. of Canada images seem to have more imagery to the east and west (Newfoundland, for example, seems to be cropped off on the NOAA image)

As well, the NOAA gif seems to have black frames where the Govt. of Canada shows imagery (2019-06-26 04:20, 04:30, 04:50, 05:00, for example).

Very puzzling.

We’re up and running in v0.95.0! The documentation has also been posted at https://www.home-assistant.io/components/environment_canada.

Thanks again for everyone’s help, and please let me know if you have any trouble.

1 Like

Thank you Michael!
Great timing for a very happy Canada Day!
:maple_leaf:

1 Like

I’m trying to follow the documentation but this:

weather:
  - platform: environment_canada

will not pass config checking : Platform not found: weather.environment_canada
I’m running HA 0.95.1.
Camera and sensor component are not doing anything, I don’t see anything new (maybe related to weather component?).

Weird. I just spun up a new container using the 0.95.1 image and everything came up fine. Here’s the configuration.yaml I used:

default_config:

weather:
  - platform: environment_canada

camera:
  - platform: environment_canada

sensor:
  - platform: environment_canada

And here’s the docker-run.sh:

docker run -d \
    --name="ha-test" \
    --restart=always \
    -v /home/core/ha-test:/config \
    -e TZ=America/Toronto \
    -p 8124:8123 \
    --dns=1.1.1.1 \
    homeassistant/home-assistant:latest

I’m running HA in virtualenv. This is what I have in the log:

2019-06-27 21:25:33 ERROR (MainThread) [homeassistant.components.homeassistant] Platform not found: weather.environment_canada

2019-06-27 21:18:13 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up platform environment_canada
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 126, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/environment_canada/sensor.py", line 91, in setup_platform
    from env_canada import ECData
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/__init__.py", line 1, in <module>
    from .env_canada import *
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/env_canada.py", line 3, in <module>
    from PIL import Image
  File "/srv/homeassistant/lib/python3.7/site-packages/PIL/Image.py", line 93, in <module>
    from . import _imaging as core
ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directory
2019-06-27 21:18:14 ERROR (MainThread) [homeassistant.components.camera] Error while setting up platform environment_canada
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 126, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/environment_canada/camera.py", line 43, in setup_platform
    from env_canada import ECRadar
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/__init__.py", line 1, in <module>
    from .env_canada import *
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/env_canada.py", line 3, in <module>
    from PIL import Image
  File "/srv/homeassistant/lib/python3.7/site-packages/PIL/Image.py", line 93, in <module>
    from . import _imaging as core
ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directory

It looks like some of the dependencies are missing. I just installed in a new venv and everything came up properly, so I would suggest maybe reinstalling in a new venv and letting it pull in the dependencies on startup. Let me know if that works.

Pillow requires a jpeg dependency that cannot be installed via pip. I have this same issue with my custom_component for an NWS camera radar feed, shamelessly based off of yours and the BOM camera. Some users don’t have the required jpeg dependency and have to manually install via their local OS method.

I’m having the same problem as @e_jp, config check gives me a platform not found error.

I didn’t add the camera nor sensor, only the base platform.

I’m running in a brand new, just updated to 3.7.3 venv. I also don’t have the /srv/homeassistant/lib/python3.7/site-packages/env_canada folder.

Well it is a new virtualenv I installed for 0.94 :expressionless:

got this:

pi@raspberrypi:~ $ sudo apt-get install libopenjp2-7
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package libopenjp2-7 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'libopenjp2-7' has no installation candidate

Update: after doing a sudo apt-get update, I was able to install the lib
However I always do an apt-get update before each HA update, so I don’t know what was the issue here

new error:

2019-06-28 13:38:46 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up platform environment_canada
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/numpy/core/__init__.py", line 40, in <module>
    from . import multiarray
  File "/srv/homeassistant/lib/python3.7/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/srv/homeassistant/lib/python3.7/site-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 126, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/environment_canada/sensor.py", line 91, in setup_platform
    from env_canada import ECData
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/__init__.py", line 1, in <module>
    from .env_canada import *
  File "/srv/homeassistant/lib/python3.7/site-packages/env_canada/env_canada.py", line 9, in <module>
    import imageio
  File "/srv/homeassistant/lib/python3.7/site-packages/imageio/__init__.py", line 22, in <module>
    from .core import FormatManager, RETURN_BYTES
  File "/srv/homeassistant/lib/python3.7/site-packages/imageio/core/__init__.py", line 10, in <module>
    from .util import Image, Array, Dict, asarray, image_as_uint, urlopen
  File "/srv/homeassistant/lib/python3.7/site-packages/imageio/core/util.py", line 27, in <module>
    import numpy as np
  File "/srv/homeassistant/lib/python3.7/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/srv/homeassistant/lib/python3.7/site-packages/numpy/core/__init__.py", line 71, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using /srv/homeassistant/bin/python3.7),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory

But:

(homeassistant) homeassistant@raspberrypi:/home/pi $ pip3 install numpy
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: numpy in /srv/homeassistant/lib/python3.7/site-packages (1.16.4)

Thanks, I’m digging into this:

  • I’ve split up the env_canada classes and their dependencies, so the sensor won’t depend on image processing.
  • I’ve tested the new build of the env_canada library in an Ubuntu container and it worked.
  • What OS are you guys seeing the problems on?

Thanks for your help on this.

I’m running Hassbian.

I tried sideloading env_canada using pip in my venv, but it didn’t change anything. Platform is still not recognized.

I can do some tests if you want me to.

It looks like Hassbian is based on Debian Buster, so I spun up a container based on debian:buster and it seemed work. Here’s what I did:

Docker:

docker run -dit debian:buster --name debian
docker attach debian

Debian:

apt update
apt upgrade -y
apt install python3 python3-pip
pip3 install ipython env_canada
ipython

iPython:

In [9]: from env_canada import ECData, ECRadar

In [10]: data = ECData(station_id='ON/s0000430')

In [11]: print(data.conditions['condition'])
Mostly Cloudy

In [12]: radar = ECRadar(station_id='XFT')

In [13]: print(radar.get_loop()[:50])
b'GIF89aD\x02\xe0\x01\x87\x00\x00\xfe\xfe\xfe\xff\xff3\xf5\xec\xef\xf1\xdc\xdb\xff\xcc\x00\x99\xcc\xff\xed\xca\xc8\xee\xc0\xbb\xec\xb7\xb3\xe9\xac\xa9\xea\xa0\x9c\xff\x99\x00\x98'

I’m not sure if that helps, but at the same time I’m not sure how to replicate the error.

According to How do I fix the Pollen platform after upgrading to .82?, this error was solved previously by running:

apt install libatlas3-base

Ok, here’s what I did so far:

Sideloaded env-canada using pip in venv
upgraded env-canada to latest version (0.0.13) in venv (didn’t seem to have updated the HA package though)
–> ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directory
sudo apt-get install libopenjp2-7
–> ImportError: libtiff.so.5: cannot open shared object file: No such file or directory
sudo apt-get install libtiff5
–> ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory
sudo apt-get install libatlas3-base
–> ImportError: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIBC_2.27’ not found (required by /srv/homeassistant/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-arm-linux-gnueabihf.so)
sudo apt-get install libc6
–> libc6 is already the newest version (2.24-11+deb9u4).

ldd --version
–> ldd (Debian GLIBC 2.24-11+deb9u4) 2.24

So now, i’m kinda stuck as 2.24 seems the latest version for Hassbian, but numpy wants 2.27…

I’ll try to override the HA env_canada with a custom_component to see if I can get it to work.

Still getting the “platform not found” when trying to add the weather platform, but I also tried with another weather platform (met) and I get the same error… So maybe this platform has a bigger problem than just the environment_canada thing…

Still stuck on the sensor error… I’m now using env_canada 0.0.13 code, but still no bueno.

GLIBC 2.24 is the latest stable release for debian/raspbian/hassbian. 2.27 is unstable.

This doesn’t look to good for us Hassbian users…