Environment Canada integration - 2021.11 onwards

ClockworkOrange38 · October 8, 2023, 7:20pm

Hello, I noticed Environment_Canada weather has stopped working yesterday after performing an update yesterday. (I think it was the HA 2023.10.1 update)

Current system info:
Home Assistant 2023.10.1
Supervisor 2023.10.0
Operating System 10.5
Frontend 20231005.0 - latest

I tried removing it and re-adding it back to integration, but failed to do so.

Steps taken:

removed and re-added Environment_Canada integration (tried to)
add Province Code/Site Name (Lat and Long were auto populated.)
returns [Unexpected error]
cannot proceed further

Your help is appreciated.

Edit: Did some testing and found that it’s an issue with my area. Other areas seem to be fine. Not sure if Environment_Canada removed my area… (ON/s0000585)

mchauhan · October 17, 2023, 1:13am

Hi, I tried multiple locations for Ontario, none of them worked and it kept giving the same error message “!Unexpected error”. We are supposed to input either station ID or longitude & latitude but it does not work either or both. Issue seems something different, Environment Canada integration does not work with existing installations and new installation.
If anyone else observed something different please do share! Thank You.

michaeldavie · October 17, 2023, 2:43am

That’s really odd. What version are you running, and is there anything showing in the logs?

michaeldavie · October 17, 2023, 2:46am

Yes, Environment Canada stopped reporting current conditions for that site. See the discussion at [environment canada] Weather update failed; no timestamp found · Issue #101533 · home-assistant/core · GitHub

This issue was fixed in version 2023.10.2.

mchauhan · October 18, 2023, 2:33am

I am running 2023.7.1, 2023.7.3 and 2023.10.1; three different installations. 2023.7.x both were working but now showing status sensor status as “Unavailable” whereas 2023.10.1 is a fresh installation and was unable to add Environment Canada integration.

I upgraded 2023.10.1 to 2023.10.3 and it worked. All the weather sensors are getting updates. Thanks for pointing to issue#101533, they fixed it in version 2023.10.2. Your link really helped in fixing the issue that has been going on for almost a month. Thanks again.

sutefu80 · December 6, 2023, 9:03pm

Hi, is there any way to access the hourly forecast information? The integration page describes an hourly sensor (Current conditions and hourly forecast, disabled by default), but I can’t find any way to enable this in the integration. However, I can see the hourly data is displayed in the weather card. Thank you.

ValMarDav · December 6, 2023, 10:04pm

Hi, There will be a change in early 2024 to get the forecasts. Weather.xxxx entities will no longer provide forecasts automatically, you must request them. Here’s how I do it.

template: 
          
  - trigger:
      - platform: time_pattern
        hours: /1
    action:
      - service: weather.get_forecasts
        data:
          type: hourly
        target:
          entity_id: weather.shawinigan
        response_variable: previsionshoraires
    sensor:
      - name: Prévisions météorologiques horaires
        unique_id: previsions_meteorologiques_horaires
        state: "{{ now().isoformat() }}"
        icon: mdi:hours-24
        attributes:
          forecast: "{{ previsionshoraires.forecast }}"

This will create a sensor under the name sensor.previsions_meteorologiques_horaires which will contain the forecasts.

Then you can get the forecasts like this.

template:
  - sensor:

    - name: 'def_previsions_temperature_h0'
      unique_id: 'def_previsions_temperature_h0'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[0].temperature }}"
      
    - name: 'def_previsions_temperature_h1'
      unique_id: 'def_previsions_temperature_h1'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[1].temperature }}"
  
    - name: 'def_previsions_temperature_h2'
      unique_id: 'def_previsions_temperature_h2'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[2].temperature }}"
  
    - name: 'def_previsions_temperature_h3'
      unique_id: 'def_previsions_temperature_h3'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[3].temperature }}"
  
    - name: 'def_previsions_temperature_h4'
      unique_id: 'def_previsions_temperature_h4'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[4].temperature }}"

    - name: 'def_previsions_temperature_h5'
      unique_id: 'def_previsions_temperature_h5'
      unit_of_measurement: "°C"
      state: "{{ state_attr('sensor.previsions_meteorologiques_horaires', 'forecast')[5].temperature }}"

JackJackattack4 · December 12, 2023, 5:47pm

Tried to extract the hourly weather forecast with your code above @ValMarDav. I could not make it work. I believe there is a difference in the data contained depending on the weather station I use (Sherbrooke).

Here is what I believe is the entity from Environment Canada that contains the best forecast information.

I placed the following code into the “template” section of my configuration.yaml.

trigger:
- platform: time_pattern
  hours: /1
  action:
- service: weather.get_forecasts
  data:
  type: hourly
  target:
  entity_id: weather.sherbrooke_previsions
  response_variable: previsionshoraires
  sensor:
- name: Prévisions météorologiques horaires
  unique_id: previsions_meteorologiques_horaires
  state: “{{ now().isoformat() }}”
  icon: mdi:hours-24
  attributes:
  forecast: “{{ previsionshoraires.forecast }}”

It creates the entity but it does not contain the forecast, see below:

I believe the issue lies within the data that is available from Environment Canada. I don’t see the hourly forecast, so I can’t extract it.

Does someone have a clue how to get the hourly temperature forecast out?

ValMarDav · December 12, 2023, 6:21pm

Hi, What I published 6 days ago, no longer works because they have again changed the response of the service in the publication of core 2023.12. Now the result contains the name of the Weather entity because you can specify several in the same query. So to read the forecasts, you must specify which weather entity you want to read from. Here is the new way to create the template.


template:

# Obtenir les pévisions horaires
          
  - trigger:
      - platform: time_pattern
        hours: /1
    action:
      - service: weather.get_forecasts
        data:
          type: hourly
        target:
          entity_id: weather.shawinigan
        response_variable: previsionshoraires
    sensor:
      - name: Prévisions météorologiques horaires
        unique_id: previsions_meteorologiques_horaires
        state: "{{ now().isoformat() }}"
        icon: mdi:hours-24
        attributes:
          forecast: "{{ previsionshoraires['weather.shawinigan'].forecast }}"

JackJackattack4 · December 12, 2023, 6:35pm

ValMarDav:

    sensor:
      - name: Prévisions météorologiques horaires
        unique_id: previsions_meteorologiques_horaires
        state: "{{ now().isoformat() }}"
        icon: mdi:hours-24
        attributes:
          forecast: "{{ previsionshoraires['weather.shawinigan'].forecast }}"

Hello,

It does work!

Thank you/merci.

My gosh, if they change the way the data is provided so often, how are you keeping track of this?

Montreal666 · January 26, 2024, 10:08pm

Does the integration provide a way to extract the full warning details:

As it only seem to provide the title/summary:

I don’t mind polling the data manually/externally and create a sensor but just want to check if someone figured out a way to achieve this before reinventing the wheel.

Thx!

pcwii · January 27, 2024, 1:12am

I don’t think this is built in. I did notice that there was an RSS feed link provide by environment Canada on the bottom of the warning (not shown in your picture). I am not sure if the HA RSS feed reader would create a sensor for this. I will try at my next opportunity.

Montreal666 · January 27, 2024, 3:30pm

Thanks, will try with the RSS feed. Will report back if positive.
Cheers

michaeldavie · January 27, 2024, 5:12pm

The integration used to pull the full alert data from the ECCC website using HTML scraping, since it’s not included in the XML data. However, HA took a hard stance against HTML scraping so I had to remove this functionality. I haven’t looked at this in a while, so if anyone has any suggestions I’m all ears!

pcwii · January 27, 2024, 10:24pm

I appreciate that scraping is no longer a part of the integration, but scraping is still an available option with HA. I am aware that EC could change their website at any time and thus break this scraping…But do you have any details on how we could scrape the website ourselves based on the Scrape integration that exists? No warranty is expected or implied by anything you can provide.

michaeldavie · January 28, 2024, 1:37pm

The code I was using to get the HTML is here:

github.com

michaeldavie/env_canada/blob/f3147825e25b9a4e4db8aebf682f69feacefd29e/env_canada/ec_data.py#L288


      
          
          # Update text summary
          period = get_condition(summary_meta['forecast_period'])['value']
          summary = get_condition(summary_meta['text_summary'])['value']
          
          self.conditions['text_summary'] = {
              'label': summary_meta['label'][self.language],
              'value': '. '.join([period, summary])
          }
          
          # Update alerts
          for category, meta in alerts_meta.items():
              self.alerts[category] = {'value': [],
                                       'label': meta[self.language]['label']}
          
          alert_elements = weather_tree.findall('./warnings/event')
          alert_list = [e.attrib.get('description').strip() for e in alert_elements]
          
          if alert_list:
              alert_url = weather_tree.find('./warnings').attrib.get('url')
              alert_html = requests.get(url=alert_url).content

This was quite a while ago, but I don’t think they’ve changed the site much.

Pepe_Le_Pew · February 9, 2024, 4:42pm

I cannot figure out how to turn on hourly forecasts again, I just had a SD card die and I am rebuilding. Hourly forecasts, were those lost when scraping was banned?

Is it possible to put the full version of Environment Canada addon into HACS with all it’s webscraping glory?

pcwii · February 19, 2024, 12:36am

@michaeldavie, I may have mentioned this before but would it be possible to have the summary added to an attribute so that it is not truncated by the 255 character limit?

gwww · February 19, 2024, 2:49am

The limit is imposed by Home Assistant. The max length of a field is 255.

pcwii · February 19, 2024, 3:02am

This limit does not exist for sensor attributes, so the idea would be to move the summary to an attribute field to overcome the 255 character limit.