24h precipitation forecast / 24u buienradar buienalarm

studioIngrid · November 27, 2023, 11:21am

Hi GertJan,

Thanks for the heads-up, I did notice, but didn’t connect the dots jet.

The gpsapi changed and no longer accepts extra parameters. The original request used &c= […] str(random.randint(0,999999999999999), to disable caching. It’s not pretty, but it works.

I thought the solution was to change that: change the command_line sensor, change the “command” to:

python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"

The code above sends a header with the request disabling caching, arguably prettier . Now the sensor works fine.

However, it’s still not giving me updated data. A normal browser request delivers the same data. I think they are downsizing this api.

So I went looking around for a different data source. And tada: https://graphdata.buienradar.nl/2.0/forecast. Still data blocks of 5 minutes.

I’m just testing this, when the sensor works without error for a few days, I will write an update.

Kind regards,
– Ingrid

studioIngrid · December 3, 2023, 12:33pm

Hi all,
this post is archived.
See this post for the updated version.

old post

This is the updated version of a two-hour forecast chart, blocks of 5 minutes. The scan_interval is set to 240 seconds = 4 minutes.

Hope this helps,

– Ingrid

STEP 1

Get the latitude and longitude of your location.

STEP 2
Set up sensors in your configuration.yaml file.

Buienalarm

Change <lat-3-decimals> and <lon-3-decimals> to your values.

command_line:
  - sensor:
      name: weather_buienalarm_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=<lat-3-decimals>&long=<lon-3-decimals>&region=nl&unit=mm%2Fu',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701605718
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_direct.last_changed | default(now())}}"
      scan_interval: 240

Buienradar

Change <lat-2-decimals> and <lon-2-decimals> to your values.

command_line:
  - sensor:
      name: weather_buienradar_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://graphdata.buienradar.nl/2.0/forecast/geo/RainHistoryForecast?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701083815
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_direct.last_changed | default(now())}}"
      scan_interval: 240

STEP 4

To display the graph you need the ApexCharts card by @RomRider.

The data_generator starts with some JS removing timepoints that have already passed.

type: custom:apexcharts-card
series:
  - entity: sensor.weather_buienalarm_direct
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var
      start=input.start*1000,delta=input.delta*1000;var
      out=[];input.precip.map((entry,index)=>{if(new
      Date(start+(index*delta))>new Date()){out.push([new
      Date(start+(index*delta)),entry])}}); console.dir(out.slice(0,22)); return
      out.slice(0,22);
  - entity: sensor.weather_buienradar_direct
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input=entity.attributes.data.forecasts;var
      out=[];input.map((entry,index)=>{if(new Date(entry.datetime)>new Date()){
      out.push([new Date(entry.datetime),entry.precipitation])}});return
      out.slice(0,22);
header:
  show: true
  title: 2h precipitation forecast
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: rain (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(2)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Light'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Medium'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Heavy'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'

GertJan · December 3, 2023, 8:39pm

Hi Ingrid, good work

GertJan · December 4, 2023, 5:14pm

Hi @studioIngrid,
I was analyzing the entity sensor.weather_buiradar_hourly. The data retrieved via forecast.buienradar.nl. (Simply out of curosity)
I saw that the entity is filled with 2 weeks of information. That’s why I was working on adjusting the code in the data_generator so that I can retrieve data over several days. Unfortunately, I have not been able to do that so far Do you know if there is a tool that can be used to generate the code based on the entity? That would be nice. I also saw that it not only contains the rain information, but much more information. I hope you have some tips for me. Thanks in advance!

1mfaasj · December 11, 2023, 12:19pm

Hi Ingrid
thanks for this project! One question about buienalarm, the graph remains at 0, so no data is shown in the graph.
The rain radar does work.
any idea what this is related to?

the sensor works and gives data in the attributes:

studioIngrid · December 11, 2023, 12:53pm

Hi 1mfaasj,
This is just the data that is published by buien alarm, they predict it won’t rain .

This is the data on their website. No rain!

And this is from the buien radar website, they do predict light rain.

– Ingrid

1mfaasj · December 11, 2023, 12:55pm

ah I thought if one expects it, so does the other. but then it will just work I guess. thanks for your quick response!

studioIngrid · December 11, 2023, 1:02pm

And here are some styles:
Fixing the padding or the card, the title is now the same as others. And the tooltip is a little more compact.

style: |
  ha-card #graph {
    margin: 3px 10px;
  }
  ha-card .apexcharts-canvas * {
    font-family: var(--ha-card-font-family, inherit) !important;
  }
  ha-card #header__title {
    color: var(--ha-card-header-color, --primary-text-color);
    font-size: var(--ha-card-header-font-size, 24px);
    padding-top: 10px;
    font-weight: normal;
  }
  ha-card .apexcharts-tooltip * {
    line-height: 0.5;
    font-size: 12px !important;
  }

1mfaasj · December 19, 2023, 1:31pm

I’m watching the buienalarm sensor for a week now but it stays on 0.
It’s quite rainy today and when I look at Buienalarm Den Haag - Weersverwachting en buien radar - Buienalarm it shows data.

Am I missing something?

command_line:
#buienalarm
  - sensor:
      name: weather buienalarm direct
      unique_id: weather_buienalarm_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=52.07667&long=4.29861&region=nl&unit=mm%2Fu',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_direct.last_changed | default(now())}}"
      scan_interval: 240 #4min

The sensor works I guess:

studioIngrid · December 22, 2023, 7:44am

So it turns out my “new” data source has been stale for over 2 years .

Will have to go back to finding a good data source. Up to now I only found webscrapers, no data end-pints.

During Christmas break, I’m going to (try to) find out which source the app on my phone uses. Nice project .

To be continued.
— Ingrid

diy62 · December 30, 2023, 9:41pm

Hi everyone,

I tried to make buienradar gspgadget to work in Homeassistant but until now I didn’t manage to make it a succes.

website (https://gpsgadget.buienradar.nl/data/raintext?lat=51.46&lon=3.55)

I’ve got this sensor in my command_line.yaml

  - sensor:
      name: buienradar_gpsgadget
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=51.46&lon=3.56&c='+str(random.randint(0,999999999999999))).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_direct.last_changed | default(now())}}"
      scan_interval: 300

I looked around on this forum and found and tweaked a little bit in the template-editor

{% set value = state_attr('sensor.buienradar_gpsgadget','data') %}

{% set data = value.split() %}
{% set value = namespace(list=[]) %}
{% for n in data %}
{% set value.list = value.list + [{ "time": n.split('|')[1], "rain": ( 10** (( n.split('|')[0] | float - 109 ) / 32 )) | round(1) }] %}
{% endfor %}

{% set time = value.list | map(attribute = "time") | list | to_json %}
{% set rain = value.list | map(attribute = "rain") | list | to_json %}

{{ time.replace('"','')}}
{{ rain }}

I got the next result, 2 list array’s

[23:15,23:20,23:25,23:30,23:35,23:40,23:45,23:50,23:55,00:00,00:05,00:10,00:15,00:20,00:25,00:30,00:35,00:40,00:45,00:50,00:55,01:00,01:05,01:10]
[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]

Can anybody help how to go on with this result to make it work in some bar-graph?

Thanks in advance

studioIngrid · December 31, 2023, 9:35am

Hi Eric,
Welcome to the topic. The problem is that https://gpsgadget.buienradar.nl/ datasource is a bit wobbly.
If you’re interested in learning: check this post.

I am currently testing new datapoints.

Kind regards,
– Ingrid

diy62 · December 31, 2023, 10:02am

Hi Ingrid,

Thanks for your reply, I watched this post before and saw your journey on this project. But the apexcharts are a bit slow on my opinion

I must admit that https://gpsgadget.buienradar.nl/ was not stable and reliable in last November but I use it for 3 years now on a little screen. I build it based on a Raspberry Pi Zero and a Display HAT Mini from Pimoroni and it works fine for 99% of the time. But I tried to make it work in Homeassistant now but it seems a bit complicated for someone who doesn’t have a programming background.

Anyway, thanks for your reply and I’ll keep an eye on this post.

studioIngrid · December 31, 2023, 10:36am

Hi everyone,

Another day, another data source.
I decided to go back to the drawing board, and rewrite the sensors with the most stable data source I could think of, the ones used by their apps. These are cdn.buienalarm.nl and graphdata.buienradar.nl.
The buienradar api was updated to version 3.0, thats why the previous sensor wasn’t working.

This is the – yet again – updated version of a two-hour forecast chart, blocks of 5 minutes. I increased the scan interval to 150 seconds = 2,5 minutes.

– Ingrid

STEP 1

Get the latitude and longitude of your location.

STEP 2
Set up sensors in your configuration.yaml file.

Buienalarm

Change <lat-2-decimals> and <lon-2-decimals> to your values.

command_line:
  - sensor:
      name: weather_buienalarm_minutes
      command: python3 -c "import requests; dataRequest = requests.get('https://cdn.buienalarm.nl/api/4.0/nowcast/timeseries/<lat-2-decimals>/<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); print(dataRequest);"
      unique_id: 1704013893
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_minutes.last_changed | default(now())}}"
      scan_interval: 150

Buienradar

Change <lat-3-decimals> and <lon-3-decimals> to your values.

command_line:
  - sensor:
      name: weather_buienradar_minutes
      command: python3 -c "import requests; response = requests.get('https://graphdata.buienradar.nl/3.0/forecast/geo/RainHistoryForecast?lat=<lat-3-decimals>&lon=<lon-3-decimals>',headers={'Cache-Control':'no-cache'}).text; data = '{\"data\":'+response.replace('\r\n',' ')+'}'; print(data);"
      unique_id: 1704016555
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_minutes.last_changed | default(now())}}"
      scan_interval: 150

STEP 4

To display the graph you need the ApexCharts card.

The data_generator starts with some JS removing timepoints that have already passed.

type: custom:apexcharts-card
series:
  - entity: sensor.weather_buienalarm_minutes
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var out=[];input.map((entry,index)=>{if(new Date(entry.time)>new Date()){out.push([new Date(entry.time),entry.precipitationrate])}});return out.slice(0,22);
  - entity: sensor.weather_buienradar_minutes
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input = entity.attributes.data.forecasts;var out=[];input.map((entry,index)=>{console.log(new Date(entry.dateTime));if(new Date(entry.dateTime)>new Date()){out.push([new Date(entry.dateTime),entry.dataValue])}});return out.slice(0,22); 
header:
  show: true
  title: 2h precipitation forecast
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: rain (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(1)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Light'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Medium'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 5 ? ''Heavy'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'
card_mod:
  style: |
    ha-card #graph {
      margin: 3px 10px;
    }
    ha-card .apexcharts-canvas * {
      font-family: var(--ha-card-font-family, inherit) !important;
    }
    ha-card #header__title {
      color: var(--ha-card-header-color, --primary-text-color);
      font-size: var(--ha-card-header-font-size, 24px);
      padding-top: 10px;
      font-weight: normal;
    }
    ha-card .apexcharts-tooltip * {
      line-height: 0.5;
      font-size: 12px !important;
    }

GertJan · December 31, 2023, 12:15pm

Hi Ingrid,

Thanks for the update !!!
I think the URL in the sensor has to be:
https://graphdata.buienradar.nl/3.0/forecast/geo/RainHistoryForecast?lat=xx.xxx&lon=x.xxx

Is there also an (3.0) update for the 24hour version?

studioIngrid · December 31, 2023, 4:19pm

Typo, thanks, changed it. Found another one in the annotations.
And good question, the answer is probably yes.
These will have to wait, now it’s time for champagne and ‘olliebollen’.

studioIngrid · January 3, 2024, 7:43pm

Hi all,

There are new data sources for the 24h sensors.
I will update the original post.

– Ingrid

GertJan · January 3, 2024, 8:32pm

Installed and looks fine.
Thanks Ingrid!

Jeroen.Rommers · August 16, 2024, 9:21pm

Nice work indeed, and I managed to get it going over here as well…

Maybe a bit off-topic, but is it now also possible to calculate the integrated amount of rainfall expected for the coming 24h, so that I can use that one to determine whether or not to turn on the sprinklers in my garden?

studioIngrid · August 25, 2024, 1:39pm

Certainly @Jeroen.Rommers , if you mean just a simple addition or the precipitation in mm per hour.
If you want something like area-under-the-curve, more math is required.

{{ state_attr('sensor.weather_buienradar_hours', 'data').forecasts
| sum(attribute='dataValue') 
}}

{{ state_attr('sensor.weather_buienalarm_hours', 'data') 
| sum(attribute='precipitationrate') 
}}

For a great sensor I would combine rate per 24 hours, and max rate.


{{ (state_attr('sensor.weather_buienradar_hours', 'data').forecasts
| max(attribute='dataValue')).dataValue | float
}}

{{ (state_attr('sensor.weather_buienalarm_hours', 'data') 
| max(attribute='precipitationrate')).precipitationrate | float
}}

Kind regards,
- Ingrid