24h precipitation forecast / 24u buienradar buienalarm

Hi Raoul,

If I recall correctly, it was a combination of learning from others, googling and inspecting the XHR requests used on the buienradar website.

You can learn a lot from these requests. However, unless the site/organization has a dedicated developer page, I rarely find documentation. It’s a deduction game, and then trial and error mostly.

Kind regards,

  • Ingrid
1 Like

@studioIngrid heb jij ook gemerkt dat het ophalen van de data buienradar (GPSDATA) niet meer werkt?

Hi GertJan,

Thanks for the heads-up, I did notice, but didn’t connect the dots jet.

The gpsapi changed and no longer accepts extra parameters. The original request used &c= […] str(random.randint(0,999999999999999), to disable caching. It’s not pretty, but it works.

I thought the solution was to change that: change the command_line sensor, change the “command” to:

python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"

The code above sends a header with the request disabling caching, arguably prettier :slight_smile: . Now the sensor works fine.

However, it’s still not giving me updated data. A normal browser request delivers the same data. I think they are downsizing this api.

So I went looking around for a different data source. And tada: https://graphdata.buienradar.nl/2.0/forecast. Still data blocks of 5 minutes.

I’m just testing this, when the sensor works without error for a few days, I will write an update.

Kind regards,
– Ingrid

2 Likes

Hi all,
this post is archived.
See this post for the updated version.

old post

This is the updated version of a two-hour forecast chart, blocks of 5 minutes. The scan_interval is set to 240 seconds = 4 minutes.

Hope this helps,

– Ingrid

STEP 1

Get the latitude and longitude of your location.

STEP 2
Set up sensors in your configuration.yaml file.

Buienalarm

  • Change <lat-3-decimals> and <lon-3-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienalarm_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=<lat-3-decimals>&long=<lon-3-decimals>&region=nl&unit=mm%2Fu',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701605718
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_direct.last_changed | default(now())}}"
      scan_interval: 240

Buienradar

  • Change <lat-2-decimals> and <lon-2-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienradar_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://graphdata.buienradar.nl/2.0/forecast/geo/RainHistoryForecast?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701083815
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_direct.last_changed | default(now())}}"
      scan_interval: 240

STEP 4

To display the graph you need the ApexCharts card by @RomRider.

The data_generator starts with some JS removing timepoints that have already passed.

type: custom:apexcharts-card
series:
  - entity: sensor.weather_buienalarm_direct
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var
      start=input.start*1000,delta=input.delta*1000;var
      out=[];input.precip.map((entry,index)=>{if(new
      Date(start+(index*delta))>new Date()){out.push([new
      Date(start+(index*delta)),entry])}}); console.dir(out.slice(0,22)); return
      out.slice(0,22);
  - entity: sensor.weather_buienradar_direct
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input=entity.attributes.data.forecasts;var
      out=[];input.map((entry,index)=>{if(new Date(entry.datetime)>new Date()){
      out.push([new Date(entry.datetime),entry.precipitation])}});return
      out.slice(0,22);
header:
  show: true
  title: 2h precipitation forecast
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: rain (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(2)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Light'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Medium'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Heavy'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'

1 Like

Hi Ingrid, good work :grinning:

Hi @studioIngrid,
I was analyzing the entity sensor.weather_buiradar_hourly. The data retrieved via forecast.buienradar.nl. (Simply out of curosity)
I saw that the entity is filled with 2 weeks of information. That’s why I was working on adjusting the code in the data_generator so that I can retrieve data over several days. Unfortunately, I have not been able to do that so far :frowning: Do you know if there is a tool that can be used to generate the code based on the entity? That would be nice. I also saw that it not only contains the rain information, but much more information. I hope you have some tips for me. Thanks in advance!

Hi Ingrid
thanks for this project! One question about buienalarm, the graph remains at 0, so no data is shown in the graph.
The rain radar does work.
any idea what this is related to?
image

the sensor works and gives data in the attributes:
image

Hi 1mfaasj,
This is just the data that is published by buien alarm, they predict it won’t rain :slight_smile: .

image
This is the data on their website. No rain!

image
And this is from the buien radar website, they do predict light rain.

– Ingrid

1 Like

ah I thought if one expects it, so does the other. but then it will just work I guess. thanks for your quick response!

And here are some styles:
Fixing the padding or the card, the title is now the same as others. And the tooltip is a little more compact.

style: |
  ha-card #graph {
    margin: 3px 10px;
  }
  ha-card .apexcharts-canvas * {
    font-family: var(--ha-card-font-family, inherit) !important;
  }
  ha-card #header__title {
    color: var(--ha-card-header-color, --primary-text-color);
    font-size: var(--ha-card-header-font-size, 24px);
    padding-top: 10px;
    font-weight: normal;
  }
  ha-card .apexcharts-tooltip * {
    line-height: 0.5;
    font-size: 12px !important;
  }

1 Like

I’m watching the buienalarm sensor for a week now but it stays on 0.
It’s quite rainy today and when I look at Buienalarm Den Haag - Weersverwachting en buien radar - Buienalarm it shows data.

Am I missing something?

command_line:
#buienalarm
  - sensor:
      name: weather buienalarm direct
      unique_id: weather_buienalarm_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=52.07667&long=4.29861&region=nl&unit=mm%2Fu',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_direct.last_changed | default(now())}}"
      scan_interval: 240 #4min

The sensor works I guess:

So it turns out my “new” data source has been stale for over 2 years .

Will have to go back to finding a good data source. Up to now I only found webscrapers, no data end-pints.

During Christmas break, I’m going to (try to) find out which source the app on my phone uses. Nice project :nerd_face:.

To be continued.
— Ingrid

1 Like

Hi everyone,

I tried to make buienradar gspgadget to work in Homeassistant but until now I didn’t manage to make it a succes.

website (https://gpsgadget.buienradar.nl/data/raintext?lat=51.46&lon=3.55)

I’ve got this sensor in my command_line.yaml

  - sensor:
      name: buienradar_gpsgadget
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=51.46&lon=3.56&c='+str(random.randint(0,999999999999999))).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_direct.last_changed | default(now())}}"
      scan_interval: 300

I looked around on this forum and found and tweaked a little bit in the template-editor

{% set value = state_attr('sensor.buienradar_gpsgadget','data') %}

{% set data = value.split() %}
{% set value = namespace(list=[]) %}
{% for n in data %}
{% set value.list = value.list + [{ "time": n.split('|')[1], "rain": ( 10** (( n.split('|')[0] | float - 109 ) / 32 )) | round(1) }] %}
{% endfor %}

{% set time = value.list | map(attribute = "time") | list | to_json %}
{% set rain = value.list | map(attribute = "rain") | list | to_json %}

{{ time.replace('"','')}}
{{ rain }}

I got the next result, 2 list array’s

[23:15,23:20,23:25,23:30,23:35,23:40,23:45,23:50,23:55,00:00,00:05,00:10,00:15,00:20,00:25,00:30,00:35,00:40,00:45,00:50,00:55,01:00,01:05,01:10]
[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]

Can anybody help how to go on with this result to make it work in some bar-graph?

Thanks in advance

Hi Eric,
Welcome to the topic. The problem is that https://gpsgadget.buienradar.nl/ datasource is a bit wobbly.
If you’re interested in learning: check this post.

I am currently testing new datapoints.

Kind regards,
– Ingrid

Hi Ingrid,

Thanks for your reply, I watched this post before and saw your journey on this project. But the apexcharts are a bit slow on my opinion

I must admit that https://gpsgadget.buienradar.nl/ was not stable and reliable in last November but I use it for 3 years now on a little screen. I build it based on a Raspberry Pi Zero and a Display HAT Mini from Pimoroni and it works fine for 99% of the time. But I tried to make it work in Homeassistant now but it seems a bit complicated for someone who doesn’t have a programming background.

Anyway, thanks for your reply and I’ll keep an eye on this post.

Hi everyone,

Another day, another data source.
I decided to go back to the drawing board, and rewrite the sensors with the most stable data source I could think of, the ones used by their apps. These are cdn.buienalarm.nl and graphdata.buienradar.nl.
The buienradar api was updated to version 3.0, thats why the previous sensor wasn’t working.

This is the – yet again – updated version of a two-hour forecast chart, blocks of 5 minutes. I increased the scan interval to 150 seconds = 2,5 minutes.

– Ingrid

STEP 1

Get the latitude and longitude of your location.

STEP 2
Set up sensors in your configuration.yaml file.

Buienalarm

  • Change <lat-2-decimals> and <lon-2-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienalarm_minutes
      command: python3 -c "import requests; dataRequest = requests.get('https://cdn.buienalarm.nl/api/4.0/nowcast/timeseries/<lat-2-decimals>/<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); print(dataRequest);"
      unique_id: 1704013893
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_minutes.last_changed | default(now())}}"
      scan_interval: 150

Buienradar

  • Change <lat-3-decimals> and <lon-3-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienradar_minutes
      command: python3 -c "import requests; response = requests.get('https://graphdata.buienradar.nl/3.0/forecast/geo/RainHistoryForecast?lat=<lat-3-decimals>&lon=<lon-3-decimals>',headers={'Cache-Control':'no-cache'}).text; data = '{\"data\":'+response.replace('\r\n',' ')+'}'; print(data);"
      unique_id: 1704016555
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_minutes.last_changed | default(now())}}"
      scan_interval: 150

STEP 4

To display the graph you need the ApexCharts card.

The data_generator starts with some JS removing timepoints that have already passed.

type: custom:apexcharts-card
series:
  - entity: sensor.weather_buienalarm_minutes
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var out=[];input.map((entry,index)=>{if(new Date(entry.time)>new Date()){out.push([new Date(entry.time),entry.precipitationrate])}});return out.slice(0,22);
  - entity: sensor.weather_buienradar_minutes
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input = entity.attributes.data.forecasts;var out=[];input.map((entry,index)=>{console.log(new Date(entry.dateTime));if(new Date(entry.dateTime)>new Date()){out.push([new Date(entry.dateTime),entry.dataValue])}});return out.slice(0,22); 
header:
  show: true
  title: 2h precipitation forecast
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: rain (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(1)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Light'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Medium'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 5 ? ''Heavy'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'
card_mod:
  style: |
    ha-card #graph {
      margin: 3px 10px;
    }
    ha-card .apexcharts-canvas * {
      font-family: var(--ha-card-font-family, inherit) !important;
    }
    ha-card #header__title {
      color: var(--ha-card-header-color, --primary-text-color);
      font-size: var(--ha-card-header-font-size, 24px);
      padding-top: 10px;
      font-weight: normal;
    }
    ha-card .apexcharts-tooltip * {
      line-height: 0.5;
      font-size: 12px !important;
    }
1 Like

Hi Ingrid,

Thanks for the update !!!
I think the URL in the sensor has to be:
https://graphdata.buienradar.nl/3.0/forecast/geo/RainHistoryForecast?lat=xx.xxx&lon=x.xxx

Is there also an (3.0) update for the 24hour version?

Typo, thanks, changed it. Found another one in the annotations.
And good question, the answer is probably yes.
These will have to wait, now it’s time for champagne and ‘olliebollen’.

Hi all,

There are new data sources for the 24h sensors.
I will update the original post.

– Ingrid

Installed and looks fine.
Thanks Ingrid!