24h precipitation forecast / 24u buienradar buienalarm

Thanks @studioIngrid for sharing! I archieved almost the same but this version is much simpler. I created two python scripts for Buienradar and Buienalarm and format it’s output for Apexcharts.

Using meteoserver is a excellent choice!

Will definitely start using your version.

Hi Ingrid,
Works great. Thank you !

Question: can you make a same routine, but only for buienradar and only today? That’s more accurate then one day.
Thanks !

Hi @GertJan,

To only use buienradar, just only include that entity under series.

What do you mean by ‘only today’? Buienradar service supplies two data files, one for the next 3 hours and one for the next few days, I can image there is some difference in accuracy between the two. So do you mean you would like to use the forecast for the next 3 hours?

– Ingrid

Hey @studioIngrid,

Yes, the forecast for the next 3 hours would be great. I now use the ‘custom:neerslag-card’ from aex351, but sometimes the card is not visible. It’s not working fine, so I would like to have your ApexChart solution !. Tried myself but I don’t have enough knowledge to get the data into Apex :frowning:

Would be great: Many thanks !

UPDATE 20-03-2023
I encountered the following error: “State attributes for sensor.weather_buienradar_hourly exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored”.
I updated the command line sensors, to truncate the JSON strings before storing them.

See original post.

@GertJan
I didn’t have time to test it thoroughly. But here you go.

I have included both buienradar and buienalarm. If you would like to use only one, just delete the other under series.
If you have the neerslag-card, then I assume that you have a buienradar sensor? For those who don’t have the 2h forcast sensors yet, use these:

sensor:
  - name: forecast_buienalarm
    platform: command_line
    command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=<lat-3-decimals>&lon=<lon-3-decimals>&region=nl&unit=mm%2Fu&c='+str(random.randint(0,999999999999999))).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
    unique_id: f90f2131fd05b6533705e59f7a18b037abec97f6e63aa9d0b831f85dd8383f69
    json_attributes:
      - data
    value_template: "last_changed: {{states.sensor.forecast_buienalarm.last_changed | default(now())}}"
    scan_interval: 240
  - name: forecast_buienradar
    platform: command_line
    command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=<lat-2-decimals>&lon=<lon-2-decimals>&c='+str(random.randint(0,999999999999999))).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"
    unique_id: 394757364d9b855917cbb50c8349ebbbade1db69a867a12ded54c625d023a637
    json_attributes:
      - data
    value_template: "last_changed: {{states.sensor.forecast_buienradar.last_changed | default(now())}}"
    scan_interval: 240

And the configuration of the card would be:

type: custom:apexcharts-card
series:
  - entity: <your-buienalarm-sensor>
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var
      start=input.start*1000,delta=input.delta*1000;var
      out=[];input.precip.map((entry,index)=>{if(new
      Date(start+(index*delta))>new Date()){out.push([new
      Date(start+(index*delta)),entry])}});return out.slice(0,22);
  - entity: <your-buienradar-sensor>
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input=entity.attributes.data.split(' ');var  today=new Date();var
      start=new
      Date(Date.parse(today.getFullYear()+'-'+String(today.getMonth()+1).padStart(2,'0')+'-'+String(today.getDate()).padStart(2,'0')+'T'+String(input[0]).substr(4,5)+':00')).getTime();var
      delta=300000;var out=[];input.map((entry, index) => {if(new
      Date(start+(index*delta))>new Date()) {  let num = Math.pow(10,
      ((parseInt(entry.substr(0,3)) || 0) - 109) / 32); let value = 
      Math.round(num * 100) / 100;  out.push([new
      Date(start+(index*delta)),(value)])}});return out.slice(0, 22);  
header:
  show: true
  title: Neerslag 2u
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: regen (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(2)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Licht'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Matig'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Zwaar'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'

Let me know if you encounter any errors.

EDIT: I just found out they only proved about 2 hours forecast. Around 23-25 values of 5 min each. Made some changes.

Kind regards,
– Ingrid

@studioIngrid,
Super that you are helping me out! great!

I think the card is almost working, but there is one problem:

Something is wrong with the scaling.Please take al look at the examples:

  1. buienradar website card
  2. your card
  3. neerslag-card from aex351
    1
    2
    3

at 12:35 there should be .1 mm/u rain. In your card it is .8 mm/u

This is de data from the buienradar website. Looks like 077 does not stand for mm/u
077|12:35
087|12:40
077|12:45
000|12:50
000|12:55
000|13:00
000|13:05

Hai @GertJan,

We know buienradar uses different data on their website then what they publish in their api. It also differs from the multi-day forecast.

Nonetheless I will take a look.

EDIT:
Buienradar app

Card

I think it’s just the way the data is published.

EDIT 2
I stand corrected. From aex’s code

Will update later tonight.

EDIT 3
Update in post. Again, credit goes to @aex351!

Thanks!

Kind regards,
— Ingrid

1 Like

okee… thanks @studioIngrid.
So, If I understand correctly, the yaml code for the Apexchart needs some modification?

I already changed it. You will see in the data_generator of buienradar the Math.pow.
Cheers

1 Like

That’s the one Perfect!
Maybe the 24h code needs the same modification?

As you might recall, the data format of the 24h sensors are different.
If you want to learn more, find your own sensor in developer tools and look and the data format.
The field key is “precipitationmm”, another assumption from my side, but I thought that would represent mm.

1 Like

Thanks a lot @studioIngrid !! Where/how did you find the entry point https://forecast.buienradar.nl/2.0/forecast ? It is great and I am looking for its documentation.

Hi Raoul,

If I recall correctly, it was a combination of learning from others, googling and inspecting the XHR requests used on the buienradar website.

You can learn a lot from these requests. However, unless the site/organization has a dedicated developer page, I rarely find documentation. It’s a deduction game, and then trial and error mostly.

Kind regards,

  • Ingrid
1 Like

@studioIngrid heb jij ook gemerkt dat het ophalen van de data buienradar (GPSDATA) niet meer werkt?

Hi GertJan,

Thanks for the heads-up, I did notice, but didn’t connect the dots jet.

The gpsapi changed and no longer accepts extra parameters. The original request used &c= […] str(random.randint(0,999999999999999), to disable caching. It’s not pretty, but it works.

I thought the solution was to change that: change the command_line sensor, change the “command” to:

python3 -c "import requests; import json; import random; dataRequest = requests.get('https://gpsgadget.buienradar.nl/data/raintext?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":\"'+dataRequest+'\"}'; print(data);"

The code above sends a header with the request disabling caching, arguably prettier :slight_smile: . Now the sensor works fine.

However, it’s still not giving me updated data. A normal browser request delivers the same data. I think they are downsizing this api.

So I went looking around for a different data source. And tada: https://graphdata.buienradar.nl/2.0/forecast. Still data blocks of 5 minutes.

I’m just testing this, when the sensor works without error for a few days, I will write an update.

Kind regards,
– Ingrid

2 Likes

Hi all,
this post is archived.
See this post for the updated version.

old post

This is the updated version of a two-hour forecast chart, blocks of 5 minutes. The scan_interval is set to 240 seconds = 4 minutes.

Hope this helps,

– Ingrid

STEP 1

Get the latitude and longitude of your location.

STEP 2
Set up sensors in your configuration.yaml file.

Buienalarm

  • Change <lat-3-decimals> and <lon-3-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienalarm_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://cdn-secure.buienalarm.nl/api/3.4/forecast.php?lat=<lat-3-decimals>&long=<lon-3-decimals>&region=nl&unit=mm%2Fu',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701605718
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienalarm_direct.last_changed | default(now())}}"
      scan_interval: 240

Buienradar

  • Change <lat-2-decimals> and <lon-2-decimals> to your values.
command_line:
  - sensor:
      name: weather_buienradar_direct
      command: python3 -c "import requests; import json; import random; dataRequest = requests.get('https://graphdata.buienradar.nl/2.0/forecast/geo/RainHistoryForecast?lat=<lat-2-decimals>&lon=<lon-2-decimals>',headers={'Cache-Control':'no-cache'}).text; dataRequest = dataRequest.replace('\r\n',' '); data = '{\"data\":'+dataRequest+'}'; print(data);"
      unique_id: 1701083815
      json_attributes:
        - data
      value_template: "last_changed: {{states.sensor.weather_buienradar_direct.last_changed | default(now())}}"
      scan_interval: 240

STEP 4

To display the graph you need the ApexCharts card by @RomRider.

The data_generator starts with some JS removing timepoints that have already passed.

type: custom:apexcharts-card
series:
  - entity: sensor.weather_buienalarm_direct
    name: buienalarm
    color: rgb(3, 170, 244)
    data_generator: >
      var input=entity.attributes.data;var
      start=input.start*1000,delta=input.delta*1000;var
      out=[];input.precip.map((entry,index)=>{if(new
      Date(start+(index*delta))>new Date()){out.push([new
      Date(start+(index*delta)),entry])}}); console.dir(out.slice(0,22)); return
      out.slice(0,22);
  - entity: sensor.weather_buienradar_direct
    name: buienradar
    color: rgb(3, 150, 244)
    data_generator: >
      var input=entity.attributes.data.forecasts;var
      out=[];input.map((entry,index)=>{if(new Date(entry.datetime)>new Date()){
      out.push([new Date(entry.datetime),entry.precipitation])}});return
      out.slice(0,22);
header:
  show: true
  title: 2h precipitation forecast
span:
  start: minute
graph_span: 2h
all_series_config:
  stroke_width: 3
  unit: mm/u
  type: area
  opacity: 0.2
  show:
    offset_in_name: false
apex_config:
  chart:
    height: 250px
  legend:
    show: false
  xaxis:
    tooltip: false
    border:
      show: true
    labels:
      format: HH:mm
      rotate: -45
      rotateAlways: true
      style:
        fontSize: 10px
  yaxis:
    - min: 0
      max: |
        EVAL:function (max) {
          window.chart2u_max = (Math.round(max*2)/2) + 0.5;
          return window.chart2u_max;
        }
      tickAmount: 5
      title:
        text: rain (mm/u)
  grid:
    xaxis:
      lines:
        show: true
  tooltip:
    enabled: true
    shared: true
    x:
      show: true
      format: ddd d MMM HH:mm
    'y':
      formatter: |
        EVAL:function (y) {
          return y.toFixed(2)
        }
  annotations:
    position: back
    yaxis:
      - 'y': 0.4
        label:
          text: 'EVAL:(window.chart2u_max > 0.4 ? ''Light'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        borderColor: 'EVAL:(window.chart2u_max > 0.4 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Medium'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 2
        borderColor: 'EVAL:(window.chart2u_max > 2 ? ''#000'' : ''none'')'
      - label:
          text: 'EVAL:(window.chart2u_max > 2 ? ''Heavy'' : '''')'
          style:
            background: '#666'
            color: '#FFF'
        'y': 5
        borderColor: 'EVAL:(window.chart2u_max > 5 ? ''#000'' : ''none'')'

1 Like

Hi Ingrid, good work :grinning:

Hi @studioIngrid,
I was analyzing the entity sensor.weather_buiradar_hourly. The data retrieved via forecast.buienradar.nl. (Simply out of curosity)
I saw that the entity is filled with 2 weeks of information. That’s why I was working on adjusting the code in the data_generator so that I can retrieve data over several days. Unfortunately, I have not been able to do that so far :frowning: Do you know if there is a tool that can be used to generate the code based on the entity? That would be nice. I also saw that it not only contains the rain information, but much more information. I hope you have some tips for me. Thanks in advance!

Hi Ingrid
thanks for this project! One question about buienalarm, the graph remains at 0, so no data is shown in the graph.
The rain radar does work.
any idea what this is related to?
image

the sensor works and gives data in the attributes:
image