[Help request] Pollen sensor Dutch hooikoortsradar.nl

Who can help me to get the data from http://hooikoortsradar.nl and Pollennieuws | Pollenkalender

I used the ambee integraton before but since this is no longer free we can’t get get this info anymore.
Hope someone can point me the correct direction. So I (and many with me) can get this data again.
As an alternative or maybe better source we can use: Pollenrader – de pollenprognose bij jou in de buurt | Kleenex®. Kleenex provides this info for more locations so maybe this is the best solution.

Ok, are you confident in solving issues? This one will be tough!.
I made a script, in Python, it is giving a json that contains info from kleenex.

kleenex.py https://www.kleenex.nl/pollenradar?city=Maastricht

result:

{
"Gras": 0
"Bomen": 11
"Onkruid": 0
}

The script is the following

#!/usr/local/bin/python3.10
from requests_html import HTMLSession
import sys

url = sys.argv[1]

session = HTMLSession()
r = session.get(url)
r.html.render(sleep = 2, timeout = 20)
levels = r.html.find('.ppm-level')

print("{")
print("\"Gras\": " + levels[0].text.split()[0])
print("\"Bomen\": " + levels[1].text.split()[0])
print("\"Onkruid\": " + levels[2].text.split()[0])
print("}")

But for this script to work, it will require requests-html to be installed in your environment

python3.10 -m pip install requests-html

If you’re not using an x86-64 machine, chromium installation will fail.
To solve that, you have to follow what is written on this page:

If you managed to go through all those difficulties, it should work and you can run it as a command line in HA.

Good luck!

EDIT - CAVEAT
This is very dependant of the kleenex website.
It is assuming that you’ll always get 3 values, for gras, bomen and onkruid, in that order, with that specific ‘ppm-level’ class.
If the page change, everything will collapse!

EDIT2 Then, you can do gauges with the levels from the kleenex website:

Risiconiveau Boom Gras Onkruid
Laag 0-95 0-29 0-20
Gemiddeld 96-207 30-60 21-77
Hoog 208-703 61-341 78-266
Zeer hoog 704+ 342+ 267+

Go play with it thanks for your quick response and detailed answer

Additional detail, in case you don’t know how to get json values in sensor:

sensor:
  - platform: command_line
    name: pollen
    command: python3.10 kleenex.py https://www.kleenex.nl/pollenradar?city=Maastricht
    json_attributes:
      - gras
      - bomen
      - onkruid
    unit_of_measurement: ppm
    unique_id: kleenex_nl_pollenradar

Feel free to replace Maastricht with your location.


EDIT
Added unit of measurement and unique id


You can also try:

scrape:
  - resource: https://www.kleenex.nl/api/sitecore/Pollen/GetPollenContentCity
    method: POST
    payload: "city=Amsterdam"
    headers:
      User-Agent: Home Assistant
      Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    scan_interval: 3600
    sensor:
      - name: Pollen (grass)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-grass-count"
        unique_id: pollen_grass
        value_template: >-
          {{ value.split(" ")[0] }}
      - name: Pollen (weeds)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weeds-count"
        unique_id: pollen_weeds
        value_template: >-
          {{ value.split(" ")[0] }}
      - name: Pollen (trees)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-trees-count"
        unique_id: pollen_trees
        value_template: >-
          {{ value.split(" ")[0] }}
1 Like

Does that work now? I tried that in the past but as it is only rendered when javascript is running, it was not working.

EDIT Seems it does work, this is much more easier ! Nice, I’ll have some fun updating my old sensors :slight_smile:

1 Like

Thanks to both off you, I’ll test it today, i was not able to test it earlier because of illness

Do @mgosselink’s solution first, it is the easiest.
Here is a link to that integration, to put in your configuration.yaml file

Read it carefully because you might need libxml2 for this to work.

EDIT

        value_template: >-
          {{ value.split(" ")[0] | int(0) }}
        unit_of_measurement: ppm
        state_class: measurement

For the values to be number and generate statistics.

Hi @mgosselink , could you maybe add the way to add this into the Scrape integration, as Scrape via Yaml is depricated?

Per 2023.4 it is a Breaking Change :
The previously deprecated YAML configuration of the Scrape integration has been removed.

Scrape is now configured via the UI, any existing YAML configuration has been imported in previous releases and can now be safely removed from your YAML configuration files.

That would be very helpful. Most of it I can fit into this integration, but I can’t find the location to place the payload: "city=Amsterdam"

Thanks for your help!

Good remark, I had to add it in the core.config_entries file directly.
I have to think that this is not the way to do it but at least it is working again in 2023.4.0b7.

I’d rather not work in the .storage folder :sweat_smile:
I’ll just add it to my configuration.yaml for now, before upgrading to 2023.4.
So I guess it will be converted automatically (today).

I’m always going for beta to (try to) help others on this forum.

Scrape is now configured via the UI, any existing YAML configuration has been imported in previous releases and can now be safely removed from your YAML configuration files.

This was not true for me, I commented my scrape section in 2023.4.0b0 and it was just “not available” for all my scrape based sensors.

Did it work for you?

Hi,
No it didn’t convert into the Scrape integration. I decided to move on with HACS version, and compare them with the separate scrape sensors. From what I can see there is no difference in polling (every hour?).

Thanks for asking.

Is there a “official” HACS version or do I have to download the integration somewhere else?

BTW, just added the pollen counter via YAML by scraping the kleenex site. Working on 2023.5.4.

Using the same solution for a while now, scraping the sensors from Kleenex.

1 Like

Thanks for this. i made a simple view for the scraped data:

type: vertical-stack
cards:
  - type: horizontal-stack
    cards:
      - type: custom:mushroom-title-card
  - type: horizontal-stack
    cards:
      - type: picture
        image: /local/images/pollen_logo.jpg
        tap_action:
          action: none
        hold_action:
          action: none
  - type: horizontal-stack
    cards:
      - type: custom:mushroom-template-card
        primary: Gras
        secondary: '{{ states(''sensor.pollen_grass'') }}'
        icon: mdi:grass
        layout: vertical
        icon_color: cyan
      - type: custom:mushroom-template-card
        primary: Bomen
        secondary: '{{ states(''sensor.pollen_trees'') }}'
        icon: mdi:tree
        layout: vertical
        icon_color: pink
      - type: custom:mushroom-template-card
        primary: Onkruid
        secondary: '{{ states(''sensor.pollen_weeds'') }}'
        icon: mdi:sprout-outline
        layout: vertical
        icon_color: amber
  - type: horizontal-stack
    cards:
      - type: picture
        image: /local/images/pollenlegenda.jpg    

And the pictures:
pollen_logo
pollenlegenda

2 Likes

On the Kleenex website, the pollen counts are clickable, which leads to a screen that shows the specific pollen species. I can’t quite figure out how to add that info to the sensors you have configured here. Is that also possible?

What to config here:

And here:

Tried a lot of settings without succes yet :frowning:

With some specific settings I et sometimes value 272, almost like it’s protected to scrape the site?

Figured out my own question (solution on the bottom of this reply) and also yours:

in ‘Source/Bron’ add https://www.kleenex.nl/api/sitecore/Pollen/GetPollenContentCity?city=CITYNAME or postal code with the four numbers first, then %20 then the two letters (%20 being a space) So something like 1234%20AB

Method should be POST
Authentication should be basic
Skip username and password
Headers should be this:

User-Agent: Home Assistant
Content-Type: application/x-www-form-urlencoded;

leave the rest and continue to the next page

This one configures specific sensors. You should do this part three times if you want to add all three sensors, and seventeen times if you want to add detailed sensors (specific pollen types) too. I’ll explain how that works further on. For now:
Give it a name like Pollen (grass) to specify which sensor this will be.
Selector should be li.day-container:nth-child(1) > button
Leave index at zero
Attribute should be data-grass-count for grass, data-trees-count for trees, data-weeds-count for weeds, data-grass-detail for detailed grass pollen (1 type, Poaceae), data-tree-detail for detailed tree pollen (9 types, Hazelaar, Iep, Pijnboom, Els, Populier, Eik, Plataan, Berk, Cipres), data-weeds-count for detailed weed pollen (4 types, Bijvoet, Ganzevoet, Ambrosia, Brandnetel).
Template should be {{ value.split(" ")[0] }} for the count types (grass, trees, weeds) and {{ value.split(" ,)[1] }} for the detailed pollen types. The number 1 should go up by two for each type in the list (Eg: for Poaceae it should be 1, for Hazelaar also 1 but for Iep 3. If you don’t want to use all detailed ones but just a few, their indexes still go up. So Populier will always be index 9)
You can choose a device class and state class of your own liking. Same goes for the last setting.

If you have saved this, go to settings → integrations → scrape, click on ‘configure’ and click on ‘add sensor’ to add extra sensors.

A way quicker method is to use yaml:

scrape:
  - resource: https://www.kleenex.nl/api/sitecore/Pollen/GetPollenContentCity
    method: POST
    payload: "city=YOUR CITYNAME OR POSTAL CODE"
    headers:
      User-Agent: Home Assistant
      Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    scan_interval: 3600
    sensor:
      - name: Pollen (gras)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-grass-count"
        unique_id: pollen_gras
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(" ")[0] }}
      - name: Pollen (Poaceae)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-grass-detail"
        unique_id: pollen_poaceae
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[1] }}
      - name: Pollen (kruiden)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weeds-count"
        unique_id: pollen_kruiden
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(" ")[0] }}
      - name: Pollen (Bijvoet)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weed-detail"
        unique_id: pollen_bijvoet
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[1] }}
      - name: Pollen (Ganzevoet)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weed-detail"
        unique_id: pollen_ganzevoet
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[3] }}
      - name: Pollen (Ambrosia)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weed-detail"
        unique_id: pollen_ambrosia
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[5] }}
      - name: Pollen (Brandnetel)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-weed-detail"
        unique_id: pollen_brandnetel
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[7] }}
      - name: Pollen (bomen)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-trees-count"
        unique_id: pollen_bomen
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(" ")[0] }}
      - name: Pollen (Hazelaar)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_hazelaar
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[1] }}
      - name: Pollen (Iep)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_iep
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[3] }}
      - name: Pollen (Pijnboom)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_pijnboom
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[5] }}
      - name: Pollen (Els)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_els
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[7] }}
      - name: Pollen (Populier)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_populier
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[9] }}
      - name: Pollen (Eik)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_eik
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[11] }}
      - name: Pollen (Plataan)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_plataan
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[13] }}
      - name: Pollen (Berk)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_berk
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[15] }}
      - name: Pollen (Cipres)
        select: "li.day-container:nth-child(1) > button"
        attribute: "data-tree-detail"
        unique_id: pollen_cipres
        unit_of_measurement: ppm
        value_template: >-
          {{ value.split(",")[17] }}

edited to add unit_of_measurement: ppm to each sensor so it shows as a graph instead of numbers.

5 Likes