PostNL Integration

watermarkhu · October 16, 2024, 9:30pm

Thanks for this.

Here is my auto-entities based card that combines both the PostNL integration and this DHL sensor via multiscape. Unfortunately I’ve only got one package from PostNL at the moment.

For DHL package a DHL icon is displayed using custom brand icons.

type: custom:auto-entities
card:
  type: entities
  title: Package Tracking
filter:
  template: |
    {% set ns = namespace(packages=[]) %}
    {% if state_attr('sensor.postnl_delivery', 'enroute') is not none %}
      {% for item in state_attr('sensor.postnl_delivery', 'enroute') %}
        {% set expected_datetime = item.expected_datetime %}
        {% if not expected_datetime and item.planned_from and item.planned_to %}
          {% set start_time = as_timestamp(item.planned_from) | timestamp_custom('%Y-%m-%d %H:%M') %}
          {% set end_time = as_timestamp(item.planned_to) | timestamp_custom('%Y-%m-%d %H:%M') %}
          {% if start_time[:10] == end_time[:10] %}
            {% set expected_datetime = start_time[:10] + ' ' + start_time[11:] + '-' + end_time[11:] %}
          {% else %}
            {% set expected_datetime = start_time + ' - ' + end_time %}
          {% endif %}
        {% endif %}
        {% if not expected_datetime %}
          {% set expected_datetime = "unknown" %}
        {% endif %}
        {% set ns.packages = ns.packages + [{
          'type': 'custom:template-entity-row',
          'entity': 'sensor.postnl_delivery',
          'name': item.name,
          'state': item.status_message if item.status_message else 'No status available',
          'secondary': expected_datetime
        }] %}
      {% endfor %}
    {% endif %}
    {% if state_attr('sensor.dhl_packages', 'parcels') is not none %}
      {% for parcel in state_attr('sensor.dhl_packages', 'parcels') %}
        {% set receiving_time = parcel.receivingTimeIndication %}
        {% if receiving_time %}
          {% if receiving_time.start and receiving_time.end %}
            {% set start_time = as_timestamp(receiving_time.start) | timestamp_custom('%Y-%m-%d %H:%M') %}
            {% set end_time = as_timestamp(receiving_time.end) | timestamp_custom('%Y-%m-%d %H:%M') %}
            {% if start_time[:10] == end_time[:10] %}
              {% set display_time = start_time[:10] + ' ' + start_time[11:] + '-' + end_time[11:] %}
            {% else %}
              {% set display_time = start_time + ' - ' + end_time %}
            {% endif %}
          {% else %}
            {% set display_time = receiving_time %}
          {% endif %}
        {% else %}
          {% set display_time = "unknown" %}
        {% endif %}
        {% set ns.packages = ns.packages + [{
          'type': 'custom:template-entity-row',
          'entity': 'sensor.dhl_packages',
          'name': parcel.barcode + ' - ' + (parcel.sender.name if parcel.sender.name else 'unknown Sender'),
          'state': parcel.status | replace('_', ' ') | lower() | capitalize(),
          'secondary': display_time,
          'icon': "phu:dhl"
        }] %}
      {% endfor %}
    {% endif %}
    {% set sorted_packages = ns.packages | sort(attribute='secondary') %}
    [
      {%- for package in sorted_packages %}
        {{ package | tojson(indent=2) }}{{ ',' if not loop.last else '' }}
      {%- endfor %}
    ]
show_empty: false

Alwin_Hummels · October 17, 2024, 4:12am

Have you solve this problem, I have exactly the same unavailable message. And there is post on my way.

ptnijssen · October 17, 2024, 7:05pm

No issues on my side. However I think DHL is changing their URLs.

Login call: https://my.dhlecommerce.nl/api/user/login
Package call: https://my.dhlecommerce.nl/receiver-parcel-api/parcels

There are a couple of steps you can do to debug.

Step 1
First of all log in here:
https://my.dhlecommerce.nl/account/sign-in

Then open this page:
https://my.dhlecommerce.nl/receiver-parcel-api/parcels

You should see your parcels in JSON format.

Step 2
Enable logging in multiscrape. See the details here:

Next up, check in your logs what it says.

Intro_Verti · October 20, 2024, 2:42pm

Nope. Strange things are happening, but since 2 days it is working…
I did not change anything, so I keep my fingers crossed

neilsmith1805 · October 26, 2024, 4:11am

Im trying to install the browser extension (chrome) and I get this error:

Manifest is not valid JSON. expected value at line 7 column 1

Could not load manifest.

Has anyone came across this before?
thx

neilsmith1805 · October 26, 2024, 4:21am

Hey, @mSilencers , I’m curious how you track when the dog has been out?

Edwin_D · October 26, 2024, 10:40am

Using the postnl integration? . Probably because the postman returned the package to sender after being bit

watermarkhu · October 26, 2024, 11:20am

Another bit of inspiration for anyone. After setting up the PostNL sensor sensor.postnl_delivery and the DHL sensor sensor.dhl_packages, I’ve created a calendar named calendar.packages. This automation below automatically adds the to the calendar once a delivery time window is known.

From here, there are quite some possibilities to display on the frontend.

alias: Sync packages to calendar
description: ""
triggers:
  - trigger: state
    entity_id:
      - sensor.postnl_delivery
  - trigger: state
    entity_id:
      - sensor.dhl_packages
conditions: []
actions:
  - variables:
      packages: |
        {% set ns = namespace(packages=[]) %}
        {% if state_attr('sensor.postnl_delivery', 'enroute') is not none %}
          {% for item in state_attr('sensor.postnl_delivery', 'enroute') %}
            {% if item.planned_from and item.planned_to %}
              {% set ns.packages = ns.packages + [{
                'name': '📦POSTNL: ' + item.name,
                'state': item.status_message if item.status_message else 'No status available',
                'secondary': item.url,
                'expected_start': as_timestamp(item.planned_from),
                'expected_end': as_timestamp(item.planned_to)
              }] %}
            {% endif %}
          {% endfor %}
        {% endif %}
        {% if state_attr('sensor.dhl_packages', 'parcels') is not none %}
          {% for parcel in state_attr('sensor.dhl_packages', 'parcels') %}
            {% set receiving_time = parcel.receivingTimeIndication %}
            {% if receiving_time and receiving_time.start and receiving_time.end %}
              {% set ns.packages = ns.packages + [{
                'name': '📦DHL: ' + parcel.sender.name if parcel.sender.name else parcel.barcode,
                'state': parcel.status | replace('_', ' ') | lower() | capitalize(),
                'secondary': 'https://www.dhl.com/nl-en/home/tracking/tracking-parcel.html?submit=1&tracking-id=' + parcel.barcode,
                'expected_start': as_timestamp(receiving_time.start),
                'expected_end': as_timestamp(receiving_time.end)
              }] %}
            {% endif %}
          {% endfor %}
        {% endif %}
        {{ ns.packages }}
  - repeat:
      count: "{{ packages | length }}"
      sequence:
        - action: calendar.get_events
          metadata: {}
          data:
            start_date_time: |-
              {{ packages[repeat.index - 1].expected_start |
                  timestamp_custom('%Y-%m-%d') + ' 00:00:00' }}
            end_date_time: |-
              {{ packages[repeat.index - 1].expected_end |
                  timestamp_custom('%Y-%m-%d') + ' 23:59:59' }}
          response_variable: events
          target:
            entity_id: calendar.packages
          enabled: true
        - if:
            - condition: template
              value_template: >-
                {% set name = packages[repeat.index-1].name %}
                {% set found = events['calendar.packages']['events'] |
                selectattr('summary', 'equalto', name | trim) | list %}
                {{ found | length == 0 }} 
          then:
            - action: calendar.create_event
              metadata: {}
              data:
                start_date_time: >-
                  {{ packages[repeat.index - 1].expected_start |
                  timestamp_custom('%Y-%m-%d %H:%M:%S') }}
                end_date_time: >-
                  {{ packages[repeat.index - 1].expected_end |
                  timestamp_custom('%Y-%m-%d %H:%M:%S') }}
                summary: "{{ packages[repeat.index - 1].name | trim }}"
                description: >-
                  {{ packages[repeat.index - 1].state + ' ' +
                  packages[repeat.index - 1].secondary }}
              target:
                entity_id: calendar.packages
              enabled: true
    enabled: true
mode: single

neilsmith1805 · October 28, 2024, 8:45pm

For any others having same issue… you need to download the zip folder, unzip - and then install from the folder it creates (even if you create the same named folder).

So, I was able to get it installed an authenticate.

Trying to install the lovelace but there is so much conflicting posts about this I’m struggling to get anywhere.

neilsmith1805 · October 28, 2024, 9:14pm

Hello,
I have now managed to install the integration. I can see the basic sensors (See below).

However, I’m so confused how to install the card?
When I go to this page - the first message says: This card is not compatible with the current Home Assistant component

So, given i’ve the most recent integration, I’m thinking this means that this wont work for me. (although I have tried to install it)

When I go through all the posts in this post… it seems to be referencing this very lovelace card, so I’m a bit confused. It’s also 5 years old…

And then those who have had similar issues - there’s no similarity /resolution…

So, I have some questions:

Where and what is the correct lovelace card?
Is there a definitive guide on how to install this properly?

I know this will likely be down to my lack of dev. knowledge - but I am learning.

Thank you - and thanks to everyone who has made this integration possible, it’s a great idea!

sjanssen15 · October 30, 2024, 2:16pm

Same issue here, in the browser I do get the json output (before logging in ofcourse). I also tried Postman to troubleshoot the credentials and there it works.

Logs from HA:

2024-10-30 15:03:35.391 ERROR (MainThread) [custom_components.multiscrape.coordinator] Scraper_noname_0 # Updating failed with exception: Client error '400 Bad Request' for url 'https://my.dhlecommerce.nl/api/user/login'

For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

2024-10-30 15:03:35.514 ERROR (MainThread) [custom_components.multiscrape.coordinator] Scraper_noname_1 # Updating failed with exception: Client error '400 Bad Request' for url 'https://my.dhlecommerce.nl/receiver-parcel-api/parcels'

For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

2024-10-30 15:03:35.514 ERROR (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_1 # DHL Pakketten # Unable to scrape data: Skipped scraping because data couldn't be updated

Consider using debug logging and log_response for further investigation.

2024-10-30 15:03:35.514 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.parcels | selectattr('category', 'search', '(PROBLEM|CUSTOMS|DATA_RECEIVED|EXCEPTION|INTERVENTION|IN_DELIVERY|LEG|UNDERWAY|UNKNOWN)') | list }}'

2024-10-30 15:03:37.480 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)

EDIT: Somehow after some time it started working. I added a refresh button and this somehow triggered it?

FGOD1983 · December 26, 2024, 9:36am

I did add this to the HACS install and to the dashboard resources (althought with another url than mentioned in the readme due to that being the old link and nowadays hacs uses /hacsfiles/

but somehow Lovelace refuses to see the card. Is there anything I am missing to get this working?

FGOD1983 · December 28, 2024, 9:38am

Nevermind, got it working by adding the card manually to the dashboard through the raw editor… Not ideal, but it does work.

watermarkhu · January 5, 2025, 12:30pm

Although @ptnijssen’s config was working for me, the sensor was unavailable from time to time. Following multiscape’s documentation, this configuration makes use of the form_submit field. This is been working more consistently for me, but it will break if the CSS selector changes.

multiscrape:
  - name: DHL packages
    resource: "https://my.dhlecommerce.nl/receiver-parcel-api/parcels"
    scan_interval: 3600
    button:
      unique_id: refresh_dhl_packages
      name: refresh DHL packages
    form_submit:
      submit_once: True
      resubmit_on_error: True
      resource: "https://my.dhlecommerce.nl"
      select: "#root > div > div.MuiGrid2-root.MuiGrid2-container.MuiGrid2-direction-xs-column.mui-1y38ryc-drawer > div.MuiContainer-root.MuiContainer-maxWidthXs.mui-1ijkfjr > div > div > form"
      input:
        email: !secret email
        ":r1:": !secret dhl_password
    method: "get"
    headers:
      Content-Type: "application/json"
    sensor:
      - unique_id: dhl_packages
        name: DHL Packages
        value_template: "{{ value_json.parcels | selectattr('category', 'search', '(PROBLEM|CUSTOMS|DATA_RECEIVED|EXCEPTION|INTERVENTION|IN_DELIVERY|LEG|UNDERWAY|UNKNOWN)') | list | count }}"
        attributes:
          - name: parcels
            value_template: "{{ value_json.parcels | selectattr('category', 'search', '(PROBLEM|CUSTOMS|DATA_RECEIVED|EXCEPTION|INTERVENTION|IN_DELIVERY|LEG|UNDERWAY|UNKNOWN)') | list }}"

LarsA · January 12, 2025, 6:25pm

sorry, I had a nice status in my postnl_delivery today, but there is no agenda item added to the agenda. I have neatly created the calendar.packages. but unfortunately nothing is added? Could it be that my sensor.dhl_packages is unavailable?

IIIdefconIII · February 14, 2025, 11:08pm

is this still a working thing?

michelscholte · February 18, 2025, 9:41am

Hi. Is the multiscrape method still working? I’ve tried but for me it doesn’t work. I already changed “:r1:” to “:r3:” because the css selector is changed to this for what I can see. Hopefully you can help me to get it work.

olybol · February 18, 2025, 3:56pm

for me the same question. I tried several things to get DHL working. The URL gives in my case a json with parcels in my laptop browser but I think the multiscrape isn’t working due to the cookie/credentials

Milenco1 · March 10, 2025, 8:20pm

It seems DHL switched to CloudFlare protection, making it impossible to load the login-form. This is needed to retrieve the X-AUTH-TOKEN cookie to get the actual packages .json-file.

Milenco1 · March 11, 2025, 9:01pm

So, it took some time figuring it out, but I have the (Dutch) DHL integration sort of working again.

Instead of multiscrape (which can’t handle cookies), I manually get the cookie and use the command_line function instead:

Login to https://my.dhlecommerce.nl/account/sign-in
Visit https://my.dhlecommerce.nl/receiver-parcel-api/parcels and verify that you see the JSON output.
Open Developer Tools (usually F12) and get the X-AUTH-TOKEN cookie: Application → Storage → Cookies. Copy the value at X-AUTH-TOKEN

configuration.yaml:

command_line:
  - sensor:
      name: DHL Packages
      command: 'curl -v --cookie "X-AUTH-TOKEN=theverylongtokenyouhavecopied" https://my.dhlecommerce.nl/receiver-parcel-api/parcels'
      value_template: "{{ value_json.parcels | list | count }}"
      json_attributes:
        - parcels

I’m not sure how long the X-AUTH-TOKEN works, so we’ll see how long it lasts…