Extract from IMAP sensor into calendar events

poudenes · August 10, 2023, 2:51pm

Hi All,

For me an impossible job. Maybe can help me with this?
Below you see a message that is added to a sensor by imap_content.

Output of the sensor:

Message: "Donderdagse Week van 33-2023\r\n\r\n390757 PM OUDENES\r\n\r\nDatum    Dienst  Begin Einde Fun Stdp\r\n________ ______  _____ _____ ___ ____\r\nma 14-08   6031  09:30 17:30 NRP NVT\r\ndi 15-08      R     \r\nwo 16-08   6002  07:00 15:00 NRP NVT\r\ndo 17-08   6002  07:00 15:00 NRP NVT\r\nvr 18-08  *  --     \r\nza 19-08      R     \r\nzo 20-08      R"
friendly_name: Imap full info sensor

Here easy readable way:

Donderdagse Week van 33-2023

390757 PM OUDENES

Datum    Dienst  Begin Einde Fun Stdp
________ ______  _____ _____ ___ ____
ma 14-08   6031  09:30 17:30 NRP NVT
di 15-08      R     
wo 16-08   6002  07:00 15:00 NRP NVT
do 17-08   6002  07:00 15:00 NRP NVT
vr 18-08  *  --     
za 19-08      R     
zo 20-08      R

- trigger:
    - platform: event
      event_type: "imap_content"
      id: "custom_event"
      event_data:
        sender: !secret imap_sender_2
  sensor:
    - name: "Imap full info sensor"
      state: >-
        {% if 'DiSys' in trigger.event.data["subject"] %}
          Nieuwe DW
        {% endif %}
      attributes:
        Message: "{{ trigger.event.data['text'] }}"

I want extract the week and use the into to add a event in a local calendar.
Given example will be than:

ma 14-08 6031 09:30 17:30 NRP NVT:

service: calendar.create_event
data:
  summary: "6031"
  start_date_time: "2023-08-14 09:30:00"
  end_date_time: "2023-08-14 17:30:00"
target:
  entity_id: calendar.work_shift

di 15-08 R:

service: calendar.create_event
data:
  summary: Rest day
  start_date: "2023-08-15"
  end_date: "2023-08-15"
target:
  entity_id: calendar.work_shift

Etcetera…

Every week I receive this overview and working and rest days are different also start and end times.
Als the first line change every week. Week number change also.
Can someone code this into automation? Im really grateful if this can be possible…

Cheers,
Peter

poudenes · August 12, 2023, 12:31pm

Asked ChatGTP to create something and gave me this:

Tried it. Didn’t work. If this was working only i needed to do was add the info from a sensor into the text part inside the script.

# configuration.yaml (of een vergelijkbaar bestand)
sensor:
  - platform: command_line
    name: planning_parser
    command: "python3 /config/parse_planning.py"  # Het pad naar het Python-script

# parse_planning.py (plaats dit script in de configuratiedirectory van Home Assistant)
import re
from datetime import datetime
from dateutil.parser import parse
import json

text = """
Donderdagse Week van 33-2023

390757 PM OUDENES

Datum    Dienst  Begin Einde Fun Stdp
________ ______  _____ _____ ___ ____
ma 14-08   6031  09:30 17:30 NRP NVT
di 15-08      R     
wo 16-08   6002  07:00 15:00 NRP NVT
do 17-08   6002  07:00 15:00 NRP NVT
vr 18-08  *  --     
za 19-08      R     
zo 20-08      R
"""

lines = text.split('\n')
data = []

for line in lines:
    if re.match(r'^[a-z]{2}\s\d{2}-\d{2}', line):  # Lijn bevat een datum en dienst
        parts = line.split()
        date_str = f"{datetime.now().year}-{parts[1]} {parts[0]}"
        date = parse(date_str)
        service = parts[2]
        begin_time = parts[3]
        end_time = parts[4]
        function = parts[5]
        data.append({
            "date": date.strftime("%Y-%m-%d"),
            "service": service,
            "begin_time": begin_time,
            "end_time": end_time,
            "function": function
        })

print(json.dumps(data))

Didgeridrew · August 12, 2023, 2:00pm

I haven’t fully tested this, but the template works and I have used a similar repeat for each to populate calendar event previously…

trigger:
  - platform: state
    entity_id: sensor.imap_full_info_sensor
    attribute: Message
condition:
  - "{{ trigger.to_state.state not in ['unknown', 'unavailable', '', 'None']}}"
action:
  - variables:
      message: "{{ trigger.to_state.attributes.Message }}"
      event_list: >-
        {%- set m_list = message.rsplit('\n')|map('trim')|reject('eq', '')
        | map('replace', '   ', ' ')|map('replace', '  ', ' ') 
        | select('match', '^[a-z]{2}\s\d{2}-\d{2}') | list %}
        {%- set rest_list = m_list | select('search', ' R')|list %}
        {%- set work_list = m_list | reject('search', ' R')|list %}
        {%- set ns = namespace(events=[] ) %}
        {%- for item in rest_list %}
          {%- set date_list = item.split(' ')[1].split('-') %}
          {%- set (month, day) = (date_list[1]|int, date_list[0]|int)%}
          {%- set date = today_at().replace(day=day, month=month) %}
          {%- set summary = "Rest" %}
          {%- set ns.events = ns.events + [{"summary": summary, "start": date|string, 
          "end": (date+timedelta(days=1))|string} ] %}
        {% endfor %}
        {%- for item in work_list if item.split(' ')[3] != '--' %}
          {%- set date_list = item.split(' ')[1].split('-') %}
          {%- set (month, day) = (date_list[1]|int, date_list[0]|int)%}
          {%- set date = month~'-'~day %}
          {%- set start = now().year~'-'~date ~' '~ item.split()[3]~':00'~now().strftime('%z') %}
          {%- set end = now().year~'-'~date ~' '~ item.split()[4]~':00'~now().strftime('%z') %}
          {%- set summary = item.split()[2] %}
          {%- set ns.events = ns.events + [{"summary": summary, "start": start, "end": end}] %}
        {% endfor %}
        {{ ns.events }}
  - repeat:
      for_each: "{{ event_list }}"
      sequence:
        - service: calendar.create_event
          data:
            summary: "{{ repeat.item.summary }}"
            start_date_time: "{{ repeat.item.start }}"
            end_date_time: "{{ repeat.item.end }}"
          target:
            entity_id: calendar.work_shift

poudenes · August 12, 2023, 2:04pm

wow. great. Thanks. Let me test this and let me study this script. Maybe i understand how it work. Ill let it know if this will work for me also.

poudenes · August 12, 2023, 2:26pm

Did the test with sensor itself and seems to work correct:

{% set message = state_attr('sensor.imap_full_info_sensor','Message') %}

        {%- set m_list = message.rsplit('\r')|map('trim')|reject('eq', '')
        | map('replace', '   ', ' ')|map('replace', '  ', ' ') 
        | select('match', '^[a-z]{2}\s\d{2}-\d{2}') | list %}
        {%- set rest_list = m_list | select('search', ' R')|list %}
        {%- set work_list = m_list | reject('search', ' R')|list %}
        {%- set ns = namespace(events=[] ) %}
        {%- for item in rest_list %}
          {%- set date = now().year~'-'~item.split(' ')[1] %}
          {%- set summary = "Rest" %}
          {% set ev_dic = dict.from_keys([("summary", summary), ("date", date)]) %}
          {%- set ns.events = ns.events + [ev_dic] %}
        {% endfor %}

        {%- for item in work_list if item.split(' ')[3] != '--' %}
          {%- set date = item.split(' ')[1] %}
          {%- set start = now().year~'-'~date ~' '~ item.split()[3] %}
          {%- set end = now().year~'-'~date ~' '~ item.split()[4] %}
          {%- set summary = item.split()[2] %}
          {% set ev_dic = dict.from_keys([("summary", summary), ("start", start), ("end", end)]) %}
          {%- set ns.events = ns.events + [ev_dic] %}
        {% endfor %}
        {{ ns.events }}

output:

[
  {
    "summary": "R",
    "date": "2023-15-08"
  },
  {
    "summary": "R",
    "date": "2023-19-08"
  },
  {
    "summary": "R",
    "date": "2023-20-08"
  },
  {
    "summary": "6066",
    "start": "2023-14-08 09:30:00",
    "end": "2023-14-08 17:30:00"
  },
  {
    "summary": "6002",
    "start": "2023-16-08 07:00:00",
    "end": "2023-16-08 15:00:00"
  },
  {
    "summary": "6002",
    "start": "2023-17-08 07:00:00",
    "end": "2023-17-08 15:00:00"
  }
]

I go implement this and give it some weeks. Every week it change so lets hope.

poudenes · August 12, 2023, 5:18pm

This error shows up.

2023-08-12 19:17:56.488 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Export DW to Calendar: Repeat at step 2: If at step 1: Error executing script. Invalid data for call_service at pos 1: Could not parse date for dictionary value @ data['start_date']
2023-08-12 19:17:56.491 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Export DW to Calendar: Repeat at step 2: Error executing script. Invalid data for if at pos 1: Could not parse date for dictionary value @ data['start_date']
2023-08-12 19:17:56.495 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Export DW to Calendar: Error executing script. Invalid data for repeat at pos 2: Could not parse date for dictionary value @ data['start_date']
2023-08-12 19:17:56.503 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Error while executing automation automation.export_dw_to_calendar: Could not parse date for dictionary value @ data['start_date']

Didgeridrew · August 12, 2023, 6:21pm

It seems to be an issue with the calendar.create_event service and date-only inputs… I altered the automation above to create and use date time strings for all the events.

poudenes · August 12, 2023, 6:33pm

Thank you for the help first!

A part is working:

output now is this:

[
  {
    "summary": "Rest",
    "start": "2023-08-15 00:00:00+02:00",
    "end": "2023-08-16 00:00:00+02:00"
  },
  {
    "summary": "Rest",
    "start": "2023-08-19 00:00:00+02:00",
    "end": "2023-08-20 00:00:00+02:00"
  },
  {
    "summary": "Rest",
    "start": "2023-08-20 00:00:00+02:00",
    "end": "2023-08-21 00:00:00+02:00"
  },
  {
    "summary": "6031",
    "start": "2023-14-08 09:30",
    "end": "2023-14-08 17:30"
  },
  {
    "summary": "6002",
    "start": "2023-16-08 07:00",
    "end": "2023-16-08 15:00"
  },
  {
    "summary": "6002",
    "start": "2023-17-08 07:00",
    "end": "2023-17-08 15:00"
  }
]

Only the Rest is added to calendar the rest isn’t. Can this because its not “2023-14-08 09:30:00” with :00 at the end?

2023-08-12 20:30:14.429 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Export DW to Calendar: Repeat at step 2: Error executing script. Invalid data for call_service at pos 1: not a valid value for dictionary value @ data['start_date_time']
2023-08-12 20:30:14.435 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Export DW to Calendar: Error executing script. Invalid data for repeat at pos 2: not a valid value for dictionary value @ data['start_date_time']
2023-08-12 20:30:14.442 ERROR (MainThread) [homeassistant.components.automation.export_dw_to_calendar] Error while executing automation automation.export_dw_to_calendar: not a valid value for dictionary value @ data['start_date_time']

entity_id: |-
                {% if repeat.item.summary == 'Rest' %} calendar.ha_holidays
                {% else %} calendar.work_shift {% endif %}

can be:

entity_id: calendar.work_shift

was a typo in the first post

Didgeridrew · August 12, 2023, 10:50pm

I think I found the issue, the “Rest” entries were using “YYYY-MM-DD” date format but the others were using “YYYY-DD-MM”… I’ve adjusted it above, and tested it on my instance and the calendar populates as expected. Hopefully your results are the same.

poudenes · August 13, 2023, 4:48am

Yes, that did the work. Thank you so much. For me a good reason to try to learn this myself. Can do a lot but if it comes to this kind of complex coding im lost… haha

stomko · October 6, 2024, 1:51pm

Sorry for hijacking this thread, but I posted here Parse data for waste collection and seems like this may be something similar, I just don’t know how to achieve it. I have an array of data on a web page, and I would like to create calendar events based on that list. @Didgeridrew perhaps you may be able to help?