Local Calendar feature request 5 - Recognise overlapping events

Yes it is as now implemented, but if my feature request were implemented then anything starting after midnight would become current.

I admit an ambiguity if two events start at the same time (midnight or any other time). In that case I would want the shorter event (earliest end time) to take precendence. If two events have the same start and end time, then there is no clear precedence and that would have to be treated as an error, or one selected at random

I believe that’s how an All Day event is handled now. It starts at 00:00 and runs until the end of the day. It effectively monopolizes the “current” calendar event all day long.


All of the scheduling situations you described is why it’s not recommended to reference the “current” calendar event’s state and attributes.

Ostensibly, it’s supposed to display the current scheduled event but that only works for the most simplest of calendar schedules (no All Day event with other events, no overlapping events, no concurrent events, etc). A single Calendar entity can’t accurately report more complex scheduling permutations (worst case is two or more concurrent events with equal duration).

If your FR’s suggestions were implemented it would suit your needs but probably not someone else’s idea of what represents the “current” event in All Day/overlapping/concurrent events.

The simplest solution is to avoid referencing the “current” calendar event and use the Calendar Trigger and/or get_events service call as needed. That’s what the author recommends.


EDIT

FWIW, I also wish the “current” Calendar entity could handle the situations you described but to do so it would have to behave more like a list of calendar events. The list would contain all the calendar events that are currently in effect, be it an All Day event, overlapping events, or concurrent events.

Which is what you get from the calendar.get_events service call.

1 Like

Thanks for the further clarifications. I will try out the calendar.get_event option as per post 3/6.

EDIT: Yes, I get that the response data is a list, which I would have to sort and extract from. It does give me full control of which event I want to consider ‘current’, but is rather more complicated than just reading the calendar attributes.

Right, so I set up a test calendar with overlapping events and a trivial automation to call calendar.get_events. I copied the result from the trace and it looks like this

  calendar.zzz_test_calendar:
    events:
      - start: '2024-02-22T18:15:00+00:00'
        end: '2024-02-22T23:15:00+00:00'
        summary: 'Overlapping event 1 '
        description: 'Temp #21#'
      - start: '2024-02-22T18:30:00+00:00'
        end: '2024-02-22T18:45:00+00:00'
        summary: 'Overlapping event 2 '
        description: 'temp #22#'"

Now I need some tips on how to

  1. write a text-manipulation template that will convert this into a (jinja) list of events – I guess something like:

    [ {‘start’ : ‘2024-02-22T18:15:00+00:00’, ‘end’ : ‘2024-02-22T23:15:00+00:00’, ‘summary’ : ‘Overlapping event 1’, ‘description’ : ‘Temp #21#’ }, { ‘start’: ‘2024-02-22T18:30:00+00:00’, ‘end’ : ‘2024-02-22T18:45:00+00:00’, ‘summary’ : ‘Overlapping event 2’, ‘description’ : ‘temp #22#’ } ]

  2. test if there are any entries in the list (count > 0)
  3. sort the list with the latest start time first
  4. extract this first list item (event)
  5. use the attributes of this list item (event)

Your assistance would be much appreciated :pray:

Look at the example in the documentation for calendar.get_events. The example uses a response_variable names agenda. That’s what you use to:

  • Determine if agenda['calendar.zzz_test_calendar'].events contains anything.
  • Sort the events list by start or end (in increasing or decreasing order).
  • Get any item in the events list by its index value.
  • Access start/end/summary/description of any item in the list.

The only reason to do that is if you want some test data for use in the Template Editor in order to experiment with it. I use an online YAML to JSON converter for that purpose (there are several to choose from).

For example, I used YAML to JSON Converter Online to convert your sample YAML data to JSON.

{
    "calendar.zzz_test_calendar": {
        "events": [
            {
                "start": "2024-02-22T18:15:00+00:00",
                "end": "2024-02-22T23:15:00+00:00",
                "summary": "Overlapping event 1 ",
                "description": "Temp #21#"
            },
            {
                "start": "2024-02-22T18:30:00+00:00",
                "end": "2024-02-22T18:45:00+00:00",
                "summary": "Overlapping event 2 ",
                "description": "temp #22#"
            }
        ]
    }
}

I can now use it in the Template Editor by assigning it to a variable named agenda (can be whatever name you want).

{% set agenda = 
{
    "calendar.zzz_test_calendar": {
        "events": [
            {
                "start": "2024-02-22T18:15:00+00:00",
                "end": "2024-02-22T23:15:00+00:00",
                "summary": "Overlapping event 1 ",
                "description": "Temp #21#"
            },
            {
                "start": "2024-02-22T18:30:00+00:00",
                "end": "2024-02-22T18:45:00+00:00",
                "summary": "Overlapping event 2 ",
                "description": "temp #22#"
            }
        ]
    }
}
%}

Now I can experiment with it:


{% set events = agenda['calendar.zzz_test_calendar'].events %}
{{ events | count > 0 }}

{% set reversed = events | sort(attribute ='start', reverse=true) | list %}
First: {{ reversed[0] }}

Last: {{ reversed[-1] }}

{% set first_event = reversed[0] %}
{{ first_event.start }}
{{ first_event.summary }

Output in Template Editor:

1 Like

Since this is for a blueprint I was thinking it might need to be able to handle multiple calendar inputs.

For testing, I set it up as a template sensor
template:
  - trigger:
      - platform: calendar
        event: start
        entity_id: calendar.a
      - platform: calendar
        event: end
        entity_id: calendar.a
      - platform: calendar
        event: start
        entity_id: calendar.b
      - platform: calendar
        event: end
        entity_id: calendar.b
    action:
      - service: calendar.get_events
        data:
          start_date_time: "{{ now() }}"
          end_date_time: "{{ today_at() + timedelta(days=1) }}"
        response_variable: cal_events
        target:
          entity_id:
            - calendar.a
            - calendar.b
      - variables:
          most_current: |
            {%- set fms = namespace(event=[]) %}
            {%- for key, value in cal_events.items() %}
              {%- for event in value.events %}
                {%- set fms.event = (fms.event + [event]) %}
              {%- endfor %}
            {%- endfor %}
            {% set x = (fms.event + [{'summary': 'now','start': now().isoformat()}]) | sort(attribute='start') %}
            {% set ns = namespace(ind='') %}
            {% for ev in x %}
              {% if ev.get('summary') == 'now' %}
                {% set ns.ind = loop.index0 %}
              {% else %}
                {% continue %}
              {% endif %}
            {% endfor %}
            {{ x[ns.ind + (-1 if ns.ind > 0 else 1)] }}
    sensor:
      - name: Current Calendar Event
        unique_id: current_calendar_event_0001
        state: "{{most_current.get('summary')}}"
        attributes:
          description: "{{ most_current.get('description')|default('')}}"
          start: "{{ most_current.get('start')}}"
          end: "{{ most_current.get('end')}}"

EDIT: Playing around with this, I realized that the second loop can be replaced with a simpler reject filter and if/then statement:

{# Combine events from calendar.get_events response from 1 or more calendars into one list #}
{%- set fms = namespace(event=[]) %}
{%- for key, value in cal_events.items() %}
  {%- for event in value.events %}
    {%- set fms.event = (fms.event + [event]) %}
  {%- endfor %}
{%- endfor %}

{# sort and select for events that have started #}            
{% set begun = fms.event | sort(attribute='start')| rejectattr('start', 'gt', now().isoformat()) | list %}
{%- if begun|count > 0 %}
  {# return event that started most recently #}
  {{- begun | last }}
{%- elif fms.event | count > 0 %}
  {# return next event if no event is currently active #}
  {{- fms.event | first }}
{%- else %}
  none
{%- endif %}
1 Like

No, I ultimately want to process the response data within the blueprint (YAML) to extract the relevant event information. Conversion to JSON ia an interesting way to go, but is there a way of doing that within the blueprint?

The documentation only gives two examples, and the first just outputs the response data unprocessed.

The second example has

    {% for event in agenda["calendar.school_calendar"]["events"] %}
    {{ event.start}}: {{ event.summary }}<br>
    {% endfor %}

… which implies that the events can be extracted without recourse to JSON conversion, as does your template sensor example. I will use it within an automation blueprint, but should be able to use code similar to that of your most_current variable.

I am not trying to do anything extraordinary! I just need a leg-up because my template text string manipulation skills are primitive. I therefore need to be able to test code first in the template editor, starting with my example of response data.

So I tried this in the template editor

{% set cal_events = "
  calendar.zzz_test_calendar:
    events:
      - start: '2024-02-22T18:15:00+00:00'
        end: '2024-02-22T23:15:00+00:00'
        summary: 'Overlapping event 1 '
        description: 'Temp #21#'
      - start: '2024-02-22T18:30:00+00:00'
        end: '2024-02-22T18:45:00+00:00'
        summary: 'Overlapping event 2 '
        description: 'temp #22#'"
%}

{%- set fms = namespace(event=[]) %}
{%- for key, value in cal_events.items() %}
  {%- for event in value.events %}
    {%- set fms.event = (fms.event + [event]) %}
  {%- endfor %}
{%- endfor %}
{% set x = (fms.event + [{'summary': 'now','start': now().isoformat()}]) | sort(attribute='start') %}
{% set ns = namespace(ind='') %}
{% for ev in x %}
  {% if ev.get('summary') == 'now' %}
    {% set ns.ind = loop.index0 %}
  {% else %}
    {% continue %}
  {% endif %}
{% endfor %}
{{ x[ns.ind + (-1 if ns.ind > 0 else 1)] }}

The output is

‘str object’ has no attribute ‘items’

I get a similar response if I try to access the calendar attributes, so I guess copying and pasting the sample text from an automation trace has somehow altered it? What should it look like to simulate response data?

If I use your JSON as the starting point, then I do get a sensible answer.

{% set cal_events = 
{
    "calendar.zzz_test_calendar": {
        "events": [
            {
                "start": "2024-02-22T18:15:00+00:00",
                "end": "2024-02-22T23:15:00+00:00",
                "summary": "Overlapping event 1 ",
                "description": "Temp #21#"
            },
            {
                "start": "2024-02-22T18:30:00+00:00",
                "end": "2024-02-22T18:45:00+00:00",
                "summary": "Overlapping event 2 ",
                "description": "temp #22#"
            }
        ]
    }
}
%}

{%- set fms = namespace(event=[]) %}
{%- for key, value in cal_events.items() %}
  {%- for event in value.events %}
    {%- set fms.event = (fms.event + [event]) %}
  {%- endfor %}
{%- endfor %}
{% set x = (fms.event + [{'summary': 'now','start': now().isoformat()}]) | sort(attribute='start') %}
{% set ns = namespace(ind='') %}
{% for ev in x %}
  {% if ev.get('summary') == 'now' %}
    {% set ns.ind = loop.index0 %}
  {% else %}
    {% continue %}
  {% endif %}
{% endfor %}
{{ x[ns.ind + (-1 if ns.ind > 0 else 1)] }}

Output:

Result type: dict

{
  "start": "2024-02-22T18:30:00+00:00",
  "end": "2024-02-22T18:45:00+00:00",
  "summary": "Overlapping event 2 ",
  "description": "temp #22#"
}

So is the response data actually in JSON format, and it is the trace program that is converting it to YAML??? If the answer to that question is yes, then I am on the right track and just need a primer on extracting the attributes from a dict.

Your latest set of assertions and questions suggest to me that you haven’t completely grasped the concepts and examples in my previous post. I recommend that you review what I wrote because the questions you asked were already answered.

One thing you need to clarify in your mind is that the Template Editor only understands Jinja. That’s why you have to define a dictionary using Jinja in the Template Editor. In contrast, Home Assistant can also handle it in YAML when processing variables in automations, scripts, etc.

My previous example shows that.

That’s why I had suggested to review it.

1 Like

Apparently not, but not for lack of trying, I assure you!

I can confirm that your example works in the template editor, with a couple of minor tweaks

{% set cal_events = 
{
    "calendar.zzz_test_calendar": {
        "events": [
            {
                "start": "2024-02-22T18:15:00+00:00",
                "end": "2024-02-22T23:15:00+00:00",
                "summary": "Overlapping event 1 ",
                "description": "Temp #21#"
            },
            {
                "start": "2024-02-22T18:35:00+00:00",
                "end": "2024-02-22T18:40:00+00:00",
                "summary": "Overlapping event 3 ",
                "description": "temp #23#"
            },
            {
                "start": "2024-02-22T18:30:00+00:00",
                "end": "2024-02-22T18:45:00+00:00",
                "summary": "Overlapping event 2 ",
                "description": "temp #22#"
            }
        ]
    }
}
%}

{% set events = cal_events['calendar.zzz_test_calendar'].events %}
{{"Any events at all = " }} {{ events | count > 0 }}

{% set reversed = events | sort(attribute ='start', reverse=true) | list %}  
{{ "First event = " + reversed[-1] | string }}

{{ "Last event  = " + reversed[0] | string }}

{% set last_event = reversed[0] %}
{{ "Last event start = " + last_event.start }}
{{ "Last event name  = " + last_event.summary }}

Output:

Result type: string
Any events at all =  True

  
First event = {'start': '2024-02-22T18:15:00+00:00', 'end': '2024-02-22T23:15:00+00:00', 'summary': 'Overlapping event 1 ', 'description': 'Temp #21#'}

Last event  = {'start': '2024-02-22T18:35:00+00:00', 'end': '2024-02-22T18:40:00+00:00', 'summary': 'Overlapping event 3 ', 'description': 'temp #23#'}


Last event start = 2024-02-22T18:35:00+00:00
Last event name  = Overlapping event 3

And I like the elegance of your method for finding the last event.

I was rather confused by your digression into an external YAML to JSON converter. Ithink I misled you by taking the data format provided by the automation trace? I now suspect that that is modified.

The first and fundamental question is what format exactly does the response data from calendar.get_events have? I do not see that in the documentation. Is is YAML, JSON or something else? It is hard to test because it will not fit in an input_text helper. How can I make it visible?

If it is already in JSON, like your example template, then the rest follows. Yours and Drew’s examples seem to suggest that this is the case but I have not seen a clear an unambiguous statement that it is.

If yes, then (as I said in my previous post) the rest is doable and just a matter of my understanding how filters work in templates (JINJA).

Sorry if I am slow on the uptake, but I very much appreciate the help both you and Drew have given.

A digression? No it was necessary to explain why you must define the agenda dictionary from scratch in JSON format if you intend to use the Template Editor for testing purposes.

My understanding is that it’s “something else”, namely a python dictionary (dict). JSON and YAML are simply two means of serializing the python dictionary (i.e. putting it in a format that human beings can read and edit).

This is a dictionary defined in YAML:

widgets:
  blue: 5
  red: 2
  green: 7

The same dictionary in JSON:

{
    "widgets": {
        "blue": 5,
        "red": 2,
        "green": 7
    }
}

They’re just two ways to define the same thing. Where you can use them depends on the context.

The contents of the agenda variable is a python dictionary. But you can’t directly access the agenda variable in the Template Editor. So, for testing purposes, you need to define it in the Template Editor from scratch and it has to be in JSON format because the Template Editor doesn’t process YAML.

1 Like

Great! So that explains why the two example above work :slight_smile:

Now, in the automation itself do I need to do some transformation before I can use the response data in a template? Drew’s example and the second example in the documentation suggest not?

I took the second documentation example and combined it with my data, giving a corect result as follows:

{% set cal_events = 
{
    "calendar.zzz_test_calendar": {
        "events": [
            {
                "start": "2024-02-22T18:15:00+00:00",
                "end": "2024-02-22T23:15:00+00:00",
                "summary": "Overlapping event 1 ",
                "description": "Temp #21#"
            },
            {
                "start": "2024-02-22T18:35:00+00:00",
                "end": "2024-02-22T18:40:00+00:00",
                "summary": "Overlapping event 3 ",
                "description": "temp #23#"
            },
            {
                "start": "2024-02-22T18:30:00+00:00",
                "end": "2024-02-22T18:45:00+00:00",
                "summary": "Overlapping event 2 ",
                "description": "temp #22#"
            }
        ]
    }
}
%}

{% for event in cal_events["calendar.zzz_test_calendar"]["events"] %}
{{ event.start}}: {{ event.summary }}
{% endfor %}

Result:

2024-02-22T18:15:00+00:00: Overlapping event 1 
2024-02-22T18:35:00+00:00: Overlapping event 3 
2024-02-22T18:30:00+00:00: Overlapping event 2

I do not fully understand the detail, but with these examples it seem I have all the pieces!
Thanks again for your persistence

After everything that’s been explained, what still leads you to believe some form of transformation is needed?

When calendar.get_events is called within an automation, script, or Trigger-based Template Sensor, the resulting response_variable contains a python dictionary. It’s readily accessible to a template (within the automation, script, or Trigger-based Template Sensor) without “transformation”.

1 Like

It is now clear that I do not.

The issue of transformation came up first with your assertion that YAML needed to be converted to JSON (now seen as as red herring).

we also have

And

Actually it came up first when you wrote this:

image

My response was that it was not necessary unless the intention was to use the Template Editor for testing purposes. Then you would need to define the contents of the agenda variable in JSON format. Given that you already had an example of its contents in YAML format, I suggested using an online conversion tool for convenience.

1 Like

I hope the tutorial on python/YAML/JSON has been useful but I hope was has been most useful is what I explained very early in this topic:

Use Calendar Trigger and calendar.get_events to ensure proper handling of All Day/overlapping/concurrent events, because the “current” calendar event can never properly handle multiple events.

1 Like

Yes, many thanks again.
I am now working on a test automation to show that I can correctly extract the last event in that context, then I will integrate it into the blueprint. That will take me a couple of days but I will report back when I get it working :slight_smile:

Couple of days? Your tutoring must have been good because I did it in a couple of hours!

The relevant code is the following fragment, which appears at the start of the automation, so runs whatever the trigger is (start_event, end_event and several others).

variables:
  local_room_calendar: !input room_calendar 

action:
## -------------------------------------------------------------------------------------------------
## ACTIONS[0-2] -- FETCH THE CURRENT AND NEXT EVENTS FROM THE CALENDAR
## Current event is the one that started last
## -------------------------------------------------------------------------------------------------
  - service: calendar.get_events
    target:
      entity_id: !input room_calendar
    data:
      duration:
        hours: 0
        minutes: 0
        seconds: 1
    response_variable: active_events

  - service: calendar.get_events
    target:
      entity_id: !input room_calendar
    data:
      duration:
        hours: 24
        minutes: 0
        seconds: 0
    response_variable: future_events

  - variables:
      any_event_active: >
        {{ active_events[local_room_calendar].events | count > 0 }}
      current_event: >
        {{ active_events[local_room_calendar].events | sort(attribute ='start') | list | last | default("") }}
      any_future_event: >
        {{ future_events[local_room_calendar].events | count > 0 }}
      next_event: >
        {{ future_events[local_room_calendar].events | sort(attribute ='start') | list | first | default ("") }} 


The rest of the automation uses the four variables I set.

  • any_event_active and any_future_event are booleans used in condition templates.

Other templates refer to

current_event.summary
current_event.start
current_event.end
current_event.description

… and similarly for next_event

I have the required temperature in the description between two hashes, so that is extracted as

current_event.description.split('#')[1]

UPDATE 25-Feb: the code above is streamlined to one template line per variable, and a default is added to stop warnings in the error log file when there are no events.

For the full code see Heating X2 blueprint

Nice!

It’s the custom of this community to assign the Solution tag to the first post that answers/solves the original question/problem.

Despite first having to be convinced your FR wasn’t necessary and was already achievable, receiving numerous detailed explanations on how to do it, including a tutorial on how to handle a response_variable, you ultimately chose to mark your own post as the Solution.

You may wish to familiarize yourself with guideline 21 in the FAQ.

The guidance says
Continuing the discussion from How to help us help you - or How to ask a good question:

I solved the overlapping events problem with a lot of help from you, a little from Drew, and some from Allenporter on what tuned out to be a related issue on Githubi. The final code took a fair bit of experimentation and further research once you had set me on the right track. I found List of Builtin Filters particularly useful.

I transferred the code to my context (in an automation, using local variables), streamlined the code (using first and last filters), used two calendar.get_events calls for current and future events, and added the default filter to eliminate warnings in the log file when there are no events.

Your help was invaluable and I am happy to acknowledge that formally if there is a means of doing so beyond ‘liking’ your postings. For a reader of the thread, however, the actual solution to the problem is in the posting that I indicated.

I can only tick one posting as the ‘solution’. Which one do you propose?

BTW I am not convinced that my FR wasn’t necessary. It was made in good faith based on the existing documentation and would still be a much simpler solution if adopted.

This one:

Because it spells out what is the recommended way of currently achieving the FR’s goal.

The majority of the topic is me tutoring you on how to implement it, including fundamentals like how a dictionary can be represented and how to access the keys and values of the dictionary within a response_variable.