📅 Calendar entry access for LLM (eg ChatGPT or Gemini)

[TheFes]

Introduction

Since some time Large Language Models like OpenAI/ChatGPT or Gemini can be used for voice commands, they can even control your house now. 2024.12 will even bring an option to prefer local processing of the command first, so the LLM will act as a fallback. This opens a whole new world of possibilities.

If you expose a script to Assist, a LLM can access and use that script. Using fields you can guide the LLM how to use the script. Using this I created a script which gathers calendar event data from one or more calendar entities (using the calendar.get_events action) and then feeds that data to the LLM, so it will give a nice summary. It will only send the events (ordered by time) to the LLM, so not the calendar it originates from.

How to use it

1. Use the button to import the blueprint
2. Create a script using the Blueprint, select the calendar entity or entities you want to use.
3. Save the script. IMPORTANT: The LLM will use the description of the script to determine how to use it. So it is important to provide a good description. I use Fetch calendar events from my calendar. In case the data for the weekend is requested, this means Saturday and Sunday
4. IMPORTANT: Expose the script to Assist

That’s it, you can now ask an LLM voice agent for calendar entries and it it will tell you what’s planned (provided the data is available in your calendar entity).

Thanks to:

@jenova70 for providing the groundwork for the script

[TheFes]

I removed the inititally added LLM instructions. If you get responses including * which are spoken out by the LLM, change your LLM configuration to exclude them.

[markmghali]

How do I change my LLM configuration to exclude * from being read? IT was not reading them at first but now it is saying asterisk asterisk in it’s response. Thank you for this

[TheFes]

@markmghali
Settings > Devices & Services > the >-icon on the tile for your LLM integration > CONFIGURE

[jirkabe]

Somehow I can’t find where and what exactly to set so that it doesn’t read ** ** Thanks

[TheFes]

Go here

Then find your LLM and press the >

After that press CONFIGURE

[IAmDotorg]

What’s an example command for getting this to trigger? Google AI and ChatGPT both don’t seem to trigger it, so I’m not sure if it’s not installed right or I just don’t get how to trigger it to activate. Obvious things like “What’s on my calendar today?” don’t do anything.

[TheFes]

That should do the trick, you did expose the script to Assist?