đź“… Calendar entry access for LLM (eg ChatGPT or Gemini)

How do I change my LLM configuration to exclude * from being read? IT was not reading them at first but now it is saying asterisk asterisk in it’s response. Thank you for this

@markmghali
Settings > Devices & Services > the >-icon on the tile for your LLM integration > CONFIGURE

Somehow I can’t find where and what exactly to set so that it doesn’t read ** ** Thanks

Go here
Open your Home Assistant instance and show your integrations.

Then find your LLM and press the >

After that press CONFIGURE

What’s an example command for getting this to trigger? Google AI and ChatGPT both don’t seem to trigger it, so I’m not sure if it’s not installed right or I just don’t get how to trigger it to activate. Obvious things like “What’s on my calendar today?” don’t do anything.

That should do the trick, you did expose the script to Assist?

This is great - I’ve been trying to get Gemini to access my calender, but couldn’t quite get it to work right. This blueprint did the trick :blush:

Ok, well this is weird. This script generated from the blueprint worked fine, but, without me changing anything, now I’ve gone back to getting “I can’t access your calender” response variations, and I can’t quite figure out why :thinking:

EDIT: nevermind, appears to have been a glitch with Gemini, it sorted itself out.

1 Like

Spoke too soon. It works for a bit and then stops, reverted back to “I cannot access your calendar information”. Quite annoying!

It works fine if I ask what I have today. If I ask what I have on Wednesday, it gives me the events for Friday.

@wpigulce I’ve seen that with the weather forecast to. It’s not the script, but the LLM. If you ask it for Wednesdays date, it will return the date for Friday. You can try this

OK I applied it and now I have Friday January 30th and Saturday February 1st

@TheFes - any ideas what could be causing the issues for me? Used your blueprint, worked fine for a bit then decides it no longer has access to my calendars.

It actually feels like the the voice assistant is defaulting to local and not actually using Gemini.

But then I ask for the weather forecast (using your weather blueprint), and it works fine, so I’m confused. I’ve deleted the script, and configured it again, tried a million different ways of asking for the information, but it just won’t give me anything any more.

It was working when I went to bed last night and not working this morning, despite exactly nothing changing in between :thinking:

Which LLM are you using? It could possibly help if you tell if to use this “tool” when requesting calendar events

Using Google Gemini. I’ve tried telling it to use the script, the helper script, the full name of the script, same response. When it worked yesterday, I just asked Jarvis to, for example, “tell me what’s on my calendar for Tuesday 28th”, and it checked my calendar and told me.

I had thought it was maybe an issue with Google calendar authentication, but I’ve just added a test entry via Google, reloaded the Google Calendar integration in HA, and the new event shows up just fine, in both the Google calendar integration, and the Home Assistant Calendar.

“Use the LLM Calendar script to fetch my calender events for XXXX” got it working again. Eventually.

After that I can ask normally, for example, “What’s on the calendar for 7 February”, or “are there any hospitals appointments in my calendar for the next two weeks”.

However, if it then gets a day wrong, and I ask it to check and confirm more than once, it then takes a sulk and tells me because it’s not been accurate it can no longer recover data from my calendars. I then have to rerun the original "Use the LLM Calendar script…"command to get it running again!

I’m fairly certain this is a Gemini issue though. AI is still pretty dumb atm.

Forgive me if this has been covered, but I haven’t had much luck: My LLM is Extended OpenAI Conversation. I’ve set up the script just how you’ve described (with the same description) and everything works great with the original OpenAI Conversation, but switching over to the one linked prior results in the assistant replying with “Your calendar is currently off, so there are no scheduled events for today.”

Any chance I’m missing something? The reason for switching to Extended is due to its ability to interact with date/time fields, sliders, and etc. while the original one cannot. If I’m wrong for switching over, please let me know too!

Thanks!

I have no experience with that integration, but it sounds like it can’t use scripts as tools as the core integrations can. It might be best to ask on their Github about it.

Update

I created a new post for all my voice related blueprints

Updates will be placed there

You will need to reimport the blueprint and create a new script in order to get updates.

Understood, thank you! Still deciding if it’s maybe worth getting back over onto the default OpenAI Conversation. Just figuring out how to fill in those missing features.

Otherwise, I appreciate this guide! And by the way, your recent post has alrady been a big help :slight_smile: