💙 Blueprints for Voice Commands (Weather, Calendar, Music Assistant)

Blueprints Exchange
, , , ,
1

Version

Buy me a coffe

Paypal

Home Assistant Voice Blueprints

Image
Image2200×440 136 KB

:construction: Under construction

The blueprints are still under construction and might change quite frequently. I’ll try not to introduce any backwards incompatible changes.

:blue_heart: About these blueprints

Since I received my Voice PE units, I’m a happy user of Assist in my house. The recently added LLM fallback functionality also gave a lot more functionality to voice in Home Assistant.

However, there is still some functionality lacking. Out of the box you can’t ask for things like weather, calendar entries because this information can only be retrieved with actions and action responses.

To bridge this gap, I created a repository with voice blueprints

On this respository I will put my blueprints which are created to help issuing advanced (voice) commands using Assist.

:loudspeaker: Voice configuration

There are basically 5 ways to configure a voice pipeline in Home Assistant:

# LLM Home Control (Assist) enabled Local preference enabled Description
1 :x: NA NA Local only
2 :white_check_mark: :x: :x: LLM as voice agent, no home control, no local preference
3 :white_check_mark: :x: :white_check_mark: LLM as voice agent, no home control, with local preference
4 :white_check_mark: :white_check_mark: :x: LLM as voice agent, with home control, no local preference
5 :white_check_mark: :white_check_mark: :white_check_mark: LLM as voice agent, with home control, with local preference

On this repository different blueprints will be placed with the same goal. The will differ in which op the above mentioned configurations are supported. Based on that, also other aspects will differ.

:wrench: Blueprint options

There are 3 options for the blueprints

  • Option 1: Local automation

  • Option 2: LLM Enhanced automation

  • Option 3: Full LLM script

Description Option 1 Option 2 Option 3
Supported voice configurations 1, 2, 4 1, 3, 5 3, 4, 5
Fully local, no LLM required :white_check_mark: :x: :x:
No home control for LLM required :white_check_mark: :white_check_mark: :x:
No strict voice commands :x: :white_check_mark: :white_check_mark:
Complete freedom in voice commands :x: :x: :white_check_mark:
Variance in responses for each request :x: :white_check_mark: :white_check_mark:
Works without need for translations :x: :x: :white_check_mark:

The groundwork for option 2 was done by JLo in his blog post on GPT-powered music search. This blog post was a big inspiration for the LLM enhanced automation, and also enabled the full LLM script. So big thanks to JLo!

The Blueprints

:sun_behind_rain_cloud: Weather Forecasts

Using these blueprints you can ask for the weather forecast from a weather entity in your system.

Option Import Button
1: Local Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.
2: LLM Enhanced To be created
3: Full LLM Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

:date: Calendar

Using these blueprints you can ask for the calendar entries from one or more calendar entities in your system.

Option Import Button
1: Local Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.
2: LLM Enhanced To be created
3: Full LLM Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

:white_check_mark: ToDo

Using these blueprints you can ask for the todo list items from the todo entities in your system.

Option Import Button
1: Local Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.
2: LLM Enhanced To be created
3: Full LLM To be created

:notes: Music Assistant

For completeness I also mention these blueprints here. They are located on the Music Assistant Voice support repository.

I’m also a maintainer of that repository and did a lot of work on those blueprints together with some other guys.

Using these bleuprints you can issue voice commands to play media on your Music Assistant media players.

Option Import Button
1: Local Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.
2: LLM Enhanced Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.
3: Full LLM Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

:memo: Planned

  1. Add missing options for weather and calendar

  2. Add blueprints to create reminders

  3. Add blueprints to add and list todo items, including due dates/times

  4. …

:question: Questions/issues/bugs/feature requests?

In case you have a question, you found a bug, or have a feature request open an issue on the GitHub repository.

In case something isn’t working or you found a bug, a trace of the script will be needed in most cases to determine the cause. The trace can be downloaded as a json file. To do this follow the steps below:

  1. Depending on the blueprint type go to Open your Home Assistant instance and show your scripts. or Open your Home Assistant instance and show your scripts.

  2. Find the relevant script or automation in the list.

  3. Click on 3 dot menu icon left of the script and then select Traces

  4. Make sure to select the right trace which contains the error

  5. Press the 3 dot menu icon in the top right corner, and select Download trace

  6. Press the download icon in the top right corner.

In case you create the issue on GitHub you can upload the json files, in case you create the issue as a post here, you can copy the json files to a code sharing website like dpaste.org. Please create a different link for each json file.

:coffee: Coffee

If you think I deserve a coffe, please feel free to buy me one (I might spend it on another beverage though).

In case you decide to do so, thanks a lot!

Buy Me A Coffee

Or you can do a small donation using PayPal.

paypal

6 Likes
2

Updates

20250130

Blueprints

:white_check_mark: ToDo

  • Local Assist blueprint added to list todo items

20250129.1

Blueprints

:date: Calendar

  • Local Assist blueprint added to retrieve calendar entries
3

I’m looking forward to giving these a try, thanks!

I’ve been adding items to my todo lists by voice for a while without any need for additional intent sentences etc. Never tried anything more than that though. I did just try getting Voice to list my items and that failed miserably… Local only just said I had no area called “supermarket list” and using AI backed Voice it ended up adding 3 random items to my supermarket list… (eggs, milk and bread)

4

I have something working on my system, but only for a fixed list for now.
I’m not sure if I can make something working locally as a blueprint, as it seemed to conflict with other built in intents. Or I should really make a really strict sentence like what is on the list named {list_name}

For the LLM variant it will be quite easy to add.

There was also a specific request in one of the other former topics to add support for due dates and due times, which isn’t supported in the built in intent

5

Update

Blueprints

Todo

  • Add local Assist blueprint to list todo items
6

you can try out the newly added blueprint.

LLM enhanced will come when I have time. Then you can also ask stuff like do I have carrots on the shopping list

1 Like