This has come up a few times in the last couple of years in the context of Assist (without LLMs) and even then the answers have been hacky. LLMs complicate it even more.
I’m trying to figure out an efficient pattern for exposing input_select helpers to LLM-based Assist instances. The system isn’t exposing enough about them for the LLM to understand what to do with them. Neither the helper or it’s options have a description for the LLM to key off of, even if it was exposed, to have any context about what the options actually mean.
The pre-LLM work-around was adding the missing intents or sentences to enable interacting with them, but that isn’t sufficient for LLMs. Helper buttons that trigger the input_select to change is an option, but an overly complex one that has a lot of moving parts that have to be kept in sync. Scripts seem like they could be an option, maybe even being set up generic enough to be able to use a single input_select helper script, but scripts are so damn inefficient with how the LLM integrations work.
I’m curious if anyone has come up with anything clever for solving the problem.