I found adding this to the Prompt section greatly improved weather reporting.
For weather information, be sure to check the weather.munro_home entity.
For example, prior to adding that it would often say it didn’t have access to weather information for next week. I’d then say “how about looking at the Munro home weather entity” and it would apologise for missing it and then give me the forecast.
Maybe try suggesting something in the prompt so you don’t have to guide it each time it misses the sensor.
Thanks for the tip however for me adding the get_attributes function as documented on the (recently updated) github readme does the trick for the weather.
It also has the advantage of being a very generic function that can serve many requests so more possibilities for the same number of input api tokens.
That was in reply to the query from @yahav. get_attributes worked for me too, but about 30% of the time it would tell me that it didn’t have access to the data it actually did have access to. In other words it wasn’t 100% reliable.
The weather was just my example, but I think it would apply to a boiler or other devices that were not being accessed reliably, probably due to naming. Adding a guideline in the prompt made it 100% reliable for me.
Did anyone of you guys got a working setup with a locally hosted ai? I’m trying different models with LM Studio at the moment but can’t get it working right. The responses sometimes are completely off or change when asked a second time. So I’m wondering if anyone got tips in that regard.
Because I’m not quite sure if this is “ai model” related or misconfiguration.
I tried with LMStudio but I have never been able to execute a script. The LLM was able to give me the status of my lights, switches, etc. but was not able to activate them (it says me, ok, I’m doing it but nothing ever happened).
Jekalmin told me it was perhaps because LMStudio wasn’t able (yet) to use function calling.
Whan I started using LMStudio, I also had various answers (temps wrong, strange status, etc.). The reason was beacause I had expopse too many entities in HA. After reducing my exposed entities to the necessary ones (and not my whole entities…) the LLM was able to understand them and is now able to give me the good answers.
Now, I’m trying to make a config with a server of mine with Proxmox, in order to make a VM which could use function calling (Proxmox will be able to use my GPU in VMs, wheras Hyper-V is not).
I stopped trying to use LMStudio for that (even if it’s very easy to use and cool).
thanks for reply. I also been wondering if reducing the exposed entities maybe help with the random answers, will definitely try this.
I also got a Proxmox Server running and was playing around with a dedicated VM with GPU, but haven’t been successful. Tried LocalAI and Ollama, but had different problems. For now I’m trying Ollama but just can’t get the API working for external devices, even tough I changed the interface to 0.0.0.0.
Maybe you can share your experiences when you tried a little bit in your proxmox vm setup.
I think this is because you’re passing so many exposed devices that the prompt is too long. You can test this by modifying the ‘Prompt Template’ in the configuration for extended_openai_conversation to remove the list of devices.
Current Time: {{now()}}
Available Devices:
```csv
entity_id,name,state,aliases
I believe that the entire prompt is sent each time you make a request, which is why the token length for each request can be very long if you have lots of exposed devices.
Is there any way to provide a more human-readable value for entities that Assist can use?
As an example, I have a sensor for my cats location where one of the states can be “Away (GPS)” but i’d like the Assist response to say “Away, connected via GPS”.
Should I just create a separate sensor for Assist and expose that instead?
My solution to this is to add an attribute to the existing sensor called conversation_agent_state and then use that attribute as the state in the OpenAI prompt:
Available Devices:
```csv
entity_id,name,state,aliases
{% for entity in exposed_entities -%}
{% if state_attr(entity.entity_id,'conversation_agent_state') == None -%}
{% set state = entity.state -%}
{% else -%}
{% set state = state_attr(entity.entity_id,'conversation_agent_state') -%}
{% endif -%}
{{ entity.entity_id }},{{ entity.name }},{{ state }},{{entity.aliases | join('/')}}
{% endfor -%}
This allows me to tweak the state specifically for the conversation agent without affecting the existing sensor.
Elegant solution.
Just out of curiosity, when you set state again with the conversation_agent_state attribute, does it erase the value of state for the entities that do not have this atttibute? Or does it only replace the state for the ones that have the attribute and leave the others as is?
I am trying to make the integration to answer this prompt:
“How many hours is the AC Fan turned on in the last 30 days?”
If I try “What is the AC Fan state now”
I get an answer of: "The current state of the AC Fan is idle." which tells me ChatGPT can see the “AC Fan” state (as I created an alias and exposed it).
If I try “How many hours is the AC Fan turned on in the last 30 days?” I am getting:
“I’m sorry, but I don’t have access to the historical data of the AC Fan usage. Is there anything else I can help you with?”
I did add this query_histories_from_db that someone shared before but still, ChatGPT can’t see the historic info.
**Any help with this? Or maybe some suggestion on how to debug it?
Thank you VERY Much !**
- spec:
name: query_histories_from_db
description: Use this function to query histories from Home Assistant SQLite database.
parameters:
type: object
properties:
query:
type: string
description: A fully formed SQL query.
function:
type: sqlite
query: >-
{%- if is_exposed_entity_in_query(query) -%}
{{ query }}
{%- else -%}
{{ raise("entity_id should be exposed.") }}
{%- endif -%}
Absolutely amazing component @jekalmin - having a lot of fun playing with this, great work!
It seems possible to set up a lot of interesting behavior by providing the OpenAI models with the ability to call functions as you have created here: native, rest, template, scrape, composite and sqlite. For example, one thing I am trying to do is chain together some stateful API calls - for Spotify, for example, it will require an authentication token to be used when searching for things, so first we make a POST request for a temporary token, then we attach that token to some GET requests against their api for things like searching for artists, songs, albums, playlists. I am thinking that if I create one function like “get token” which does a POST against the right API endpoint (provided a user’s authentication credentials) and then a separate function for search, and provide the token as required input to the search, then it will be “smart” enough to figure out what it needs to do. Only one problem - I am not sure how to configure the rest function to do things other than GET, is this possible? I am imagining it like this:
I cannot really help you with your specific request as it is a bit above my paygrade.
However, if what you are trying to achieve is searching and playing spotify from a voice query, I found an easy way to do just that.
First, install the “spotcast” custom component, this will store your spotify credentials so no need to supply them to gpt and it also integrates a search function.
Now you just need to add a function call like the one below. I usually just say “play X” and it reliably does.
Cheers @Rafaille I’ll give that a shot. I know also people are doing this with Music Assistant as well. I am curious for the answer to my question in case it can be used for other things than just Spotify, but really playing music is the thing I am after right now
I forgot to mention, here spotcast is setup to cast on a chromecast compatible device. If you would rather send to the native spotify app, use the spotify connect “device id”, see spotcast documentation.
If gpt doesn’t generate a sql query accordingly, we have to manually write a query to get it work.
“get_total_time_of_entity_state” function’s long query is work done to answer your question.
