# configuration.yaml
environment_variable:
OPENAI_BASE_URL: http://192.168.3.241:9997/v1
Modify URL, restart HA。Key can be freely filled in。
1 Like
Forget the visionary approach and just give me an LLM to write scripts and automations.
Existing LLM’s can already do this.
I have been experimenting with local LLMs controlling HomeAssistant for a while now (I’m the author of the blog post from January), and have noticed that RAG actually makes a massive impact with (relatively) little work. I blogged about it a little while back if you’re interested in the details.
The idea is, it’s unlikely that the user ever wants to take action on their entire house, but rather only some portion of their house (for example a room, a floor, or a specific entity). We can use RAG in its simplest form (retrieve pre-computed embeddings for each area/entity and get the top X results ranked by cosine similarity) and it will drastically cut down on processing times. I have been using this at my house and it made a big impact despite adding even more things to the LLM’s total context, while not causing any quality issues I could perceive.
This performance boost is because prefill cost (the processing required before anything is ever output from the LLM) increases exponentially based on the context, so reducing it has a non-linear effect on performance. This is notably the case without GPUs because as far as my experiments go, prefill is the worst part of CPU inference.
I have also included examples that are dynamically generated for in-context learning (based on the smart home state itself), which I found massively improves function calling accuracy while giving the LLM the ability to adapt to new HomeAssistant APIs that they are not trained for.
With the combination of all of this, I am actually able to use the whole pipeline locally with usable speeds (it’s not fast, but it’s no longer 8 seconds per query, at least if you have GPUs) locally.
3 Likes
Thats interesting! Cheers