I hope I’m not sparking a holy war here, but I’d love to get some advice on the right setup for something I’m trying to do.
I’d like to use an LLM to regularly analyze my Home Assistant device states and history. The goal is to regularly alert me to anything concerning/out of the ordinary, as well as to explore trends. (E.g., is something using more power than normal? Are air quality measurements chronically high at certain times of day in certain rooms, etc.). I have no illusions that LLMs are brilliant oracles (nor that they’ll catch everything that’s important), but I’ve used them enough to believe that they may spot interesting things that I miss. However, I absolutely do not want to send that type of information to Claude/Gemini/ChatGPT/whatever. So I’m left needing to do something local.
After a bit of poking around, my preliminary impressions are:
- the right approach would be to set up a separate local computer running Ollama and somehow have HA basically offboard the LLM analysis to that computer.
- HA histories are pretty substantial (e.g., a month backup can be in the high hundreds of megabytes).
- I’ll likely need a pretty large model (e.g. 70B parameter or larger) to produce results that are satisfying
Fortunately I should soon have access to a some moderately-beefy local compute (Mac Studio with a lot of RAM and high memory bandwidth). But there’s a lot of time/$ investment to get something like this set up and I want to make sure I’m barking up the right tree. Does anyone have any advice to share on this? For example:
- are my impressions above generally right? Is that type of architecture even possible?
- Am I likely to ever realistically get the type of insights I’m looking for at the end of this journey? E.g., I’d love to get a “morning report” highlighting things like “power usage was high yesterday because it looks like someone left the bathroom fan on” or “I’m noticing that particulate matter counts tend to get quite high throughout the upstairs whenever you sleep with the bedroom windows open”… and I’d also like to be able to interactively explore issues, e.g., asking it whether there is anything in the logs to indicate why a specific z-wave device is not talking to the network this morning, or identifying which devices might be wasting power.
- What does the interface look like between HA and the system running the LLM? Is this a case where I basically just grant the LLM read access to the Home Assistant history (how?) and then unbeknownst to HA there is an agent looking over its shoulder and I query this agent directly? Or do I somehow invoke the LLM from within Home Assistant? If so, how? Is the only way via some obnoxious microphone and voice assistant (I really hate those!) or can I open up a text chat or put the results of a standard prompt/query on a Lovelace dashboard?
- Is Ollama+Mac Studio using a large model likely the right technical setup for this, or should I be reconsidering? Does the application I’m describing really require such a large context window?
Again, I realize there’s a lot of subtlety to these questions, and I have a lot of digging and learning to do – I’m not expecting anyone to give me Ikea-style instructions for setting this up. I’d just rather confirm that I’m digging in the right place before investing too much!
Thanks!