Read the thread - this is an ongoing project.
First there is HA itself…- its not optimized for context, that’s where the index came from. this is step 1.
Second it’s not optimized for secure access. This is step 2 (SP1 will bring most of it and the plumbing is there now) - and will be done before I EVER look at tuning a model. (If you want to knock yourself out)
Third - HA sucks at providing a way for users to add written context. The cabinets solve that. (Yes, they can be in a DB in the future, thanks for asking. Redis probably, definitely a link to Obsidian vaults with a neo4j+Opensearch index)
The purpose of this is to reduce the context in an average home to something that can be consumed by a commodity tool use model in UNDER a 64K context)
As of this week I can consume the context from a 25000 entity richly instrumented HA install in under 64K. This includes enhanced controls such as hot tub chemistry and full alert and task monitoring. STRONGLY suspect I can get it under 48K and no rag implementation comes close to the capability.
WHY. Nathan…
Because most (vast majority of people) don’t know training a model is a thing or that specific models are required, or desired for certain tasks or that that those models have limits or things called quantization… Or the first thing about what’s REQUIRED for context in the first place. This does that part for you.
Because at those sizes I listed above, you can run a fully local commodity model that doesn’t HAVE to be tuned on a 16G card. And it’s even pretty snappy. (rn i just deal with a 1-2 second lag as the ctx/kv loads)
Why else, this way…
- If you choose to BYO tuned or superawesome model - it only makes it better. Use my tools your model, woot!
- If you choose to use someone’s plugin MCP - then the tools still work. IN FACT I plan on making them more secure than any other tool you can currently use on HA. See that caller token? Yes I plan to make the caller ID itself for tool perms.
- If you choose to do this then it standardizes the tool use format and gives the model some incredibly neat tricks… Index can tell you if you have an appointment if its tagged correctly.
- It also helps the model onboard understand the system, making it wayyyy easier to instruct said model without having a PHd. in machine learning
It can troll all the context sources you didnt know you had like label details.
Its the framework of what HA should have for users to be successful with the base - purpose built to drag context out of ha SECURELY then you plug in what you want from there.