@NathanCu Thanks for mentioning my little project! 
I am currently working on packaging it as a HACS integration. Now, its being tested and a release will come soon. It will also be renamed to “Lookout”
Watch this space!
Thanks once again!
1 Like
Even Claude Fable 5 makes mistakes…
Making a voice assistant is out of my scope - I’ll elt someone else handle that Im intentionally staying away from the hardware aspect I consider myself an AI SME but there are much better resources for the hardware than I. (Hint see the special thanks section…)
That said, Start here
Voice Assistant - ESPHome - Smart Home Made Simple
And now you know why I wrote this - Yes it has that.
For the record they’re ALL the longest. I had to edit every chapter down to get under 32000 chars.
Whew! I just finished reading…Chapter 1. 
Learned a lot already and cleared up my misconceptions of using AI in HA. I look forward to reading all the chapters, eventually. Thanks, Nathan!
2 Likes
Ok, but where is ch. 10?
1 Like
how could you inject it before those comments?
I reserved all ten slots before you guys started reading, 
1 Like
So, will the MCP integration work without some stupid external server?
And H.O.W. should I use it with Claude Desktop?
Define - MCP IN MCP OUT? And stupid is relative.
Thanks for this! I hope everyone interested of bringing AI to HA reads this first. It’s a lot to take but worth it.
One question that I needed to find answer first was why do I need different agents? What is the “weakness” of AI that I cannot just give it all the instructions and then it understands everything, every command, every question. Well, the local models I use, are basically just puppies. The longer the command (=context), the more they fail to follow orders. So give one agent one personality, give it commands and let it shine when it’s performing as helpful assistant or DJ. It cannot be both.
1 Like
I read about the library and Weaviate. Problem with Weaviate was of course it’s ‘cloud-native’, so I quick-hacked a MCP server using Postgresql pgvector and embeddings from llama.cpp.
It will have bugs and I will update it. because I will be using this. Main usage will be device manuals. I asked my assistant “If I press manual button in Ikea Knycklan, for how long the vent stays open?” and it took three minutes to find answer and it was right. Right now that’s good enough for me. (I’m running Qwen3.8 Flash Next and have to update to latest llama.cpp to get pp 2x )
1 Like
Awesome you identified the deficiency and applied an appropriate update. (add rag)
Not the fastest but does the job.
Now tackle your speed issue. Find out how much context is being passed v how much it needs RIGHT NOW… (Any extra is speed loss)
What you’ve got now is basically a big ol data source so it takes a bit for your model to cut through and as you saw it finds.
There are other approaches which start to get into opinions. Let’s just say you re riding the reason why I did Friday and ZenOS-AI without traditional Vector RAG. I personally find structured Agentic RAG (the ability to reason through an index) way more effective and efficient than just vector. I mean don’t get me wrong it’s a Valid solution and a tuned vector RAG screams. But do you have time to tune it rn?
I am using that same integration for Fridays local inference stack and weaviate or some other vector source is in her future… But understanding the house is my current priority and that’s ARAG through custom. Data provider (ZenOS Monastery Library Index)
Looking forward to hear more! I’m glad the docs are helping.