I agree with this part wholeheartedly… I’m actually thinking of a feature request for Spook.boo for same because exact reason.
How do I handle it. Hmm well this part is at the later end of a loooooong chain…
see Friday's Party: Creating a Private, Agentic AI using Voice Assistant tools
if you’re interested. Cause entity state is only about 1/5th of your problem 
(working on smarter context not more) the index let’s you reduce exposed ents when there callable Info only so expose becomes live state v indexed state, not live state v gone… And it’s structured to boot. Exposed becomes more a decision… Do I need real-time active control? Than -do I need to read this for context.