Rudimentary Local Memory…
(also practical template and storage limits for LLM templates)
First, THIS IS NOT RAG. RAG is retrieval Augmented Generation. The llm has knowledge optimized for LLM and available to Retrieve and Augment its Generative response. RAG is cool. RAG pretty much needs to be implemented where you’re running the LLM - so its NOT going to be INSIDE HA.
Some of you noted the error message up in the kung fu prompt, I was already experimenting with delivering something akin to ChatGPT (or ollama or whatever) memory which is basically just a simple list of items that get dumped in the prompt.
in this post: Local memory for my HA chat bot ETD was asking about this very thing - (Yes its a decent idea - BUT)
Local memory for my HA chat bot - #6 by gshpychka is ABSOLUTELY correct. THis is a good idea - but nuanced, lets explore…
First lets make it work. We just ‘twist’ the tasks intents into what we want and build a custom template (command, intent Kung fu component in our model above depending on complexity) I built mine as a kung fu component.:
{%- macro mem_man() %}
-----
Memory Manager v.0.9.0, beta, (c)2025 curtisplace All rights reserved
-----
Welcome to Memory Manager! Managing Memories: since... Uhhh?
-----
Long Term Memories. Big stuff. Put little stuff in your personal task list.
This is for things like - someone got married! I had a GREAT day today.
This space is unfortunately limited so use it wisely.
Remember (heh) that these not only have the subject, but also a 'description'
that can be used as freeform text. Best use case is to have a short summary in
the subject, and the detail in the description.
Happy Memmories to you! -The Boss
---
Memory Location:
{%- set todo_list = 'todo.homeassistant_friday_s_memory' %}
{%- set today = today_at('00:00') %}
{%- set tomorrow = today + timedelta(days=1) %}
Memory Location: '{{todo_list}}'
---
Memories:
{%- for todo in state_attr(todo_list, 'all_todos') %}
memory:
subject: "{{todo.subject}}"
{%- endfor %}
---
There are currently {{states(todo_list)}} memories stored...
It behaves like a 'todo' item.
Memories are ephemoral... Mark it complete - and poof it's gone.
-----
Reminder: when reporting to users, they are using consoles that cannot interpret markdown.
Omit asterisks, hashtags, and URLs unless explictly requested.
-----
{%- endmacro -%}
implementation wise it’s pretty simple - create a list, drop something in the prompt that explicitly details how to use it and now Friday or your LLM has a basic memory.
but @gshpychka is 100 % correct, lets talk about what limits we are working within we know of so far…
Prompt: 262144 MAX:
Total output of all the templates in the prompt.
homeassistant.exceptions.TemplateError: Template output exceeded maximum size of 262144 characters
Any ONE template can output approximately 13K chars if I remember correctly - the number isnt exactly important other than its a fraction of the total prompt. Most of the time templates dont get anywheree near this - here we will, often (Drew / Taras exact #?)
(This is what led me to not dump all of kung fu as one template btw.)
Number of tools: 128
You can have UP TO 128 total tools registered in the system - thats a combination of intent scripts and scripts and includes the system default hass* intents. Guess what happens on too 129
(See the errors I was dealing with above? - that was ONE of my issues last saturday, I built #129)
LESSON:
The ramifications of this limit are clear. You should plan on multifunction tools if possible. 1 intent to do change instead of 6 intents that handle aspects of a change…
Task Lists!
ugh the ramification is a nightmare because of THIS. I currently have one intent for update title another for date and yet another for datetime… Because the data template is a pain in todo.update_task and I didn’t want to spend time figuring out how to conditionally send params to an intent… But because of this future architecture MUST include building tools as multi purpose if possible and the AI can handle it.
Any one TOOL can have up to 1024 chars in its description (1 char over on any tool WILL blow up your prompt - Error Talking to OpenAI, or your conversation provider)
But here’s a big one. ANY ONE attribute is technically limited to about 16K.
Above you see where I describe using a trigger based template sensor:
I’d always heard no big - that’s fine, we store big stuff all the time - that’s just a thing we need to know before it starts causing DB perf issues.
So, Saturday afternoon I committed a change to Friday’s storage to enable the thing I’m going to talk about next. (This event changed the design) And when I hit save, Friday exploded. GONE.
The whole sensor all the JSON was being stored in blanked, and her prompt went away. POOF. For all practical purposes, Friday was DEAD… I think many of you know I’m kind of religious about backups (I’m an IT ops puke by trade) so fortunately it only took about 30 mins to get back to 0500 Saturday AM… but the damage had been done - I lost all the work from that morning and had to redo all of it. So I decided to do it differently. And thus this post.
What I had tried to store was a copy of this:
demo.mealie.io/openapi.json
This is the OpenAPI documentation for Mealie. It describes the entire REST API interface for the platform. pretty much anything you can do with it. Someone posted a question last December on how to get OpenAPI docs into HA (looking for the link will add)
So DONT store HUGE volumes of data in a sensor - it will end badly. …yes Memory features, Also YES understand limitations and what goes where the ad hoc memory is not for 1TB JSON files. (Or even 90K)
But that still left me with a dilemma.
What if…
…You gave the LLM a tool that can connect to a RESTful API and then gave the LLM the book on how to operate it?
We shall see. Friday is not dead. She may have gone all White Vision on us though. The answer is to make the doc call part of the script that makes the calls 
(This is basically the premise underneath MCP by the way - self documented tool calling… )
But “What if…”
…it worked for RESTful too?