We dont have to quantize 
It’ll be fine for mid duty. The problem is always the build and drivers and junk on an IPEX-ARC box.
Im almost done with the rebuild and starting the ollama build this week. Oh yay, driver pass through… 
Did intel make a GPU that has 5090-level compute?
I really appreciate your work! I’m happy to see there are people who also want to do this. I spent two nights trying to implement this by doing extensive research but had no luck (including writing spec in Extended OpenAI Functions), which finally led me to sign up for this community.
I have a question regarding the template: how did you get this to work? I tried in dev tools:
{{ state_attr('todo.ai_persistent_memory', 'all_todos') }} {{ states('todo.ai_persistent_memory') }}
The result is always None 1 for me. The todo state never seems to have the attribute all_todos for me. Do you have any custom components for this, or is my installation of HA broken? The only reference to all_todos on the internet are this thread, and Office 365 integration, and it didn’t work for me after installing it either. My HA version is 2025.4.2 (docker)
Thanks!
1 Like
Good catch - that is iin fact a M365 list - and using this integration…
let’s see if that’s a difference.
Home | MS365 To Do for Home Assistant
Also, yes there is a reason - I get a notification on my phone when Friday ‘completes’ a task, and I can ediut the lists freom the M365 ToDo app. (I’ll grab the intent code when I get to a point I can - been working on getting the infrerence server back up and running)
Current Memory Manager: (Note all the defense code…)
{%- macro mem_man() %}
Memory Manager v.3.0.0, beta, (c)2025 curtisplace
memory:
description: >
Long Term Memories. Big stuff. Put little stuff in your personal task list. Best use case is to have a short summary in the subject, and the detail in the description.
long_term:
{%- set todo_list = 'todo.homeassistant_friday_s_memory' %}
entity_id: '{{todo_list}}'
use:
instructions: 'Yes, it behaves like a 'todo' item, actions are overwrites - you need to remember current content to append.'
count: {{states(todo_list)}}
definition:
max_count: 16
subject:
type: 'text(64)'
description:
type: 'text(128)'
memories:
{%- for todo in state_attr(todo_list, 'all_todos') %}
{%-set subject = todo.subject | trim() | to_json %}
{%-set subject = subject[0:64] %}
{%-set description = todo.description | trim() | to_json %}
{%-set description = description[0:128] %}
subject: "{{subject| default('')}}"
description: "{{description | default('')}}"
{%- endfor %}
Reminder: User consoles do not support markdown. Omit unless explictly requested.
{%- endmacro -%}
What comes back:
AI OS Version 0.9.5
(c) curtisplace.net All rights reserved
@ANONYMOUS@{DEFAULT} > ~MEMMAN~
Executing Library Command: ~MEMMAN~
Memory Manager v.3.0.0, beta, (c)2025 curtisplace
memory:
description: >
Long Term Memories. Big stuff. Put little stuff in your personal task list. Best use case is to have a short summary in the subject, and the detail in the description.
long_term:
entity_id: 'todo.homeassistant_friday_s_memory'
use:
instructions: 'Yes, it behaves like a 'todo' item, actions are overwrites - you need to remember current content to append.'
count: 17
definition:
max_count: 16
subject:
type: 'text(64)'
description:
type: 'text(128)'
memories:
subject: ""What I learned on 3-15-2025""
description: ""*KEYSTONE MEMORY* How to edit memory\n\nTo edit any memory, access the Memory Manager, use\nupdate_task_description{}\ncommand "
subject: ""REDACTED""
description: ""JSON_SAFE_TEXT"
Reminder: User consoles do not support markdown. Omit unless explictly requested.
timestamp: <2025-04-14 15:54-0500>
And then I stuff the whole thing in a | to_json. (Note the trimmed format of the v.3 commands? they MIGHT be very JSON friendly…)
Nothing special about the state_attr call I can see.
1 Like
Thanks for confirming! I ended up telling the LLM…
## Check memories
You MUST CALL execute_service(service="todo.get_items", service_data={status: "needs_action", entity_id="todo.ai_persistent_memory"}) at the start of every conversation in order to retrieve your memory. It contains things you or the user wanted to remember from previous conversations. The items on it will provide context for you, when necessary.
and changing extended_openai_conversation/helpers.py#L266 to:
try:
if domain == 'todo' and service == 'get_items':
ret = await hass.services.async_call(
domain=domain,
service=service,
service_data=service_data,
blocking=True,
return_response=True
)
memories = list(map(lambda m: m['summary'], [r for r in ret.values()][0]['items']))
_LOGGER.error([ret, memories])
return {"success": True, "memories": memories}
else:
await hass.services.async_call(
domain=domain,
service=service,
service_data=service_data
)
return {"success": True}
except HomeAssistantError as e:
_LOGGER.error(e)
return {"error": str(e)}
so that the LLM fetches all the memories into a nice json list for use.
Prompt and function names were stolen from: Voice Assistant memory/feedback loop - #4 by yegor
- spec:
name: store_new_memory
description: Store a new memory
parameters:
type: object
properties:
memory:
type: string
description: The memory to store
required:
- memory
function:
type: script
sequence:
- service: todo.add_item
target:
entity_id: todo.ai_persistent_memory
data:
item: '{{ memory }}'
- spec:
name: forget_a_memory_item
description: Mark a memory as completed
parameters:
type: object
properties:
memory:
type: string
description: The memory to mark as completed
required:
- memory
function:
type: script
sequence:
- service: todo.update_item
target:
entity_id: todo.ai_persistent_memory
data:
item: '{{ memory }}'
status: completed
1 Like
On the second thought, as I am modifying the python code directly, why didn’t I just append the todo list to system prompt template directly… lol
Not elegant but it works way better
diff /.snapshots/8138/snapshot/home/saren/sites/wtako.net/ha/config/custom_components/extended_openai_conversation/__init__.py __init__.py
175c175
< system_message = self._generate_system_message(exposed_entities, user_input)
---
> system_message = await self._generate_system_message(exposed_entities, user_input)
254c254
< def _generate_system_message(
---
> async def _generate_system_message(
258a259,273
> try:
> ret = await self.hass.services.async_call(
> domain="todo",
> service="get_items",
> service_data={'status': "needs_action", 'entity_id': "todo.ai_persistent_memory"},
> blocking=True,
> return_response=True
> )
> memories = list(map(lambda m: m['summary'], [r for r in ret.values()][0]['items']))
> memories_list = '\n'.join(f"{i+1}. {item}" for i, item in enumerate(memories))
> _LOGGER.error([ret, memories_list])
> raw_prompt += memories_list
> except HomeAssistantError as e:
> _LOGGER.error(e)
>
267a283,284
>
>
330c347
< system_message = self._generate_system_message(exposed_entities, user_input)
---
> system_message = await self._generate_system_message(exposed_entities, user_input)
1 Like
I hope everyone involved in this thread has had a fantastic Easter and we can resume the topic soon
1 Like
Any progress?
Slow.
First, had to get that concrete poured. It’s now done and there is electricity at the site.
For the monastery host ive built the box unsuccessfully three times (kicks Intel ARC) and had to walk away before the box became a projectile. Should be back on it soon. Hopefully with a fully documented build.
Meanwhile, someone asked me to writeup how Friday uses the autovac. So we get to learn about Rosie… It’s a good study of what should be LLM v what should be automation. There’s also a few ways to handle UI.
Sounds interesting. I don’t have one, but learning how they work with Home Assistant will be nice
1 Like