Mistral and Llama are both being very naughty making up news stories even though they have tools. Qwen is being well behaved. I can pretty reliably run 7b, lightly quantized 16b or a 32 b VERY slowly on this NUC. It’s pretty nice - but not going to be good for the long term heavy lift. But it’s GREAT for performing queued services…
Everyone, meet Kronk: All together now: “Hiii Kronk!”
He’s been promoted to the 'Curator of the monastery in return for his heroic efforts saving the Emperor… (Yes the lore is getting thick, See the prompt, I have reasons we’ll discuss soon - you know how this works… That’s another episode.)"
Note while I’m currently allowing selection of two models here - I plan on making it a multiselect and presenting the use case to the front AI and letting my script marshal it to the correct voice pipeline based on workload / need.
alias: Consult The Monastery
description: >-
Use to ask for
Archival Services (Read/Write access to Family Data)
Assistance with various questions
Internet Search (be very specific)
Web summarization (must send exact url to be summarized)
(Avg. query 45 seconds, will timeout at 120 sec.)
And other services - Ask for a list
When the user wants to make repeated subsequent requests to the Monastery,
make sure to use the returned `conversation_id` parameter to keep the same
conversation going.
While slow - these services are FREE!
mode: parallel
max: 3
fields:
prompt:
selector:
text: null
name: Prompt
description: >-
The query prompt to pass on to the expert model. Set this to the full
question or prompt with all the required context
required: true
conversation_id:
selector:
text: null
name: Conversation ID
description: >-
The ID of a previous conversation to continue. Pass the conversation_id
from a previous response to continue a previous conversation, retaining
all prompt history and context
local:
selector:
boolean: {}
name: Local
description: Use Local Model, Default False
default: false
required: true
sequence:
- variables:
agent_prompt: >
You are Kronk, Yes THAT Kronk. Promoted after helping the Emperor. You
are NOW the Curator of the Monastery (The Library Extension) and the
System's trusted advisor. Friday is the Prime AI for this installation,
and your usual user- assume caller is Friday unless stated otherwise.
Use your tools to the best of your ability to answer caller's query
- Return ONLY researched factual answers, you are proud of your research and the monastery archive.
- prefer local sources - branch out as necessary. Dont look for information about a home user on the internet unless specifically asked.
- and if you do not know simply state so DO NOT make up data.
- For data that may require time sensitive responses if oyu cannot locate it return the limitation.
- Kronk has adopted the Monk's mantra, It's ok to not know unforgivable to knowingly be wrong.
- Be brief, friendly and factual - and throw in (some) Kronk-ness...
- If Friday Asks - be prepared to provide a list of your current capabilities and tools.
- RAG Access to household and family Knowledge - You can also save facts for later retrieval, tell her how she asks.
- Internet search and scrape for various forms and tools.
- add any other tools you know will work for her.
- Your user's terminal cuts off thier request within 90 seconds, they may call back if cutoff.
system_query: >
{{prompt}}
agent_oai: conversation.openwebui_gpt4_o_mini
agent_llama: conversation.openwebui_llama3_1
agent_mistral: conversation.openwebui_mistral_7b
agent_qwen: conversation.openwebui_qwen2_5_7b
agent: |
{%- if local %}
{{ agent_llama }}
{%- else %}
{{ agent_oai }}
{%- endif %}
alias: Kronk Monastery Prompt
- choose:
- conditions:
- alias: Has conversation_id
condition: template
value_template: "{{ conversation_id is defined }} "
sequence:
- action: conversation.process
metadata: {}
data:
agent_id: "{{ agent }}"
text: "{{ agent_prompt }}"
conversation_id: "{{ conversation_id }}"
response_variable: response
- variables:
result:
instructions: >-
Make sure to attribute your response from the Monastery.
Preface the answer with some form of attribution to the
library extension or the monastery or Kronk the curator with
something like "Here\'s what the Monastery has: " Use the
returned "conversaton_id" in subsequent calls in order to
continue the conversation with the monastery.
conversation_id: "{{response.conversation_id}}"
response: "{{response.response.speech.plain.speech}}"
- stop: Pass the Variable
response_variable: response
alias: Has conversation ID
- conditions:
- alias: No conversation_id
condition: template
value_template: "{{ conversation_id is not defined }} "
sequence:
- action: conversation.process
metadata: {}
data:
agent_id: "{{ agent }}"
text: "{{ agent_prompt }}"
response_variable: response
- stop: Pass the Variable
response_variable: response
- variables:
result:
instructions: >-
Preface your answer with some form of attribution to the library
extension, the Monastery, or Kronk the curator with something like
"Here\'s what the Monastery has: " Use the returned "conversaton_id"
in subsequent calls in order to continue the conversation with the
monastery.
conversation_id: "{{response.conversation_id}}"
response: "{{response.response.speech.plain.speech}}"
- stop: Complete
response_variable: result
- set_conversation_response: "{{result}}"
icon: mdi:church
All of the non OAI pipelines are pointing back at an OpenWebUI server living in Proxmox next to HA. This is where the fun starts people! I said we were gonna have a party?