The blog post for the 2024.04 release mentions an Ollama integration, but there are nö docs yet 2024.4 Beta: Organize all the things! - Home Assistant
That’s really good news, as ollama is really the best and easiest way to have local AI and its API is working perfectly if you test from command line:
curl http://192.168.0.100:11434/api/generate -d ‘{ “model”: “llama2:13b”, “prompt”: “What is water made of?” }’
We need a reliable integration in HA.
I’m using the Llama2:13b model and the responses are great via chat…but horrific for Home Assistant!
Any ideas what could be wrong?
Is there a better model to use?
I decided to see what happens sending it a command:
Do I need to modify the prompt template?
For context, Home Assistant’s conversation agent works fine:
I know it won’t help, but it’s basically the same for me. Tried a couple of different models, even llama like in the live stream, home3b (fixte) and mixtral 8x7b, but it was just garbage all over.
I’m not even sure if the prompt is working correctly since I didn’t find a way to check the output of the template.
i installed ollama in docker but I found out that my i3 2nd gen was too weak to run ai locally although I have p-1000 nvidia and 30 GB of ram. It can run some chatboot but slowly. Some other models is basically impossible to run it on my comp that i use as a server for ha.
That’s my 2 cents.
I was able to get a different integration to leverage LocalAI and a non-Meta / Llama model successfully but the performance was dreadful.
The performance issue was with LocalAI and that particular model…I just cannot find something else that works. I believe the issue is the prompting, but I have not been willing to modify it.
Is it possible to use the output of the Ollama conversation agent to be sent to a media player?
I have local stt and tts setup with Whisper and Piper and can create an automation to send tts to a media player.
Can the same be done with Ollama?
I’m guessing it’s not possible at this stage as it seems that Ollama cannot ‘control’ anything.
1 Like
I used this guide and have an LLM setup. The Text Generation Webui works in concert with the Llama Conversation Integration in HA. I tried to use the prompt from FutureProofHomes localai.io install with modest success . Text Generation Webui gives the opportunity to load different models to experiment with. I have only been at this for a day or so. If any one alse has installed this I woud love to see what model you installed, and what your prompt looks like. The default prompt in Llama Conversation is useless.
1 Like
Ollama says that it is controlling devices but the states do not actually change…which is WEIRD.
During my experimentation, I noticed some false reporting. Hallucinations I guess.
Any plans to include The new mistral?
https://www.reddit.com/r/LocalLLaMA/comments/1cy61iw/mistral7b_v03_has_been_released/
I have ollama running here, but what can I do with the HA integration? Only use the text interface which I can anyway use with ollama directly?
Is there a way to set up sentences that trigger automations and if no automation is registered then it sends the prompt to the ollama AI??
I’m in beta and for HA next release, you can now use Ollama for Local LLM control of HA, but i’m having a hard time getting it to work.
Anyone sucessful?
I conducted a test and it was successful. But there will be hallucinations, I use llma3.1:8b. Perhaps due to my use of Chinese, the success rate may be lower, as the more entities exposed, the slower the speed. When I use the qwen2 model and the old integration of ollama (which only supports queries), the success rate is high and the speed is also fast. The official ollama integration of HA prompted me that the qwen2 model does not support tool calls. So a suitable LLM model and accurate prompts will have a very good effect. But I didn’t conduct any testing
Did you figure it out? I am trying to use llama3.1 8b and set it up properly. But when asking it to turn on a light I get an error that tools are not supported
“Sorry, I had a problem talking to the Ollama server: llama3.1:latest does not support tools”
Unfortunately I don’t have that problem. I have the problem where you ask it a question and answer something completely different it doesn’t even know how to use any services.
Also for me the new control feature with ollama (llama3) is not working. A single unavailable device prevents the system to do anything even if its a totally different device you want to control. Also the language settings does not work. I configured it to use german, but it always answers in english.
If i remove all unavailable entities from assist, he tries to control a power switch instead of turning on the lights, what the request was.
Theres a PR to have context window configurable, which may improve the process: