Hi, I was able to get Ollama to connect to home assistant to serve a local LLM, but I’ve moved to using more llama.cpp for fine grain control. However, when I go to try to connect things together again, I haven’t found a way to connect a home assistant to my local llama server.
Has anyone been able to get to work? I see a thread that seems to use a specific llama.cpp python docker container. Was wondering if there is a way by just linking into the the llama-server? I’m able to get open webui to connect that way via a local lan ip address and port, but have tried using extended openai and custom conversations without any success…
Any help is appreciated!