@dumbdevice What type of hardware did you run the llama and mistral models on? And did you try running on the same computer where you run Home Assistant?
Asking to further help answer the questions posted above by @Psi-nz to explain that running an LLM locally on the same computer as Home Assistant OS will be very limited unless have better hardware than what Home Assistant Green and Home Assistant Yellow (Raspberry Pi 4/5) can offer.