Assist AI conversation agent running locally


I would like to share that it is possible to host your own conversation agent on for example a different pc or VM locally with your own API and connecting it with homeassistant.

I did this with the help of a homeassistant HACS intergration url:
HACS intergration AI

The software iam running for the AI is called "text generation webui url:

you can download the 1 click windows installer zip file and run “start_windows.bat”.
once its finished it will start the server, close it and add the following lines in CMD_FLAG.txt assuming you use cpu only: --cpu --listen --openai

then start server again and add your server IPADRESS:5001/v1 in homeassistant HACS intergration you just added no need to fill in a API key just leave it.

Last thing you want to do is go to IPADRESS:7860 your webui and go to model tab and download for example: bigscience/bloomz-560m then load the model and add the same bigscience/bloomz-560m in homeassistant voice assist as conversation agent name.

hopefully this will help anyone willing to try out a own conversation agent.
Ofcource the better server the better results of AI conversation.




The documentation is very minimal so I have a hard time understanding what it does and if I will be able to figure out how to set it up. This is a shame because it looks like a fun addition many users would enjoy.

Could you start a topic under “community guides” so other people can extend the installation instructions?