HA on esxi vm with Voice Assistant

I’m new. I’ve been reading, watching YT and looking here and I’ve found a few tutorials on setup. But overall, none seem to get me where I want to go.

I want a locally run “Hey ”. I have a full esxi with other virtual machines so I figured why not? I downloaded the ova and got it installed.

Next, I want to try to get the voice working. That’s the big sticking point right now that I need someone to point me to a good direction.

I don’t have any smart devices in the house and wasn’t planning on putting any just yet but probably will when I move in the next year or so. For now, I just want my own, locally run/controlled, “Hey Whoever”. I was going to setup the voice part on a Pi 5. Have a USB mic, bluetooth speaker and have done some testing to verify that hardware works. So how do I get a voice assistant setup on there and then integrate it into HA?

Also, once that’s there, will it even work or do I still need something else? Should I just use the Preview edition? Isn’t that hooked to the internet though? I’ve read on a lot of articles/blogs about a locally run LLM but no articles/tutorials on what it is (other than a language modle) and how to build one locally.

I’ve clearly missed some things in my reading/research so would appreciate the community pointing out what I missed and give me the links for additional reading/learning.

I was told - just from reading online - this was a pretty active project. Did I post this question in the wrong place? No replies in 4 days? I wouldn’t think this is that hard to do and that I’m just missing something simple. So I’m not asking to fix the issue just point me in a direction to get the information to fix it myself.