Using custom voice models with Piper

pdoyle · December 2, 2023, 8:36pm

I followed the following video: https://youtu.be/67QvWOp3dkU?si=LT41AcpV4bCi3J0Y

After many hours I was able to generate a custom voice model. Assuming you are creating a voice called “voicename”, the video takes you through all of the steps and at the end you have two files like this:

en-us-voicename-high.onnx

en-us-voicename-high.onnx.json

Supposedly, from another post, if you place these files in your HA’s /share/piper directory and restart HA you should see your new voice as an option. I tried this and do not see voice name as an option. Has anyone been able to get this to work?

ignacio82 · December 16, 2023, 1:53am

I’m on the same boat. I installed the two files but I cannot get it to work:

This is what I see on the logs of my piper container

INFO:__main__:Ready
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-9' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio
INFO:wyoming_piper.download:Downloaded /data/en_US-hfc_male-medium.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/hfc_male/medium/en_US-hfc_male-medium.onnx.json)
INFO:wyoming_piper.download:Downloaded /data/en_US-hfc_male-medium.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/hfc_male/medium/en_US-hfc_male-medium.onnx)
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-23' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio
INFO:__main__:Ready
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-6' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio

So, piper cannot see the voice despite the fact that is in the right folder and I can actually see it in the drop-down menu:

Ideas?

pdoyle · December 17, 2023, 7:31pm

Elsewhere, someone said you have to reload Piper. I have not tried this yet however.

ignacio82 · December 17, 2023, 11:54pm

I did that and had no luck. The piper container is not happy with the files I added:

INFO:__main__:Ready
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-9' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio
INFO:wyoming_piper.download:Downloaded /data/en_US-hfc_male-medium.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/hfc_male/medium/en_US-hfc_male-medium.onnx.json)
INFO:wyoming_piper.download:Downloaded /data/en_US-hfc_male-medium.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/hfc_male/medium/en_US-hfc_male-medium.onnx)
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-23' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio
INFO:__main__:Ready
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-6' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.9/dist-packages/wyoming/server.py:28> exception=VoiceNotFoundError('ignacio')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/handler.py", line 73, in handle_event
    piper_proc = await self.process_manager.get_process(voice_name=voice_name)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/process.py", line 114, in get_process
    ensure_voice_exists(
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 77, in ensure_voice_exists
    find_voice(name, data_dirs)
  File "/usr/local/lib/python3.9/dist-packages/wyoming_piper/download.py", line 183, in find_voice
    raise VoiceNotFoundError(name)
wyoming_piper.download.VoiceNotFoundError: ignacio
INFO:__main__:Ready
INFO:__main__:Ready

ignacio82 · December 18, 2023, 3:29am

So, I managed to get it to work. I’m not sure if this is the right way of doing it, but given the lack of documentation and support, here’s what I did:

I copied my onnx.json and .onnx files to the same folder where all the other models are—specifically, the /data folder inside the container.
I modified the voices.json file by mapping the /usr/local/lib/python3.9/dist-packages/wyoming_piper from the container to my server. In that file, I added an entry for my new model, as follows:

    "ignaciov0": {
        "key": "ignaciov0",
        "name": "ignacio",
        "language": {
            "code": "en_US",
            "family": "en",
            "region": "US",
            "name_native": "English",
            "name_english": "English",
            "country_english": "United States"
        },
        "quality": "medium",
        "num_speakers": 1,
        "speaker_id_map": {},
        "files": {
            "data/ignaciov0.onnx": {
                "size_bytes": 63511038,
                "md5_digest": "146fe9c7382ec3111d2abb6b2d78da9a"
            },
            "data/ignaciov0.onnx.json": {
                "size_bytes": 7082,
                "md5_digest": "90eb2051cbe6e80cf7a5e1a8447082fd"
            },
            "data/MODEL_CARD": {
                "size_bytes": 320,
                "md5_digest": "24a2232470ca1be071debf53c984666e"
            }
        },
        "aliases": [
             "en-us-ignacio-medium"
        ]
    },

I believe the key here is to modify the file size and hash for both files, but I’m not entirely certain. I hope this helps others, and I hope that better documentation is provided at some point

GR86Drifter · March 14, 2024, 9:42pm

hi thanks for your help and i got a question i converted a pth file to onnx file but problem is there is no onnx.json file do you have a suggestion for that i want to use rcv models in my system

ai-Ev1lC0rP · July 16, 2024, 7:50pm

@ignacio82 , this was the First post that came up when I was searching for “custom voice models in home assistant”.

Did you ever come up with a solution to this?
I figured since it was the top result. If you DO have a solution. Others would appreciate it. Otherwise I’m looking for myself, if I figure it out I’ll post below.

ai-Ev1lC0rP · July 16, 2024, 8:18pm

good ole reddit.

https://www.reddit.com/r/homeassistant/comments/1dm74s7/comment/lb5cu4v/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

mylegitches · December 11, 2024, 7:19am

I had the same issue but resolved it.

It hit me while showering LOL. There were two things:
I offloaded all the heavy lifting to a dedicated AI machine, so it makes sense that the model files belong there, and not in /share/piper. DUH!
Then, the second part was that there is a language dropdown. I didn’t realize that when i had trained my model following the directions on Git, that the language would not end up as en-US, but English. This means you have to pull that dropdown to select English, which i never tried because I always assumed mine would be grouped by en-US alongside the other models.