Thank you for your answer. That’s exactly the kind of information I was looking for.
I was also curious to know if whisper add-on will run faster in a desktop machine. Since I did not have a linux server I had to install ubuntu on a VirtualBOX, exposed the virtual machine to my network, installed docker on ubuntu then downloaded whisper. I could easily integrate remote whisper to my HomeAssistant installed on a rpi4. All works as expected. So far all good.
When I check the debug logs in HomeAssistant it is clear that it does not work faster when whisper is installed in a remote, more powerful server.
So for my tests, using the model tiny_int8 takes 5-6 seconds to run in rpi4 and 4-5seconds in my desktop pc. Same using model BASE, it takes 11-12 seconds to run in whisper as addon installed on my rpi4 where also HA is installed, and around 9-10 seconds when using whisper installed as a docker on ubuntu on a VirtualBox on windows 10.
It could be that the whisper docker needs some tweaking to have a better performance and works faster when run in a powerful machine but I don’t know how that can be done or if it is even possible.
Bottom line my conclusion is, it is not worth running whisper in a remote powerful machine, especially for the moment being. Maybe in the future this will change.
2 Likes
Got my hands on some better hardware recently, works really well … in English, in Polish no matter what model I used, speech is barely recognizable, like 1 on 20 tries, with very slow and clear speech.
Well, need to wait little longer, maybe there will be some better solution or alternative in few months.
1 Like
I share the observation that the Whisper addon is slow regardless of hardware power. Even with 8 vCPUs and 8GB of RAM it takes about 2 seconds at least for it to process speech. The larger the model, the longer it takes. For me the best balance is with tiny. Larger models often even render completely nonsensical interpretations. I guess it’ll improve in upcoming updates.
1 Like
Would be great if it would support GPU, that would make it way faster. So I created an issue here: GPU support · Issue #3 · rhasspy/wyoming-addons · GitHub
1 Like
I haven’t had a chance to try it out yet, but I did find this for NVIDIA GPUs; Home Assistant - Enabling CUDA GPU support for Wyoming Whisper Docker container | tarball.ca
1 Like
I created a PR that is a WIP: [WIP] Add initial GPU support by edurenye · Pull Request #4 · rhasspy/wyoming-addons · GitHub
2 Likes
Using n5105 for proxmox running home assistant and openwrt. HA has most resources and struggles with anything more than base (which has a terrible error rate for Polish). small-int8 barely works (simple command takes over a minute to process), medium-int8 crashes (with 12GB RAM assigned). I’m looking into doing the same (putting whisper on external server, I have one with Ryzen 3900x;))
Just wanted to share my experience with getting Whisper to run on hardware other than a Pi 4 8GB, hoping it will help others who also want Assist to be responsive.
I couldn’t get satisfactory response times on the Pi (using the tiny-int8 model), making it unusable for other members of the house (“it’s way slower than Goggle/Alexa, why would I use it?”)
So I picked up a HP EliteDesk 800 G1 SFF PC second hand with an Intel(R) Core™ i5-4570 CPU @ 3.20GHz. I maxed out the RAM to 32GB (4x8GB) & installed a ASUS Geforce GTX 1650 OC 4GB GPU which was short enough to fit & came with a SFF bracket. Loaded it with Ubuntu 22.04.3 LTS.
I followed this guide to get it installed. Running Whisper on the CPU made a noticeable difference & I was even able to use the small-int8 model which was definitely more accurate at STT than tiny & base.
I followed this guide to get my system ready for CUDA GPU support in docker. It took quite a bit of research, trial & error but I eventually managed to get it all working.
Running Whisper on the GPU made it instantaneous & in my opinion as fast or faster than Google/Alexa. My family are now happy to use it.
Using the medium-int8 model didn’t make a noticeable delay but has improved the accuracy.
The icing on the cake is being able to use it on our phones as the default Assistant.
Now to install a multizone audio system with ceiling speakers & ESP32 mics in each room while awaiting Assist wake word support… 
12 Likes
Thx a lot for this. It’s highly appreciated. I’ll give it a try.
Hi Everyone,
I am trying to install the Whisper Docker to be able to use the new Voice Assist.
I am using this docker compose:
whisper:
container_name: whisper
image: homeassistant/amd64-addon-whisper:latest
restart: unless-stopped
ports:
- "10300:10300"
command: ["--model", "tiny-int8", "--language", "en"]
volumes:
- ./wyoming:/data
But I am receiving error of curl: (6) Could not resolve host: supervisor
’
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service whisper: starting
s6-rc: info: service whisper successfully started
s6-rc: info: service discovery: starting
curl: (6) Could not resolve host: supervisor
[20:55:15] ERROR: Something went wrong contacting the API
[20:55:17] INFO: Service exited with code 256 (by signal 4)
’
I already tried to add
‘network_mode: host didn’t work’
and
’ networks:
internal:
ipv4_address: xxx.xxx.xxx.xxx (my HA IP)’
But still the same error.
Do you guys have some suggestion?
Where did you get that compose file?
I am curious as the image url looks unfamiliar to me.
I am running docker on an x86 based system and my url is definitely not amd64.
If you are in an rpi then the amd64 is definitely wrong as rpi is an arm system.
Here is my “stack” for my x86 server running everything docker.
# 2023-05-07
version: "3"
services:
## wyoming includes whisper and piper
whisper: # stt
container_name: whisper
command: --model small --language en
#command: --model medium --language en
image: rhasspy/wyoming-whisper:latest
volumes:
- /home/pcwii/docker/whisper:/data
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10301:10300
# Piper Text To Speech
piper: # tts
container_name: piper
image: rhasspy/wyoming-piper:latest
command: --voice en-us-libritts-high
volumes:
- /home/pcwii/docker/piper:/data
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10201:10200
# Open Wake Word Dection
openwakeword:
container_name: openwakeword
image: rhasspy/wyoming-openwakeword
command:
--preload-model 'kelsey'
--custom-model-dir /custom
--model 'kelsey'
--uri 'tcp://0.0.0.0:10400'
volumes:
- /home/pcwii/docker/openwakeword-data:/data
- /home/pcwii/docker/openwakeword-data:/custom
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10400:10400
It sounds like you are using regular Docker/Compose, but that is the add-on container for a HAOS installation?
Another option for a stack to run on x86 shown below… All of these containers should have Arm/RPi versions.
All containers are in a single stack, and they communicate via an internal docker network (hass). The only exposed container is Home-Assistant, using HTTPS with a Let’s Encrypt certificate. You might want to skip some of the networking part, as I am using a MACVLAN network for… reasons…
I am also running the containers as a non-root user (1001), though I had to change that for HA recently… don’t remember why right now… I need to revisit that.
version: '3.9'
###
networks:
hass:
dockermacvlan:
external: true
###
services:
homeassistant:
container_name: Home-Assistant
image: ghcr.io/home-assistant/home-assistant:stable
# user: 1001:1001
networks:
dockermacvlan:
ipv4_address: 192.168.xxx.xxx
hass:
volumes:
- ./config:/config
- /docker/certbot/certs/live/example.com/fullchain.pem:/ssl/fullchain.pem:ro
- /docker/certbot/certs/live/example.com/privkey.pem:/ssl/privkey.pem:ro
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
- TZ=Australia/Melbourne
restart: always
##
whisper:
container_name: Whisper
image: rhasspy/wyoming-whisper
user: 1001:1001
networks:
- hass
volumes:
- ./whisper:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
- TZ=Australia/Melbourne
command: --model small-int8 --language en
restart: unless-stopped
##
piper:
container_name: Piper
image: rhasspy/wyoming-piper
user: 1001:1001
networks:
- hass
volumes:
- ./piper:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
- TZ=Australia/Melbourne
command: --voice en-us-ryan-medium
restart: unless-stopped
##
openwakeword:
container_name: openwakeword
image: rhasspy/wyoming-openwakeword
user: 1001:1001
networks:
- hass
volumes:
- ./wakeword:/data
- ./wakeword:/custom
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
- TZ=Australia/Melbourne
command: --preload-model 'ok_nabu' --custom-model-dir /custom
restart: unless-stopped
Git repo with further details here, but I have not updated it with the changes I made yesterday… 
Cheers.
Hi pcwii,
I am quite sure I tried with this image you have in your composer too. Anyways, I just delete mine and trying with yours.
So, now the error is different:
/run.sh: line 5: 7 Illegal instruction (core dumped) python3 -m wyoming_faster_whisper --uri ‘tcp://0.0.0.0:10300’ --data-dir /data --download-dir /data “$@”
/run.sh: line 5: 7 Illegal instruction (core dumped) python3 -m wyoming_faster_whisper --uri ‘tcp://0.0.0.0:10300’ --data-dir /data --download-dir /data “$@”
/run.sh: line 5: 7 Illegal instruction (core dumped) python3 -m wyoming_faster_whisper --uri ‘tcp://0.0.0.0:10300’ --data-dir /data --download-dir /data “$@”
This is my composer now:
whisper:
container_name: whisper
command: --model small --language en
image: rhasspy/wyoming-whisper:latest
volumes:
- /opt/whisper/data:/data
environment:
- TZ=America/New_York
restart: unless-stopped
ports:
- 10301:10300
I have a DELL Intel running Ubuntu Linix …
Thank you for trying to help. Do you have some other suggestion ?
Which version of Ubuntu do you have? I remember reading something about the latest version having an issue with HA. I can’t remember where I saw it.
I m using Ubuntu 22.0.43
Hi Fraddles,
No, not the add on. I don’t have hassos. I have a Linux Box with Ubuntu and Docker Portainer.
I did read your GitHub … which SO do you user? Linux too?
Hi, Do you know the difference between cloud nabu casa and whisper on the external server?
I tried (Slovak Language) both - whisper and the cloud solution, cloud solution is working perfectly, it understand very well, but whisper on the external server with a medium language model does not understand correctly Slovak language.
1 Like
Looks like this is the command the docker is running
python3 -m wyoming faster_whisper --uri ‘tcp://0.0.0.0:10300’ --data-dir /data --download-dir /data “$@”
And that is causing the error of
Illegal instruction
I will try to find the Wyoming Python project …. If you have any idea please 
give me some help
I am using Debian at the moment…