For those folks lucky enough to have an Espressif Box3 (order one from AliExpress), you can now try Willow with the new Willow Add-on for HA. Note that this is just the Willow Application Server (WAS) which configures the STT service your Box3 uses and it send your spoken word to the Willow team’s cloud hosted, best-efforts Willow Inference Server (WIS) which then spits it back to HA in under 1 second. You do not need to install a local WIS Inference server for this to work.
NOTE: I am not affiliated w/ Willow (http://heywillow.io) in any way. I’m just an avg user who loves what these guys are doing in OpenSource Voice, and like the vast majority of users on these forums and Discord, I became hyper frustrated working w/ the HA Voice implementation. The slick demos look fantastic. Real-world use isn’t cutting it. Im an advanced, career audio electronics engineer. For several months now, I’ve tried the M5, ESPHome, an ESP32 on a breadboard w/ mics & amp/spkr, etc, etc and the results are terrible. No more.
Willow’s implementation of all this just works. And works really, really well. This should take you 15 minutes to get it working.
I currently run the local WIS server with an nVidia GTX 1070 GPU I bought for $75 on eBay and put it into an ancient Dell Optiplex from 2011 w/ 16GB of RAM running Ubuntu 20.04. It gives me highly accurate results in under 250ms. You probably don’t want to go this route. Yet… so just use this Add-On and test it out.
With the new Add-on, this will work on any HA system, but you do need a Espressif Box3 device. Any version is supported, but the Box3 is the one to get if you don’t have one.
First, make sure your Assistant Pipeline in HA is working. If not, this will never work. There are other guides to show you how to do that. Test it by turning a light on/off:
If you can’t get this basic HA Assist functionality working, stop now and fix it before proceeding. troubleshooting info on that is here ‘’‘Troubleshooting Assist - Home Assistant’‘’
Install the HA Willow add on by going to the add-on store, then adding this repository (https://github.com/nwithan8/hassio-addons) using the 3 dots in the upper right corner of your screen, then select Repositories
Paste the URL https://github.com/nwithan8/hassio-addons into the box and click ADD
Now select the Willow Add-on
Click download, wait a few min until it finishes, then click START. You should see this screen. Click Open Web UI:
You should now get the WAS server config screen. Most options should be filled in for you. Check the address of your HA server and port. Chose the Wake Word you want to use.
In HA, navigate to your username profile in the lower left of the HA interface, scroll all the way to the bottom and select Long Lived Tokens. Click create. Give it a name and copy the token.
Paste the token into the config screen where it says Home Assistant Token.
Click SAVE to save your settings.
Now click on Willow Web Flasher. NOTE: YOU MUST use Chrome or other browser that can access web serial ports so you can flash your Box.
Enter your wifi password. This will be used to flash your Box3. Click Connect and select the port your BOX 3 is plugged into on the PC you are now using. once you are connected to the right port, click Flash Willow. You should see this:
You can keep the log open to see how things work.
Once the Box reboots, it should then boot up,connect to your WiFi, and say “Welcome to Willow” on the screen. You should now be able to say “Hi {WakeWord}, turn {light} switch on”, where {WakeWord} is the word you chose during config and {light} is the name of a KNOWN light entity in HA. Test this out in Assist as I showed earlier if you see problems. You should see the responses and you should now be able to control all HA entities via voice. Accurately, reliably, in multiple languages and from across the room!
Helpful Tips:
ONLY Espressif Box devices work with Willow. This device was designed from the ground up to support voice applications. No, you cannot use regular ESPs, at least not yet.
Join their Discord channel if you have questions.
The only wake words that work are Alexa, Hi ESP, and Hi Lexin. Suffice to say, custom wake words are HIGHLY complex. You will soon see that these wake words work very, very well.
If you like what you see, check out their other stuff. If you want truly local and highly accurate TTS with Auto Correct (another REALLY cool piece of the puzzle), go dust off an old tower PC you can put a GPU into, buy a cheap nVidia GTX1070 on ebay and get rolling.
I have my local WIS server on my Linux box set up to forward unknown things I say to Amazon Alexa using Willow Auto Correct (GitHub - kovrom/willow-autocorrect: Willow Auto Correct). Setting my Box3 wake word to “Alexa” and renaming my Alexa device’s wake word to “Echo” you get this:
“Alexa turn on christmas lights” - Willow tells HA to turn on the lights
“Alexa set living room heater to 68 degrees” - Willow tells HA to set the thermostat
“Alexa, set a timer named Bread Rising for 2 hours” - HA has no clue how to do this so it gets forwarded to Alexa which sets a timer for 2 hrs
“Alexa how many ounces in a pound?” - again, HA has no clue, so Alexa answers.
“Alexa, play rock & roll radio on Pandora?” HA has no easy way to do this (no actual way on HA Supervised) on Pandora or Apple Music, so this gets sent to Amazon to play music.
Its wicked fast, accurate, self learning and auto-correcting. It learns how to do more and more things over time when you also install WAS - Willow AutoCorrect.
What I like (love) about this solution is Amazon no longer has ANY contact with my home devices and has no idea what I’m doing inside my home unless knowing what my named timers are doing and what music I listen to counts. And a a bonus, Amazon Alexa can be MUTED since the commands are sent to it programmatically. No more eavesdropping by those nosy engineers at Amazon. And as time goes on, fewer and fewer commands will be forwarded to Amazon as more intents are created and refined in HA.
If there’s interest in this deeper solution, I’ll write up a guide on how to do it. You’ll need to graduate to a local WIS server to start.
